WebSocket API Reference

MuxitServer exposes a WebSocket server for browser clients. All messages are JSON objects with a type field.

Connection

ws://localhost:8765/ws
wss://localhost:8765/ws  (when HTTPS enabled)

Default port is 8765, configurable in workspace/config/server.json.

WebSocket Authentication

The auth token is sent via the Sec-WebSocket-Protocol header to avoid token exposure in server/proxy access logs:

javascript

// Browser client — token sent as subprotocol
const ws = new WebSocket("ws://localhost:8765/ws", ["muxit", "auth-YOUR_TOKEN"]);

The server extracts the token from any subprotocol prefixed with auth-, and accepts the muxit subprotocol. Query string ?token= is still supported for backward compatibility with non-browser clients, but is not recommended.

Authentication

Local Access (Loopback)

The UI auto-fetches the auth token via GET /api/auth/token (loopback-only endpoint). No user interaction needed.

Remote Access

When security.remoteAccess is enabled and a password is set:

GET /api/auth/token returns 403 for non-loopback requests
GET /api/auth/status returns { requiresLogin: true, ... }
POST /api/auth/login with { "password": "..." } returns { "token": "<auth-token>", "sessionToken": "<session>" }
Use the auth token for WebSocket connections (via subprotocol header) and HTTP API calls (X-Auth-Token header)

Password Requirements

Passwords must meet all of the following:

At least 8 characters
At least one uppercase letter
At least one lowercase letter
At least one digit

HTTP API Endpoints

Endpoint	Method	Auth	Description
`/api/auth/token`	GET	Loopback only	Get startup auth token
`/api/auth/status`	GET	None	Check auth requirements
`/api/auth/login`	POST	None	Authenticate with password
`/api/auth/password`	POST	Local or authenticated	Set/update access password
`/api/auth/password`	DELETE	Local only	Remove access password

Rate Limiting

Message Format

Request (client → server)

json

{
  "type": "message.type",
  "requestId": "optional-correlation-id",
  ...
}

Response (server → client)

json

{
  "type": "response.type",
  "requestId": "echoed-if-provided",
  "value": ...
}

Error

json

{
  "type": "error",
  "code": "CONN_NOT_FOUND",
  "requestId": "echoed-if-provided",
  "message": "Human-readable error description"
}

The code field is a structured error code (UPPER_SNAKE_CASE) for programmatic error handling. See Error Codes for the full list.

Error Codes

All error responses include a code field from the table below. Use these codes for programmatic error handling and when reporting issues.

Protocol Errors

Code	Description
`PROTOCOL_INVALID_JSON`	Message could not be parsed as JSON
`PROTOCOL_MISSING_TYPE`	Message has no `type` field
`PROTOCOL_UNKNOWN_TYPE`	Message `type` does not match any handler

Validation Errors

Code	Description
`VALIDATION_MISSING_PARAM`	A required parameter is missing from the request
`VALIDATION_INVALID_VALUE`	A parameter has an invalid value

Connector Errors

Code	Description
`CONN_NOT_FOUND`	The specified connector does not exist
`CONN_NOT_INITIALIZED`	The connector exists but has not been initialized
`CONN_LOAD_FAILED`	The connector config file failed to parse or load
`CONN_CALL_FAILED`	A method call on the connector failed
`CONN_DISABLED`	The connector is disabled
`CONN_LIMIT_REACHED`	License tier connector limit exceeded
`CONN_RELOAD_FAILED`	Hot-reload of connectors failed

Driver Errors

Code	Description
`DRIVER_NOT_FOUND`	The specified driver does not exist in the registry
`DRIVER_LOAD_FAILED`	The driver DLL/assembly failed to load
`DRIVER_BLOCKED`	The driver requires a license that is not active

Script Errors

Code	Description
`SCRIPT_NOT_FOUND`	The script file was not found
`SCRIPT_ALREADY_RUNNING`	A script with this name is already running
`SCRIPT_EXEC_FAILED`	The script threw an error during V8 execution
`SCRIPT_LIMIT_REACHED`	License tier concurrent script limit exceeded

Prompt Errors

Code	Description
`PROMPT_TIMEOUT`	`ask.*` timed out with no `default` configured
`PROMPT_NOT_FOUND`	`prompts.answer` referenced an id that is no longer open

Agent Errors

Code	Description
`AGENT_NOT_FOUND`	The agent config or instance was not found
`AGENT_LIMIT_REACHED`	Maximum concurrent agents limit exceeded
`AGENT_SAFETY_BLOCKED`	The safety gate blocked the action (rate, workspace, speed, force)
`AGENT_DEVICE_MISSING`	A required device for the agent is not available
`AGENT_AI_NOT_CONFIGURED`	No AI provider configured for the agent
`AGENT_START_FAILED`	The agent failed to start

License Errors

Code	Description
`LICENSE_LIMIT`	A generic license tier limit was exceeded
`LICENSE_ACTIVATION_FAILED`	License key activation failed
`LICENSE_EXPIRED`	The license subscription has expired
`LICENSE_DEACTIVATION_FAILED`	License deactivation failed

Auth Errors

Code	Description
`AUTH_RATE_LIMITED`	Too many login attempts — try again later
`AUTH_INVALID_SESSION`	The session token is invalid or expired
`AUTH_INVALID_PASSWORD`	The provided password is incorrect

Config Errors

Code	Description
`CONFIG_READ_FAILED`	Failed to read a configuration file
`CONFIG_WRITE_FAILED`	Failed to write a configuration file
`CONFIG_INVALID`	The configuration data is invalid

AI Errors

Code	Description
`AI_NOT_CONFIGURED`	No active Muxit license for AI
`AI_REQUEST_FAILED`	The AI/LLM API request failed

Marketplace Errors

Code	Description
`MARKETPLACE_INSTALL_FAILED`	Driver package install failed (download, extract, or validation error)
`MARKETPLACE_UNINSTALL_FAILED`	Driver uninstall failed

Internal Errors

Code	Description
`INTERNAL_ERROR`	An unexpected internal error occurred

Connector Messages

`connectors.list`

List all loaded connector names.

json

// Request
{ "type": "connectors.list", "requestId": "1" }
// Response
{ "type": "connectors.list", "requestId": "1", "value": ["psu", "robot", "panel"] }

`connector.schema`

Get the schema for a single connector.

Pass "includeHidden": true to receive items that are filtered out by hide/expose — they come back in the same arrays with a "hidden": true flag so the GUI can render them greyed out in a "show hidden" view.

json

// Request
{ "type": "connector.schema", "requestId": "2", "name": "psu", "includeHidden": false }
// Response
{
  "type": "connector.schema", "requestId": "2",
  "value": {
    "name": "psu", "driver": "MockInstrument",
    "properties": [
      { "name": "voltage", "type": "number", "access": "rw", "unit": "V", "hidden": false }
    ],
    "actions": [
      { "name": "reset", "description": "Reset to defaults", "hidden": false }
    ],
    "streams": [ { "name": "scpi", "hidden": false } ],
    "driverConfig": [
      { "key": "serialPort", "value": "COM8", "type": "string", "editable": true },
      { "key": "baudRate",   "value": 9600,   "type": "number", "editable": true },
      { "key": "transport",  "value": "{…}",  "type": "object", "editable": false }
    ],
    "hiddenItems": [],
    "visibilityMode": "all"
  }
}

driverConfig mirrors the currently-merged driver config as key/value pairs. Framework-injected internals (anything whose key starts with _, e.g. _workspacePath, _eventBus, _licenseManager) are omitted from this list. Only entries with "editable": true (primitive string / number / bool / null) can be changed through connector.driverConfig.set. hiddenItems lists the names currently hidden for the connector (both driver and custom items); combine it with includeHidden: true to render a visibility-editor UI.

`connectors.schema`

Get schemas for all connectors at once (only enabled/loaded connectors).

json

// Request
{ "type": "connectors.schema", "requestId": "3" }
// Response
{ "type": "connectors.schema", "requestId": "3", "value": { "psu": {...}, "robot": {...} } }

`connectors.all`

Get metadata for ALL discovered connectors (enabled + disabled).

json

// Request
{ "type": "connectors.all", "requestId": "4" }
// Response
{
  "type": "connectors.all", "requestId": "4",
  "value": {
    "enabled": [
      { "name": "test-device", "driver": "TestDevice", "isTestDevice": true },
      { "name": "bk-psu", "driver": "GenericScpi", "isTestDevice": false }
    ],
    "disabled": [
      { "name": "webcam", "driver": "Webcam", "isTestDevice": false, "reason": "Not enabled (limit: 3 connectors)" }
    ],
    "maxConnectors": 3,
    "enabledNames": ["test-device", "bk-psu"],
    "enabledNonTestCount": 1,
    "tierName": "Free"
  }
}

`connectors.set_enabled`

Update which connectors are enabled. Validates against license limits.

json

// Request
{ "type": "connectors.set_enabled", "requestId": "5", "connectors": ["bk-psu", "camera", "robot"] }
// Response (success)
{ "type": "connectors.set_enabled", "requestId": "5", "success": true, "requiresRestart": true }

`connectors.reload`

Hot-reload all connectors without server restart.

json

{ "type": "connectors.reload", "requestId": "6" }

`connector.call`

Call a property (get/set) or action on a connector.

json

// Read property
{ "type": "connector.call", "requestId": "4", "connector": "psu", "property": "voltage" }

// Write property
{ "type": "connector.call", "requestId": "5", "connector": "psu", "property": "voltage", "args": [12] }

// Execute action
{ "type": "connector.call", "requestId": "6", "connector": "psu", "method": "reset" }

// Response
{ "type": "connector.response", "requestId": "4", "value": 12.0 }

`connector.visibility.set`

Rewrite the top-level hide: [ ... ] array in the connector's .js source and hot-reload just that connector. The loader honours the new visibility on the very next AI turn — no server restart is required. Only .js connectors are supported; .json configs have no visibility block.

json

// Request — make "debugChannel" and "internalCalibrate" invisible to AI + UI
{
  "type": "connector.visibility.set", "requestId": "10",
  "connector": "psu",
  "hidden": ["debugChannel", "internalCalibrate"]
}
// Response
{ "type": "connector.visibility.set", "requestId": "10", "connector": "psu", "success": true }

Pass "hidden": [] to clear the list (everything visible). An existing top-level expose: [ ... ] whitelist is removed when hidden is non-empty, so the file stays consistent with the loader's "hide XOR expose" invariant.

`connector.driverConfig.set`

Surgically edit a single primitive driver-config value (e.g. a serial port, baud rate, or IP address) in the connector's .js source and hot-reload the connector. The key is looked up in both the top-level config keys and the nested config: { ... } block; whichever side already holds the key is updated in place. New keys are appended to config: { ... } (creating it if absent).

json

// Request — change the BK-PSU serial port from COM8 to COM4
{
  "type": "connector.driverConfig.set", "requestId": "11",
  "connector": "bk-psu", "key": "serialPort", "value": "COM4"
}
// Response
{ "type": "connector.driverConfig.set", "requestId": "11",
  "connector": "bk-psu", "key": "serialPort", "success": true }

value must be a JSON primitive (string, number, boolean, or null). If the existing value is a non-primitive expression (identifier, computed value, template literal with interpolation), the server responds with a CONN_RELOAD_FAILED error and leaves the file untouched — the user is expected to edit the file directly in that case.

Driver Messages

`drivers.list`

json

{ "type": "drivers.list", "requestId": "8" }
// Response
{ "type": "drivers.list", "requestId": "8", "value": [
  {
    "name": "TestDevice", "tier": 0, "version": "1.0.0", "category": "built-in", "group": "utilities",
    "errors": [], "warnings": []
  },
  {
    "name": "Fairino", "tier": 3, "version": "2.0.0", "category": "premium", "group": "motion",
    "errors": [
      {
        "ruleId": "PREMIUM_NOT_SIGNED",
        "severity": "hard",
        "message": "Premium driver 'Fairino.dll' is not signed with the official muxit key. Premium drivers will refuse to initialize."
      }
    ],
    "warnings": []
  }
]}

errors[] and warnings[] are produced by the runtime driver validator. Each entry has ruleId (uppercase snake-case rule identifier), severity ("hard" or "soft"), and a human-readable message. A non-empty errors[] means the driver is registered for visibility but will refuse to initialize. The same fields are returned by driver.schema and marketplace.detail.

`driver.schema`

Get the full API schema for a driver, including all properties, actions with argument details, streams, and the connector template.

json

{ "type": "driver.schema", "requestId": "9", "name": "TestDevice" }
// Response
{ "type": "driver.schema", "requestId": "9", "name": "TestDevice", "value": {
  "name": "TestDevice",
  "version": "1.0.0",
  "description": "Simulated test device for development...",
  "category": "built-in",
  "tier": 0,
  "properties": [
    { "name": "temperature", "type": "double", "access": "R/W", "unit": "°C", "description": "Current temperature" }
  ],
  "actions": [
    { "name": "setThreshold", "description": "Set the alert threshold", "args": [
      { "name": "value", "type": "double", "description": "Threshold value (0-100)" }
    ]}
  ],
  "streams": ["data"],
  "connectorTemplate": "export default { ... }"
}}

`driver.template`

Get the connector config template for a driver.

json

{ "type": "driver.template", "requestId": "10", "name": "TestDevice" }

Stream Messages

`stream.subscribe` / `stream.unsubscribe`

json

{ "type": "stream.subscribe", "connector": "spectrometer", "stream": "spectrum" }
{ "type": "stream.unsubscribe", "connector": "spectrometer", "stream": "spectrum" }

`stream.data` (server → client)

json

{
  "type": "stream.data",
  "name": "spectrometer", "stream": "spectrum",
  "data": { "wavelengths": [380, 381, ...], "intensities": [0.1, 0.2, ...] }
}

For image streams (webcam), data is a base64-encoded JPEG string.

Audio streams

Drivers that produce audible output (e.g. AudioSynth, future TTS connectors) declare a stream named "audio". Each stream.data message carries a JSON string in the data field. The default codec is Opus (32 kbit/s VBR, 20 ms frames at 48 kHz mono); base64 PCM16 remains a recognised fallback for emitters that can't link an Opus encoder.

json

// Default — Opus packet, ~20 ms of audio.
{
  "op": "chunk",
  "data": "<base64-encoded opus packet>",
  "sampleRate": 48000,
  "channels": 1,
  "format": "opus",
  "frameSize": 960
}

// Legacy fallback — base64 PCM16, ~50 ms of audio.
{
  "op": "chunk",
  "pcm": "<base64-encoded little-endian PCM16 samples>",
  "sampleRate": 44100,
  "channels": 1,
  "format": "pcm16"
}

// Stop signal — drop any queued/in-flight audio for this connector.
// Sent when the driver's playback is cancelled before completion.
{ "op": "stop" }

Chunks are paced at real-time playback rate so a subscribed client can schedule them back-to-back without running its jitter buffer dry. Different connectors that both emit audio streams play independently and may overlap on the same client. Dashboards consume the chunks via the built-in ServerAudioRenderer, which decodes Opus via the browser-native WebCodecs AudioDecoder and schedules the decoded AudioBuffers through the Web Audio API.

Wire-format upgrade path

Sustained audio drops from ~1 Mbit/s (base64 PCM16) to ~100–200 kbit/s with JSON-wrapped Opus. If audio bandwidth ever becomes a bottleneck again — or when other stream types (video, large file transfers) want to skip the base64 envelope — the next step is a binary WebSocket frame variant. The JSON wrapper format stays forward-compatible: clients should ignore unknown format values rather than assume any specific codec. See docs/audio-streaming.md in the repo for the migration plan.

`driver.diagnostics` (event)

Server → client broadcast emitted during driver scan when a driver has validation findings (errors or warnings). Lets the Extensions panel show a badge without polling. The same payload is also embedded in drivers.list, driver.schema, and marketplace.detail responses.

json

{
  "type": "driver.diagnostics",
  "driver": "Fairino",
  "category": "premium",
  "errors": [
    { "ruleId": "PREMIUM_NOT_SIGNED", "severity": "hard", "message": "..." }
  ],
  "warnings": []
}

See the Driver Validation reference for the rule list.

State Messages

`state.subscribe`

Subscribe to reactive state updates. Immediately receives current state, then state.batch on changes.

json

{ "type": "state.subscribe" }
// Immediate response
{ "type": "state.batch", "updates": { "psu": { "voltage": 12 }, "robot": { "position": [100, 200, 300] } } }

`state.snapshot`

One-shot state read without ongoing subscription.

json

{ "type": "state.snapshot", "requestId": "20" }

`state.batch` (server → client)

Pushed every 50ms when polled property values change (delta broadcasting).

json

{ "type": "state.batch", "updates": { "psu": { "voltage": 12.1, "power": 6.05 } } }

Script Messages

Startup scripts placed in workspace/startup/ run automatically when the server starts. They appear in script lists with a startup/ prefix (e.g., startup/monitor) and can be stopped like any other script.

`scripts.list`

json

{ "type": "scripts.list", "requestId": "12" }
// Response — includes startup scripts with "startup/" prefix
{ "type": "scripts.list", "requestId": "12", "value": ["hello", "monitor", "startup/log-temps"] }

`scripts.start`

json

// By name (from workspace/scripts/)
{ "type": "scripts.start", "requestId": "13", "name": "my-script" }
// Startup script (from workspace/startup/) — prefix with "startup/"
{ "type": "scripts.start", "requestId": "13", "name": "startup/log-temps" }
// With inline code
{ "type": "scripts.start", "requestId": "13", "name": "my-script", "code": "log.info('Hello!')" }

`scripts.execute`

Execute synchronously — waits for completion and returns output.

json

{ "type": "scripts.execute", "requestId": "15", "name": "my-script", "code": "log.info('Hello!'); 42" }
// Response
{
  "type": "scripts.result", "requestId": "15", "name": "my-script",
  "result": 42,
  "logs": [{ "level": "info", "args": ["Hello!"] }],
  "error": null
}

`scripts.stop`

json

{ "type": "scripts.stop", "requestId": "14", "name": "my-script" }

`scripts.stopAll`

Emergency stop — cancels every running script. Used by the always-visible status strip in the web UI.

json

{ "type": "scripts.stopAll", "requestId": "15" }

Response includes the list of names that were running at the time of the call:

json

{ "type": "scripts.stopAll", "requestId": "15", "stopped": ["foo", "bar"], "count": 2 }

`script.say` (broadcast)

json

{ "type": "script.say", "script": "temp-monitor", "text": "Temperature is 25.3 degrees", "emotion": null }

emotion is null unless the script passed an opts.emotion argument to say() (e.g. "excited", "sad"). The dashboard's TTS controller forwards the hint to the active provider; the built-in browser provider approximates with rate/pitch tweaks, future server-side providers (chatterbox, cosyvoice, ElevenLabs) will honour it natively.

`script.done` (broadcast)

Emitted once per script when it finishes — successfully or otherwise. The error field is either null (clean completion) or a structured object with user-relative line/column (the wrapper offset is already applied) so the dashboard can drop a Monaco marker straight onto the offending row.

json

// Clean completion
{ "type": "script.done", "script": "monitor", "error": null }

// Failure — line/column refer to the script source as the user wrote it
{
  "type": "script.done",
  "script": "monitor",
  "error": {
    "message": "ReferenceError: foo is not defined",
    "line": 7,
    "column": 1,
    "stack": "ReferenceError: foo is not defined\n    at <anonymous>:7:1",
    "details": "ReferenceError: foo is not defined\n    at <anonymous>:7:1\n..."
  }
}

`script.line` (broadcast)

Phase-2 execution-line trace. Emitted before every blocking host call inside a running script (delay, await connector.x(), ai, ask.*, say, stream), throttled to one event per 50ms per script and de-duplicated against the prior line. Dashboards opt in via script.line.subscribe so the per-call cost of capturing a JS stack is paid only when at least one client cares.

json

// Subscribe (refcounted server-side)
{ "type": "script.line.subscribe", "requestId": "20" }
// Unsubscribe — last subscriber off ⇒ server skips the trace
{ "type": "script.line.unsubscribe", "requestId": "21" }
// Broadcast while subscribed
{ "type": "script.line", "script": "monitor", "line": 12, "column": 3 }

`connector.error` (broadcast)

Emitted when a connector config fails to parse or load. The dashboard uses line / column to place a Monaco error marker on the offending row of the source .js (line numbers are preserved through import-stripping in the loader).

json

{
  "type": "connector.error",
  "connector": "psu",
  "file": "/abs/path/to/connectors/psu.js",
  "message": "SyntaxError: Unexpected token '}'",
  "line": 14,
  "column": 1
}

Prompt Messages

Interactive prompts opened by a script via ask.confirm / ask.choose / ask.text. Dashboards subscribe, render each open prompt, and send the user's answer back. Lifecycle broadcasts (prompt.open, prompt.resolved) fan out to every connected client so a prompt stays visible across reconnects.

`prompts.subscribe` / `prompts.unsubscribe`

json

{ "type": "prompts.subscribe", "requestId": "30" }

Subscribing matters for scripts that set requireObserver: true — the server counts subscribed clients and pauses the prompt timeout while that count is zero.

`prompts.list`

Snapshot of prompts currently open on the server (useful when a dashboard reconnects mid-prompt).

json

{ "type": "prompts.list", "requestId": "31" }
// Response
{
  "type": "prompts.list",
  "prompts": [
    {
      "id": "9f2c0b5a4e11",
      "script": "calibrate",
      "kind": "confirm",
      "message": "Home all axes before run?",
      "openedAt": "2026-04-15T09:12:34.123Z",
      "requireObserver": false,
      "timeoutMs": 30000,
      "hasDefault": true
    }
  ]
}

`prompts.answer`

Deliver a user-supplied answer. The value shape must match the prompt kind: boolean for confirm, one of the offered strings for choose, any string for text. Mismatches are rejected with VALIDATION_INVALID_VALUE.

json

{ "type": "prompts.answer", "requestId": "32", "id": "9f2c0b5a4e11", "value": true }

`prompt.open` (broadcast)

json

{
  "type": "prompt.open",
  "id": "9f2c0b5a4e11",
  "script": "calibrate",
  "kind": "choose",
  "message": "Select instrument port",
  "choices": ["COM1", "COM2", "COM3"],
  "openedAt": "2026-04-15T09:12:34.123Z",
  "requireObserver": true,
  "timeoutMs": 120000,
  "hasDefault": true
}

`prompt.resolved` (broadcast)

Sent when a prompt leaves the open set (answered, timed out, or its script ended). answeredBy is "user" or "timeout".

json

{ "type": "prompt.resolved", "id": "9f2c0b5a4e11", "answeredBy": "user" }

Agent Messages

`agent.list`

json

{ "type": "agent.list", "requestId": "..." }
// Response
{
  "type": "agent.list",
  "configs": [{ "name": "pick-and-place", "description": "...", "devices": ["robot", "camera"], "autonomy": "supervised" }],
  "running": [{ "agentId": "abc123", "name": "pick-and-place", "goal": "...", "status": "executing" }]
}

`agent.start`

json

{ "type": "agent.start", "name": "pick-and-place", "goal": "Pick up the red part", "parameters": { "partColor": "red" }, "autonomy": "supervised" }
// Response
{ "type": "agent.started", "agentId": "abc123", "name": "pick-and-place", "goal": "Pick up the red part" }

`agent.stop` / `agent.pause` / `agent.resume`

json

{ "type": "agent.stop", "agentId": "abc123" }
{ "type": "agent.pause", "agentId": "abc123" }
{ "type": "agent.resume", "agentId": "abc123" }

`agent.approve` / `agent.deny`

json

{ "type": "agent.approve", "agentId": "abc123", "stepId": "step-id" }
{ "type": "agent.deny", "agentId": "abc123", "stepId": "step-id" }

`agent.status` / `agent.detail`

json

{ "type": "agent.status", "agentId": "abc123" }
// Returns: { agentId, name, goal, status, iteration, planSteps, reasoning, tokensUsed, elapsedSeconds, ... }

{ "type": "agent.detail", "agentId": "abc123" }

agent.status response fields: agentId, name, goal, status, iteration, planSteps, reasoning, tokensUsed, elapsedSeconds, error, parameters, timelineCount.

agent.detail response includes all status fields plus: timeline (last 200 entries), devices, autonomy, processFile.

`agent.state` (broadcast)

json

{
  "type": "agent.state",
  "data": {
    "agentId": "abc123def456",
    "name": "pick-and-place",
    "goal": "Pick up the red part",
    "status": "executing",
    "iteration": 5,
    "reasoning": "I can see the red part at position (320, 240). Moving to approach...",
    "tokensUsed": 1500,
    "elapsedSeconds": 45.2,
    "planSteps": [
      { "id": "a1b2c3", "description": "Moving to pick position", "status": "executing" },
      { "id": "d4e5f6", "description": "Close gripper", "status": "pending" }
    ]
  }
}

`agent.timeline` (broadcast)

Streamed in real-time as the agent executes. Types: thought, action, observation, error, user_input.

json

{
  "type": "agent.timeline",
  "agentId": "abc123def456",
  "entry": {
    "timestamp": "2025-01-15T10:30:00Z",
    "type": "action",
    "summary": "call_action({connector: \"robot\", action: \"moveJ\", ...})"
  }
}

`instructions.get` / `instructions.set`

json

{ "type": "instructions.get", "requestId": "40" }
// Returns: { type: "instructions.get", content: "# Lab Instructions\n..." }

{ "type": "instructions.set", "requestId": "41", "content": "# Lab Instructions\n## General\n..." }
// Returns: { type: "instructions.set", saved: true }

Config Messages

`config.get` / `config.set`

json

{ "type": "config.get", "requestId": "30" }

// Dot-path update
{ "type": "config.set", "requestId": "31", "path": "ai.provider", "value": "claude" }
// Patch update
{ "type": "config.set", "requestId": "31", "patch": { "ai": { "model": "anthropic/claude-sonnet-4-5" } } }

`config.test_provider`

Probe a (possibly unsaved) LLM provider configuration. Used by the AI Services settings panel to validate Ollama / LM Studio is reachable before persisting. Returns success / latency / model count.

json

{ "type": "config.test_provider", "requestId": "31a", "provider": "ollama",
  "baseUrl": "http://localhost:11434/v1", "model": "llama3.2" }

json

{ "type": "config.test_provider", "value": {
    "success": true, "latency": 12, "model": "llama3.2",
    "provider": "ollama", "modelCount": 4,
    "models": [{ "id": "llama3.2", "name": "llama3.2", "provider": "ollama" }]
} }

`models.list`

Fetch available AI models (cached server-side, 6h TTL). Merges in models from the active local provider's /models endpoint when the active provider is Ollama / LM Studio / openai-compatible.

json

{ "type": "models.list", "requestId": "32" }
{ "type": "models.list", "requestId": "33", "force": true }

Response contains an array of models with pricing:

json

{
  "type": "models.list",
  "value": {
    "models": [
      {
        "id": "anthropic/claude-sonnet-4-5",
        "name": "Claude Sonnet 4.5",
        "provider": "anthropic",
        "contextLength": 200000,
        "pricing": { "prompt": "0.000003", "completion": "0.000015" }
      }
    ]
  }
}

AI Messages

Client → Server

Type	Fields	Description
`ai.chat`	`sessionId`, `message`, `ttsEnabled?`	Send user message, triggers agentic loop
`ai.stop`	`sessionId`	Cancel the active chat for that session
`ai.contextNotice`	`sessionId`, `notice`	Drop a mid-chat context boundary into the session (e.g. TTS toggled). Appends a user→assistant pair so the LLM adopts the new rules on its next turn. No-op when the session has no messages yet. Response: `{ applied: bool }`
`ai.tool_approve`	`requestId`	Approve a pending tool call
`ai.tool_deny`	`requestId`	Deny a pending tool call

Server → Client (Streamed)

Type	Fields	Description
`ai.delta`	`content`, `sessionId`	Streaming text chunk
`ai.tool_call`	`tool`, `input`, `result`	Tool executed
`ai.tool_pending`	`requestId`, `tool`, `input`	Tool awaiting approval
`ai.image`	`tool`, `image`, `sessionId`	Camera snapshot captured
`ai.chat`	`message`, `toolCalls`	Final response

Vision Annotation Messages

Direct vision training — draw bounding boxes on a camera feed to teach objects without AI.

`vision.teach`

Teach an object by sampling color in a drawn bounding box region.

Request:

json

{
  "type": "vision.teach",
  "name": "red_cup",
  "x": 100,
  "y": 80,
  "width": 60,
  "height": 50,
  "visionConnector": "vision",
  "description": "A red coffee cup",
  "trackerType": "color"
}

Field	Required	Description
`name`	Yes	Unique name for the object
`x`, `y`	Yes	Top-left corner of bounding box (image pixels)
`width`, `height`	Yes	Box dimensions (image pixels)
`visionConnector`	No	Vision connector name (default: `"vision"`)
`description`	No	Human-readable description
`trackerType`	No	`"color"` (default) or `"contour"`

Response:

json

{ "type": "vision.teach", "success": true, "name": "red_cup", "trackerType": "color", "camera": "webcam" }

`vision.forget`

Remove a taught object and its tracker.

Request:

json

{ "type": "vision.forget", "name": "red_cup", "visionConnector": "vision" }

Response:

json

{ "type": "vision.forget", "success": true, "name": "red_cup" }

`vision.list`

List all taught object profiles.

Request:

json

{ "type": "vision.list" }

Response:

json

{
  "type": "vision.list",
  "objects": [
    { "name": "red_cup", "description": "A red coffee cup", "trackerType": "color", "camera": "webcam", "createdAt": "2026-04-08T..." }
  ]
}

License Messages

`license.get`

Returns current license state including tier, limits, trials, and usage.

Response:

json

{
  "type": "license.get",
  "value": {
    "tierId": "free",
    "tierName": "Free",
    "maxDrivers": 2,
    "maxConnectors": 3,
    "maxScripts": 2,
    "baseLicenseKey": null,
    "baseValid": false,
    "baseExpiresAt": null,
    "baseLastValidated": null,
    "isBaseTrialActive": false,
    "baseTrialDaysRemaining": 14,
    "customerName": null,
    "customerEmail": null,
    "trialDays": 14,
    "entitlements": {},
    "driverTrials": {},
    "allTiers": [
      { "id": "free", "name": "Free", "maxDrivers": 2, "maxConnectors": 3, "maxScripts": 2 },
      { "id": "pro", "name": "Pro", "maxDrivers": -1, "maxConnectors": -1, "maxScripts": -1 }
    ],
    "currentConnectors": 1,
    "currentDrivers": 2,
    "currentScripts": 0
  }
}

`license.activate`

Activate a license key. For base subscriptions, driver entitlements included as subscription add-ons are automatically synced from Lemon Squeezy — no separate driver keys needed. Standalone per-driver keys are also supported as a fallback.

json

{ "type": "license.activate", "licenseKey": "XXXX-XXXX-XXXX-XXXX" }

Response:

json

{
  "type": "license.activate",
  "success": true,
  "value": { "...full license state..." }
}

On failure, success is false and error contains a descriptive message:

Situation	Error message explains
Key already activated on max instances	How many instances are used, and how to deactivate one
Key expired	That renewal is needed via the Lemon Squeezy portal
Key inactive / not yet purchased	That the purchase needs to be completed
Key disabled	That support should be contacted
Network error	That Lemon Squeezy could not be reached (local network issue)
Unknown product	That the key doesn't match this application

`license.deactivate`

Deactivate the base subscription on this machine. Frees up the activation slot so the key can be used on another machine. Reverts to the free tier.

json

{ "type": "license.deactivate" }

`license.deactivate_driver`

Deactivate a standalone per-driver license. Subscription-managed driver entitlements cannot be deactivated individually — manage them from the Lemon Squeezy subscription portal instead.

json

{ "type": "license.deactivate_driver", "driverId": "my-driver" }

`license.start_trial`

Start a per-driver trial (14 days). The driverId field is required.

json

{ "type": "license.start_trial", "driverId": "my-driver" }

Note: Base Pro trials require a license key. Obtain a free trial key from the Muxit store and use license.activate instead.

`license.changed` (broadcast)

Broadcast to all clients whenever license state changes.

Server Log Messages

`server.logs.history`

Fetch buffered log history (last 500 entries).

`server.log` (broadcast)

Real-time server log entry.

json

{ "type": "server.log", "level": "info", "source": "connectors", "message": "Connector 'psu' initialized", "time": "..." }

`server.initializing` (broadcast)

Sent when the server begins background initialization (driver scanning, connector loading). The GUI is already accessible at this point — devices will appear progressively.

json

{ "type": "server.initializing", "phase": "drivers" }
{ "type": "server.initializing", "phase": "connectors" }

`server.ready` (broadcast)

Sent once all connectors have been initialized and startup scripts have launched.

json

{ "type": "server.ready" }

`connectors.changed` (broadcast)

Sent after each connector finishes initializing (or fails). Clients should re-fetch the connector list (connectors.schema) to get updated data.

json

{ "type": "connectors.changed" }

`drivers.changed` (broadcast)

Sent after a marketplace install or uninstall completes and the driver registry has re-scanned. Clients should re-fetch the driver list (drivers.list) so the Extensions panel's Installed tab updates without needing a reconnect.

json

{ "type": "drivers.changed" }

MQTT Broker Messages

`mqtt.broker.status`

Get the status of the built-in MQTT broker.

Response:

json

{
  "type": "mqtt.broker.status",
  "value": {
    "running": true,
    "enabled": true,
    "port": 1883,
    "clientCount": 3,
    "maxClients": 100,
    "hasAuth": false
  }
}

`mqtt.broker.clients`

List currently connected MQTT clients.

Response:

json

{
  "type": "mqtt.broker.clients",
  "value": [
    { "clientId": "esp32-sensor-01", "connectedAt": "2026-03-29T10:15:00Z" },
    { "clientId": "muxit-abc123", "connectedAt": "2026-03-29T10:16:00Z" }
  ]
}

Update Messages

`update.check`

Check for available updates. Returns current and latest version info.

Response:

json

{
  "type": "update.status",
  "value": {
    "currentVersion": "0.1.0",
    "latestVersion": "0.2.0",
    "updateAvailable": true,
    "downloadUrl": "https://github.com/muxit-io/muxit/releases/download/v0.2.0/muxit-win-x64-v0.2.0.zip"
  }
}

`update.available` (broadcast)

Emitted on startup when a newer version is available.

json

{ "type": "update.available", "current": "0.1.0", "latest": "0.2.0", "downloadUrl": "..." }

Marketplace Messages

`marketplace.search`

Search the driver registry. All filter parameters are optional.

json

{ "type": "marketplace.search", "query": "serial", "group": "communication", "tier": 1, "requestId": "..." }

Response:

json

{
  "type": "marketplace.search",
  "value": [
    {
      "id": "community/serial-logger",
      "name": "Serial Logger",
      "version": "1.0.0",
      "description": "Logs serial port data to file",
      "author": { "name": "Jane", "github": "jane" },
      "group": "communication",
      "tier": 1,
      "category": "free",
      "tags": ["serial", "logging"],
      "installed": false,
      "installedVersion": null
    }
  ]
}

category is either "free" or "premium" and reflects the manifest declaration for the registry entry. The Extensions panel uses this to render a Premium badge on the Available tab.

`marketplace.detail`

Get detailed information about a driver (works for registry drivers, marketplace-installed drivers, and locally loaded drivers).

json

{ "type": "marketplace.detail", "id": "TestDevice", "requestId": "..." }

Response:

json

{
  "type": "marketplace.detail",
  "value": {
    "id": "TestDevice",
    "name": "TestDevice",
    "version": "1.0.0",
    "description": "Simulated test instrument",
    "group": "utilities",
    "tier": 0,
    "category": "built-in",
    "installed": true,
    "schema": {
      "properties": [{ "name": "voltage", "type": "double", "access": "R/W", "unit": "V" }],
      "actions": [{ "name": "reset", "description": "Reset to defaults" }],
      "streams": [],
      "connectorTemplate": "..."
    },
    "readme": "...",
    "changelog": "..."
  }
}

`marketplace.install`

Install a driver from the registry by ID, or from a local .muxdriver file. Runs as a background task with progress messages.

json

{ "type": "marketplace.install", "id": "community/serial-logger", "requestId": "..." }

Pin a specific version (optional — defaults to the highest-versioned entry for the id):

json

{ "type": "marketplace.install", "id": "community/serial-logger", "version": "1.2.0", "requestId": "..." }

Or from a local file:

json

{ "type": "marketplace.install", "filePath": "/path/to/driver.muxdriver", "requestId": "..." }

Progress messages (sent during download/install):

json

{ "type": "marketplace.install.progress", "driverId": "community/serial-logger", "phase": "downloading", "percent": 45 }
{ "type": "marketplace.install.progress", "driverId": "community/serial-logger", "phase": "complete", "percent": 100 }

Response:

json

{ "type": "marketplace.install", "value": { "id": "community/serial-logger", "name": "Serial Logger", "version": "1.0.0", "success": true } }

`marketplace.uninstall`

Remove a marketplace-installed driver.

json

{ "type": "marketplace.uninstall", "id": "community/serial-logger", "requestId": "..." }

`marketplace.installed`

List all marketplace-installed drivers (does not include built-in or locally-built drivers).

json

{ "type": "marketplace.installed", "requestId": "..." }

`marketplace.updates`

Check for available updates to installed marketplace drivers.

json

{ "type": "marketplace.updates", "requestId": "..." }

Response:

json

{
  "type": "marketplace.updates",
  "value": [
    { "id": "community/serial-logger", "installedVersion": "1.0.0", "latestVersion": "1.1.0" }
  ]
}

`marketplace.refresh`

Force re-fetch the registry index (bypasses 24-hour cache).

json

{ "type": "marketplace.refresh", "requestId": "..." }

Example: JavaScript Client

javascript

const ws = new WebSocket("ws://localhost:8765/ws", ["muxit", "auth-YOUR_TOKEN"]);

ws.onopen = () => {
  ws.send(JSON.stringify({ type: "state.subscribe" }));
  ws.send(JSON.stringify({ type: "connector.call", requestId: "1", connector: "psu", property: "voltage" }));
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  if (msg.type === "state.batch") console.log("State:", msg.updates);
  else if (msg.type === "connector.response") console.log(`[${msg.requestId}]:`, msg.value);
};

WebSocket API Reference ​

Connection ​

WebSocket Authentication ​

Authentication ​

Local Access (Loopback) ​

Remote Access ​

Password Requirements ​

HTTP API Endpoints ​

Rate Limiting ​

Message Format ​

Request (client → server) ​

Response (server → client) ​

Error ​

Error Codes ​

Protocol Errors ​

Validation Errors ​

Connector Errors ​

Driver Errors ​

Script Errors ​

Prompt Errors ​

Agent Errors ​

License Errors ​

Auth Errors ​

Config Errors ​

AI Errors ​

Marketplace Errors ​

Internal Errors ​

Connector Messages ​

connectors.list ​

connector.schema ​

connectors.schema ​

connectors.all ​

connectors.set_enabled ​

connectors.reload ​

connector.call ​

connector.visibility.set ​

connector.driverConfig.set ​

Driver Messages ​

drivers.list ​

driver.schema ​

driver.template ​

Stream Messages ​

stream.subscribe / stream.unsubscribe ​

stream.data (server → client) ​

Audio streams ​

driver.diagnostics (event) ​

State Messages ​

state.subscribe ​

state.snapshot ​

state.batch (server → client) ​

Script Messages ​

scripts.list ​

scripts.start ​

scripts.execute ​

scripts.stop ​

scripts.stopAll ​

script.say (broadcast) ​

script.done (broadcast) ​

script.line (broadcast) ​

connector.error (broadcast) ​

Prompt Messages ​

prompts.subscribe / prompts.unsubscribe ​

prompts.list ​

prompts.answer ​

prompt.open (broadcast) ​

prompt.resolved (broadcast) ​

Agent Messages ​

agent.list ​

agent.start ​

agent.stop / agent.pause / agent.resume ​

agent.approve / agent.deny ​

agent.status / agent.detail ​

agent.state (broadcast) ​

agent.timeline (broadcast) ​

instructions.get / instructions.set ​

Config Messages ​

config.get / config.set ​

config.test_provider ​

models.list ​

AI Messages ​

WebSocket API Reference

Connection

WebSocket Authentication

Authentication

Local Access (Loopback)

Remote Access

Password Requirements

HTTP API Endpoints

Rate Limiting

Message Format

Request (client → server)

Response (server → client)

Error

Error Codes

Protocol Errors

Validation Errors

Connector Errors

Driver Errors

Script Errors

Prompt Errors

Agent Errors

License Errors

Auth Errors

Config Errors

AI Errors

Marketplace Errors

Internal Errors

Connector Messages

`connectors.list`

`connector.schema`

`connectors.schema`

`connectors.all`

`connectors.set_enabled`

`connectors.reload`

`connector.call`

`connector.visibility.set`

`connector.driverConfig.set`

Driver Messages

`drivers.list`

`driver.schema`

`driver.template`

Stream Messages

`stream.subscribe` / `stream.unsubscribe`

`stream.data` (server → client)

Audio streams

`driver.diagnostics` (event)

State Messages

`state.subscribe`

`state.snapshot`

`state.batch` (server → client)

Script Messages

`scripts.list`

`scripts.start`

`scripts.execute`

`scripts.stop`

`scripts.stopAll`

`script.say` (broadcast)

`script.done` (broadcast)

`script.line` (broadcast)

`connector.error` (broadcast)

Prompt Messages

`prompts.subscribe` / `prompts.unsubscribe`

`prompts.list`

`prompts.answer`

`prompt.open` (broadcast)

`prompt.resolved` (broadcast)

Agent Messages

`agent.list`

`agent.start`

`agent.stop` / `agent.pause` / `agent.resume`

`agent.approve` / `agent.deny`

`agent.status` / `agent.detail`

`agent.state` (broadcast)

`agent.timeline` (broadcast)

`instructions.get` / `instructions.set`

Config Messages

`config.get` / `config.set`

`config.test_provider`

`models.list`

AI Messages