WebSocket upgrade: the session's audio in, its transcript out

GET /sessions/{sessionId}/audio

Browsers cannot set headers on an upgrade, so the FIRST frame must be {"type":"auth","token":"<jwt>"} within ten seconds — the token of the person who started the session, in its organization, holding orders:write — or the server answers an error frame and closes. Then binary frames are audio (the browser's MediaRecorder chunks), and text frames pause and resume control it (paused audio is neither transcribed nor billed) while token hands over a renewed JWT. The server sends ready, state, and a transcript frame per interim guess and per final utterance; each final is also published live on the organization's stream (the events gateway's voice:session:<sessionId> room). A connection that drops can be opened again and the session carries on where it was. Appends nothing: the socket only listens — stopping is POST /sessions/:sessionId/end, and a session nobody stops is ended by the service's sweep once its audio has been gone two minutes. A plain GET without an upgrade returns 426.

Access

Public: the socket authenticates its first frame with the rep's JWT and holds it to the session's tenant and person

No token: the link, the claim or the first message is the credential, as the description says.

How scopes come from roles: Scopes and roles. Getting a token: Authentication and app tokens.

AI agent tool

No agent tool. The WebSocket the admin streams the microphone's audio into, not a request and response a tool call can make; what was said reaches an agent as voice.utterance-heard and the transcript kept on the quote (fulfillment_link_transcript).

Connecting an agent: AI agents.

Example

Upgrade, send the auth frame, stream audio; the server answers frames like this one.

curl

# A WebSocket (websocat): the auth frame first, then the socket stays open.
(printf '%s\n' "{\"type\":\"auth\",\"token\":\"$MANIFOLD_TOKEN\",\"since\":{\"inventory\":0}}"; cat) \
  | websocat 'wss://voice.getmanifold.io/sessions/vcs_01a15a32-7d10-7f3b-a2c6-4e8b1d0f5c33/audio'

JavaScript (fetch)

const socket = new WebSocket("wss://voice.getmanifold.io/sessions/vcs_01a15a32-7d10-7f3b-a2c6-4e8b1d0f5c33/audio");
// Browsers cannot set headers on the upgrade: the token is the first frame.
socket.addEventListener("open", () =>
  socket.send(JSON.stringify({ type: "auth", token: process.env.MANIFOLD_TOKEN, since: { inventory: 0 } })),
);
socket.addEventListener("message", (event) => console.log(JSON.parse(event.data)));

First frame from the server, after the upgrade (101)

{"type":"transcript","kind":"final","utteranceId":"u-3","text":"Twelve of the blue widgets, please.","speaker":1,"atMs":8420}

The panel writes this call in other languages. It does not send it: Manifold's APIs do not take calls from a web page on another site, so run the call from your own code or terminal. The whole API as OpenAPI: voice-manager.openapi.json. Base URL https://voice.getmanifold.io.

Errors

This operation declares no refusals.