> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.resemble.ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.resemble.ai/_mcp/server. # Streaming Audio Detection (WebSocket) Stream live audio to Resemble Deepfake Detection and receive per-window verdicts while the audio is still arriving. This interface is audio-only and is intended for server-to-server applications such as call monitoring, contact-center analysis, and live media workflows. ## WebSocket URL ```text wss://stream.resemble.ai/api/v1/detect/audio ``` Authenticate the WebSocket upgrade with your Resemble API key: ```http Authorization: Bearer ``` > **Warning** > > The native browser `WebSocket()` API cannot attach an `Authorization` header. Connect from a trusted backend or another server-side WebSocket client, and never expose your API key in browser code. ### Query Parameters | Parameter | Type | Required | Description | | -------------- | ----------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | `filename` | string | No | Source filename returned in `file` on result messages. | | `extra_params` | JSON object | No | URL-encoded JSON metadata echoed as an object on result messages. Use it to correlate a stream with your own call or session identifier. | Only `filename` and `extra_params` are accepted. Parameters from the file-based Detect API, including `callback_url`, `visualize`, and `intelligence`, are not supported by the streaming endpoint. > **Note** > > A streaming connection does not create a persisted Detect job. The returned `stream_id` is not a Detect UUID and cannot be passed to [Get Detection](/detect/get). Store any streaming results your application needs after the socket closes. ## Audio Format The recommended wire format is: * One channel (mono) * 16,000 Hz sample rate * Signed 16-bit little-endian PCM * A PCM WAV header at the beginning of the first binary frame For a live source whose final length is unknown, the WAV header can use `0xFFFFFFFF` for the RIFF and data lengths. Send subsequent PCM audio in binary WebSocket frames. Frames representing approximately 100 milliseconds of audio provide a useful balance between smooth real-time delivery and frame overhead. The server groups incoming audio into approximately four-second analysis windows. Windows with insufficient voice activity return a skipped result and do not run through the detection model. ## Connection Lifecycle 1. Open the WebSocket with the bearer authorization header and optional query parameters. 2. Wait for the server's `ready` message before sending audio. 3. Send the WAV header and PCM samples as binary frames. Read server messages concurrently while uploading. 4. Receive a `chunk` message for each completed analysis window. 5. Send the text frame `{"type":"end"}` after the last audio frame. 6. Keep the connection open until the server sends `final` and closes the socket with code `1000`. Do not close immediately after sending `end`; queued windows can still be processing. ## Server Messages All server messages are JSON text frames. Clients should ignore unrecognized fields so that compatible fields can be added in the future. ### Ready The first message confirms that the stream can accept audio: ```json { "type": "ready", "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1" } ``` ### Chunk Result Each `chunk` describes one source-time window. On analyzed chunks, the top-level `label`, `aggregated_score`, `score`, and `consistency` are running metrics across all analyzed windows received so far. The nested `chunk_info` fields describe only the current window. ```json { "type": "chunk", "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1", "file": "support-call.wav", "label": "real", "aggregated_score": 0.0812, "score": [0.041, 0.076, 0.094, 0.113], "duration": 0.0, "consistency": 94.3, "extra_params": { "call_id": "call_123" }, "chunk_info": { "chunk_id": 0, "total_chunks": -1, "begin_timestamp_s": 0.0, "end_timestamp_s": 4.0, "chunk_aggregated_score": 0.0812, "chunk_label": "real", "chunk_consistency": 94.3 } } ``` Because the stream length is not known in advance, `total_chunks` is `-1`. Use `chunk_info.begin_timestamp_s` and `chunk_info.end_timestamp_s` to place the result on the original audio timeline. The top-level `duration` on a chunk is not a final duration; use the `duration` from the `final` message. If a window does not contain enough voice activity, its result uses `chunk_label: "skipped"` and null top-level metrics: ```json { "type": "chunk", "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1", "file": "support-call.wav", "label": null, "aggregated_score": null, "score": null, "duration": null, "consistency": null, "chunk_info": { "chunk_id": 1, "total_chunks": -1, "begin_timestamp_s": 4.0, "end_timestamp_s": 8.0, "chunk_aggregated_score": -1.0, "chunk_label": "skipped", "chunk_consistency": -1.0 } } ``` ### Final Result The `final` message contains the aggregate verdict across all analyzed, voice-active windows: ```json { "type": "final", "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1", "file": "support-call.wav", "label": "real", "aggregated_score": 0.0921, "score": [0.041, 0.076, 0.094, 0.113, 0.087, 0.102, 0.119, 0.105], "duration": 8.0, "consistency": 92.8, "extra_params": { "call_id": "call_123" } } ``` | Field | Type | Description | | ------------------ | ----------------------- | --------------------------------------------------------------------------------------------------------- | | `stream_id` | string | Identifier for this WebSocket connection. | | `file` | string | The supplied `filename`, or a generated stream filename when omitted. | | `label` | `real`, `fake`, or null | Current or final detection verdict. It is null when no voice-active audio was analyzed. | | `aggregated_score` | number or null | Aggregate synthetic-audio score. Higher values indicate a stronger synthetic-audio signal. | | `score` | number\[] or null | Approximately one model score per second of analyzed audio. | | `duration` | number or null | On `final`, the analyzed voice-active duration in seconds. This can be shorter than the total audio sent. | | `consistency` | number or null | Consistency metric for the analyzed model scores. | | `extra_params` | object or null | Parsed metadata supplied in the `extra_params` query parameter. | When no windows contain sufficient voice activity, the server still returns `final`, with `label` and `aggregated_score` set to null, an empty `score` array, and `duration: 0.0`. ### Error Errors after the upgrade are delivered as `error` messages before the socket closes. Authorization errors include an `error_code`, a human-readable `error`, and whether reconnecting might succeed without an account change: ```json { "type": "error", "error_code": "insufficient_balance", "error": "Insufficient balance. Please add credits to continue.", "retryable": false } ``` Processing errors can instead use `error_message`. Handle both fields: ```python message = payload.get("error") or payload.get("error_message") or "Streaming failed" ``` ## Authorization, Session Length, and Usage Your API key, Deepfake Detection access, plan limits, and available balance are checked when the connection opens. During active streams they are checked again approximately every five minutes, immediately before the next complete analysis window is processed. An idle connection waits until audio resumes before performing the next check. If access is revoked or the account can no longer run detection, the server stops accepting new analysis work, sends an `error`, and closes with code `1008`. A WebSocket session can remain open for at most 55 minutes. For longer calls, create a new connection before the current session reaches that limit. Usage is measured from voice-active windows that complete detection. VAD-skipped windows are not counted as analyzed usage. A completed stream with nonzero analyzed duration below two seconds uses the existing two-second minimum billable duration for audio Deepfake Detection. ## Handshake Errors Errors that occur before the WebSocket upgrade are returned as JSON HTTP responses. | HTTP status | Meaning | | ----------- | --------------------------------------------------------------------------------- | | `400` | An unsupported query parameter was supplied. | | `401` | The bearer API key is missing, malformed, invalid, or revoked. | | `402` | Billing eligibility, balance, or a usage limit prevents the stream from starting. | | `403` | The account does not have audio Deepfake Detection access. | | `426` | The request was not a WebSocket upgrade. | | `502` | The streaming service could not accept the upstream upgrade. | | `503` | Authorization is temporarily unavailable. | ## WebSocket Close Codes | Close code | Meaning | Client action | | ---------- | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `1000` | The final result was sent and the stream completed normally. | No retry is required. | | `1003` | `extra_params` was not valid JSON. | Fix the query parameter before reconnecting. | | `1008` | Authorization, account access, or the maximum session duration ended the stream. | Resolve the reported account or request issue before reconnecting. Start a new connection after a planned session rollover. | | `1011` | The server could not process the stream. | Log the error and reconnect with backoff if appropriate. | | `1013` | A temporary authorization dependency was unavailable. | Reconnect using exponential backoff. | For a complete runnable client, see [Stream Audio for Deepfake Detection](/guides/detect/streaming-audio).