Streaming Audio Detection (WebSocket)

Stream live audio to Resemble Deepfake Detection and receive per-window verdicts while the audio is still arriving. This interface is audio-only and is intended for server-to-server applications such as call monitoring, contact-center analysis, and live media workflows.

WebSocket URL

wss://stream.resemble.ai/api/v1/detect/audio

Authenticate the WebSocket upgrade with your Resemble API key:

1
Authorization: Bearer <RESEMBLE_API_KEY>

Query Parameters

Only filename and extra_params are accepted. Parameters from the file-based Detect API, including callback_url, visualize, and intelligence, are not supported by the streaming endpoint.

Audio Format

The recommended wire format is:

• One channel (mono)
• 16,000 Hz sample rate
• Signed 16-bit little-endian PCM
• A PCM WAV header at the beginning of the first binary frame

For a live source whose final length is unknown, the WAV header can use 0xFFFFFFFF for the RIFF and data lengths. Send subsequent PCM audio in binary WebSocket frames. Frames representing approximately 100 milliseconds of audio provide a useful balance between smooth real-time delivery and frame overhead.

The server groups incoming audio into approximately four-second analysis windows. Windows with insufficient voice activity return a skipped result and do not run through the detection model.

Connection Lifecycle

1. Open the WebSocket with the bearer authorization header and optional query parameters.
2. Wait for the server’s ready message before sending audio.
3. Send the WAV header and PCM samples as binary frames. Read server messages concurrently while uploading.
4. Receive a chunk message for each completed analysis window.
5. Send the text frame {"type":"end"} after the last audio frame.
6. Keep the connection open until the server sends final and closes the socket with code 1000.

Do not close immediately after sending end; queued windows can still be processing.

Server Messages

All server messages are JSON text frames. Clients should ignore unrecognized fields so that compatible fields can be added in the future.

Ready

The first message confirms that the stream can accept audio:

1
{
2
  "type": "ready",
3
  "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1"
4
}

Chunk Result

Each chunk describes one source-time window. On analyzed chunks, the top-level label, aggregated_score, score, and consistency are running metrics across all analyzed windows received so far. The nested chunk_info fields describe only the current window.

1
{
2
  "type": "chunk",
3
  "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1",
4
  "file": "support-call.wav",
5
  "label": "real",
6
  "aggregated_score": 0.0812,
7
  "score": [0.041, 0.076, 0.094, 0.113],
8
  "duration": 0.0,
9
  "consistency": 94.3,
10
  "extra_params": {
11
    "call_id": "call_123"
12
  },
13
  "chunk_info": {
14
    "chunk_id": 0,
15
    "total_chunks": -1,
16
    "begin_timestamp_s": 0.0,
17
    "end_timestamp_s": 4.0,
18
    "chunk_aggregated_score": 0.0812,
19
    "chunk_label": "real",
20
    "chunk_consistency": 94.3
21
  }
22
}

Because the stream length is not known in advance, total_chunks is -1. Use chunk_info.begin_timestamp_s and chunk_info.end_timestamp_s to place the result on the original audio timeline. The top-level duration on a chunk is not a final duration; use the duration from the final message.

If a window does not contain enough voice activity, its result uses chunk_label: "skipped" and null top-level metrics:

1
{
2
  "type": "chunk",
3
  "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1",
4
  "file": "support-call.wav",
5
  "label": null,
6
  "aggregated_score": null,
7
  "score": null,
8
  "duration": null,
9
  "consistency": null,
10
  "chunk_info": {
11
    "chunk_id": 1,
12
    "total_chunks": -1,
13
    "begin_timestamp_s": 4.0,
14
    "end_timestamp_s": 8.0,
15
    "chunk_aggregated_score": -1.0,
16
    "chunk_label": "skipped",
17
    "chunk_consistency": -1.0
18
  }
19
}

Final Result

The final message contains the aggregate verdict across all analyzed, voice-active windows:

1
{
2
  "type": "final",
3
  "stream_id": "f241df9c-4738-48f8-8098-34e712e63bd1",
4
  "file": "support-call.wav",
5
  "label": "real",
6
  "aggregated_score": 0.0921,
7
  "score": [0.041, 0.076, 0.094, 0.113, 0.087, 0.102, 0.119, 0.105],
8
  "duration": 8.0,
9
  "consistency": 92.8,
10
  "extra_params": {
11
    "call_id": "call_123"
12
  }
13
}

When no windows contain sufficient voice activity, the server still returns final, with label and aggregated_score set to null, an empty score array, and duration: 0.0.

Error

Errors after the upgrade are delivered as error messages before the socket closes. Authorization errors include an error_code, a human-readable error, and whether reconnecting might succeed without an account change:

1
{
2
  "type": "error",
3
  "error_code": "insufficient_balance",
4
  "error": "Insufficient balance. Please add credits to continue.",
5
  "retryable": false
6
}

Processing errors can instead use error_message. Handle both fields:

1
message = payload.get("error") or payload.get("error_message") or "Streaming failed"

Authorization, Session Length, and Usage

Your API key, Deepfake Detection access, plan limits, and available balance are checked when the connection opens. During active streams they are checked again approximately every five minutes, immediately before the next complete analysis window is processed. An idle connection waits until audio resumes before performing the next check.

If access is revoked or the account can no longer run detection, the server stops accepting new analysis work, sends an error, and closes with code 1008. A WebSocket session can remain open for at most 55 minutes. For longer calls, create a new connection before the current session reaches that limit.

Usage is measured from voice-active windows that complete detection. VAD-skipped windows are not counted as analyzed usage. A completed stream with nonzero analyzed duration below two seconds uses the existing two-second minimum billable duration for audio Deepfake Detection.

Handshake Errors

Errors that occur before the WebSocket upgrade are returned as JSON HTTP responses.

WebSocket Close Codes

For a complete runnable client, see Stream Audio for Deepfake Detection.