Multi-Context WebSocket API
WebSocket reference for /ws/tts/multi: run several independent text-to-speech audio streams over one connection, with messages, voice settings and limits.
Manage up to 20 independent audio streams over a single WebSocket connection. Useful for multi-speaker conversations, pre-buffering, and interleaved audio. The conceptual guide is Multi-context streaming.
/ws/tts/multiConnection
wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEYPrepare a voice before sending text
Once connected, you can send the selected voice and model before the LLM produces text:
{"text": "", "voice_id": 1071, "model_id": "kugel-3.5"}Omit context_id for this configuration-only message. It starts optional reference
preparation without creating an audio context or charging for synthesis. There is
no preparation acknowledgement to wait for; send text normally when it becomes
available. Each generation still checks current voice permissions, and preparation
does not create a missing voice encoding ahead of an admitted generation.
This also works on /ws/tts/stream. The
top-level voice_id only warms preparation: it does not reach contexts. Each
context must still receive voice_settings: {"voice_id": ...} on its first
message, otherwise its first text fails with MISSING_VOICE_ID.
Client → Server Messages
| Message | Description |
|---|---|
{"text": "", "voice_id": 1071, "model_id": "kugel-3.5"} | Prepare the selected voice without creating a context; omit context_id |
{"text": "", "context_id": "ctx1", "voice_settings": {"voice_id": 1071}} | Create a context with its voice (replies context_created) |
{"text": "Hello", "context_id": "ctx1"} | Send text to context |
{"text": "...", "context_id": "ctx1", "flush": true} | Send text and flush buffer |
{"flush": true, "context_id": "ctx1"} | Flush context buffer |
{"text": "", "context_id": "ctx1"} | Keep-alive: an empty-text frame resets the context's inactivity timeout without generating audio |
{"close_context": true, "context_id": "ctx1"} | Close a context, letting queued sentences finish first |
{"close_context": true, "context_id": "ctx1", "immediate": true} | Barge-in: cancel the context's in-flight generation immediately and drop buffered text. See Barge-in |
{"cancel": true, "context_id": "ctx1"} | Barge-in, same as on the other endpoints: cancels that context, or every live context when context_id is omitted. Acknowledged with interrupted |
{"update_settings": {"cfg_scale": 1.5, ...}} | Change session generation parameters mid-connection (no context_id). See Updating settings |
{"close_socket": true} | Close all contexts and connection. {"close": true} and {"end_session": true} also end the whole session here (unlike on /ws/tts/stream) |
Server → Client Messages
| Message | Description |
|---|---|
{"context_created": true, "context_id": "ctx1"} | Context created |
{"generation_started": true, "context_id": "ctx1", "chunk_id": 0, "text": "..."} | Generation started; text is the chunk's full text as spoken, after normalization |
{"audio": "base64...", "enc": "pcm_s16le", "context_id": "ctx1", "idx": 0, "sr": 24000, "samples": 4800, "chunk_id": 0} | Audio chunk (field reference) |
{"chunk_complete": true, "context_id": "ctx1", "chunk_id": 0, "audio_seconds": 1.2, "gen_ms": 150} | Chunk complete |
{"word_timestamps": [...], "context_id": "ctx1", "chunk_id": 0, "text": "...", "original_text": "..."} | Word-level time alignments (when enabled), with offsets into the spoken text and your original_text (payload) |
{"settings_updated": true, "settings": {...}} | Acknowledges an update_settings message; settings holds the generation parameters now in effect. See Updating settings |
{"interrupted": true} | Acknowledges a cancel |
{"final": true, "context_id": "ctx1"} | End of audio for a flush (ElevenLabs is_final equivalent): every audio frame for text sent before your {"flush": true} has been delivered. Also sent right before context_closed on a graceful close. Not sent on an immediate (barge-in) close |
{"error": "context closed without producing audio", "error_code": "INTERNAL_ERROR", "code": 500, "context_id": "ctx1"} | Sent instead of final, right before context_closed, when a context accepted speakable text but produced no audio. A failed turn: retry the text on a new context |
{"context_closed": true, "context_id": "ctx1", "usage": {"audio_seconds": 4.1, "cost_cents": 0.37, "currency": "eur", "model_id": "kugel-3.5"}} | Context closed (terminal: all audio sent). usage carries this conversation's audio time + amount charged (EUR cents; null + cost_unavailable if undetermined) |
{"session_closed": true, "total_audio_seconds": 5.4} | Session ended (all contexts). Per-conversation usage is on each context_closed, not here |
The optional usage object on context_closed is present for organization-
backed requests:
| Field | Type | Description |
|---|---|---|
audio_seconds | number | Audio generated for this context. |
characters | integer | Input characters submitted; omitted when none were recorded. |
cost_cents | number | null | Actual charge in EUR cents; null when it could not be determined. |
currency | string | Currency of cost_cents; present only when a charge is available. |
cost_unavailable | boolean | true when the charge could not be determined; otherwise absent. |
model_id | string | Model billing identifier used for the context. |
Voice Settings
When creating a context, pass voice settings as a nested object:
{
"voice_settings": {
"voice_id": 1071,
"cfg_scale": 2.0,
"max_new_tokens": 2048
}
}| Field | Type | Default | Description |
|---|---|---|---|
voice_id | integer | string | unset | Numeric voice ID or voice handle. An empty message can create a context without one, but it is required before the first non-empty text. |
cfg_scale | number | session value (2.0 initially) | Per-context override, clamped to 1.2 to 2.5. |
max_new_tokens | integer | session value (2048 initially) | Per-context override, 1 to 2048. |
sample_rate | integer | session value | Accepted here for compatibility, but updates the session format (sticky for contexts started later), not just this context. Options: 8000, 16000, 22050, 24000, 44100. |
output_format | string | session value | Same as sample_rate: updates the session format, not just this context. |
Session-Level Config
These options can be set on any message and apply to the entire session:
| Parameter | Type | Default | Description |
|---|---|---|---|
model_id | string | kugel-3.5 | Model to use for generation: kugel-3 or kugel-3.5. See Models. |
cfg_scale | number | 2.0 | Session classifier-free guidance value, clamped to 1.2–2.5; a context's voice_settings can override it. |
temperature | number | unset | Sampling variance, 0.0–1.0. Omission leaves the engine setting unset. |
max_new_tokens | integer | 2048 | Session maximum tokens per generation, 1–2048; a context's voice_settings can override it. |
sample_rate | integer | 24000 | Output sample rate in Hz. Options: 8000, 16000, 22050, 24000, 44100. |
output_format | string | omitted | Combined codec + rate token (for example ulaw_8000). See Audio formats. May be sent top-level or inside voice_settings. |
normalize | boolean | true | Enable text normalization. |
language | string | unset | Supported language code for normalization. |
word_timestamps | boolean | false | Enable word-level timestamp alignment. |
project_id | integer | omitted | Project whose dictionaries should be loaded. When omitted, a project API key uses its own project; see project_id. |
dictionary_ids | integer[] | omitted | Per-session dictionary selection. [] = none; a non-empty list = exactly those dictionaries of the session's project (project_id, or the API key's own project), including inactive ones, bypassing the language filter. |
speed | number | 1.0 | Playback speed multiplier, 0.8–1.2. |
flush_timeout_ms | integer | 500 | Longest wait for the next token after a period only that token can explain (z. of z.B., 1299. of 1299.99). Every other sentence is synthesized as soon as it ends; see Chunking. |
max_buffer_length | integer | 10000 | Maximum buffered characters before a forced flush. |
Reuse the same context_id across turns to keep one context alive
(recommended for a single conversation), or open new ids for parallel
speakers:
// Create / address a context. Session-level fields (sample_rate,
// output_format, language, …) may be sent top-level or inside voice_settings.
{
"context_id": "call-42",
"text": "Hello, how can I help you today?",
"output_format": "ulaw_8000",
"voice_settings": { "voice_id": 1071, "cfg_scale": 2.0 }
}Updating Settings
Change the session's generation parameters mid-connection with an
update_settings message (no context_id: it is session-scoped). The server
replies with settings_updated:
{
"update_settings": {
"cfg_scale": 1.5,
"temperature": 0.3,
"speed": 1.1,
"max_new_tokens": 2048,
"language": "de",
"normalize": true
}
}Only these generation parameters are updatable; every field is optional.
Identity, project, dictionary, and audio-format fields (voice_id, model_id,
sample_rate, output_format, project_id, dictionary_ids) are rejected
inside update_settings with a VALIDATION_ERROR frame. Ordinary context
messages use the session-level fields in the table above.
import asyncio
import websockets
import json
import base64
async def multi_speaker():
uri = "wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEY"
async with websockets.connect(uri) as ws:
# Create narrator context
await ws.send(json.dumps({
"text": "",
"context_id": "narrator",
"voice_settings": {"voice_id": 1071},
}))
# Create character context
await ws.send(json.dumps({
"text": "",
"context_id": "character",
"voice_settings": {"voice_id": 1072},
}))
# Send text to different speakers
await ws.send(json.dumps({
"text": "The story begins.",
"context_id": "narrator",
"flush": True,
}))
await ws.send(json.dumps({
"text": "Hello, I'm the main character!",
"context_id": "character",
"flush": True,
}))
# Receive audio from both contexts. After both flushed turns reach
# their final frame, close the connection gracefully.
completed = set()
async for message in ws:
data = json.loads(message)
if "audio" in data:
ctx = data["context_id"]
audio_bytes = base64.b64decode(data["audio"])
print(f"[{ctx}] Chunk {data['idx']}: {len(audio_bytes)} bytes")
if data.get("context_closed"):
usage = data.get("usage", {})
# Per-context (per-conversation) usage: audio time + charge (EUR cents)
print(f"[{data['context_id']}] usage: {usage.get('audio_seconds')}s, "
f"{usage.get('cost_cents')} ct")
if data.get("final"):
completed.add(data["context_id"])
if completed == {"narrator", "character"}:
await ws.send(json.dumps({"close_socket": True}))
if data.get("session_closed"):
break
asyncio.run(multi_speaker())const API_KEY = 'YOUR_API_KEY';
const WS_URL = 'wss://api.kugelaudio.com';
const ws = new WebSocket(`${WS_URL}/ws/tts/multi?api_key=${API_KEY}`);
const audioQueues = new Map();
const completed = new Set();
ws.onopen = () => {
// Create narrator context
ws.send(JSON.stringify({
text: '',
context_id: 'narrator',
voice_settings: { voice_id: 1071 },
}));
// Create character context
ws.send(JSON.stringify({
text: '',
context_id: 'character',
voice_settings: { voice_id: 1072 },
}));
// Send text to different speakers
ws.send(JSON.stringify({
text: 'The story begins.',
context_id: 'narrator',
flush: true,
}));
ws.send(JSON.stringify({
text: 'Hello, I\'m the main character!',
context_id: 'character',
flush: true,
}));
};
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.audio) {
const ctx = data.context_id;
const binary = atob(data.audio);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
if (!audioQueues.has(ctx)) audioQueues.set(ctx, []);
audioQueues.get(ctx).push(bytes);
console.log(`[${ctx}] Chunk ${data.idx}: ${bytes.length} bytes`);
}
if (data.context_closed) {
// Per-context (per-conversation) usage: audio time + actual charge
console.log(`[${data.context_id}] usage:`, data.usage);
}
if (data.final) {
completed.add(data.context_id);
if (completed.size === 2) ws.send(JSON.stringify({ close_socket: true }));
}
if (data.session_closed) {
ws.close();
}
};# Install wscat: npm install -g wscat
wscat -c "wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEY"
# Create narrator context
> {"text": "", "context_id": "narrator", "voice_settings": {"voice_id": 1071}}
# Create character context
> {"text": "", "context_id": "character", "voice_settings": {"voice_id": 1072}}
# Send text to narrator
> {"text": "The story begins.", "context_id": "narrator", "flush": true}
# Send text to character
> {"text": "Hello, I'm the main character!", "context_id": "character", "flush": true}
# Close all contexts
> {"close_socket": true}Limits
- Maximum 20 concurrent contexts per connection
- Contexts auto-close after 20 seconds of inactivity (send the empty-text keep-alive to reset)
- Opening a context beyond the limit returns a per-context error (
error_code: "TOO_MANY_CONTEXTS",code: 429) without closing the connection. Close an existing context, or wait for an idle one to be released, then retry.
Errors
See Error Codes for the full TTS error lookup table. A refused connection is an HTTP status on the upgrade, not a close code (Refused WebSocket upgrades).