Skip to content
KugelAudioDocs

Multi-Context WebSocket API

WebSocket reference for /ws/tts/multi: run several independent text-to-speech audio streams over one connection, with messages, voice settings and limits.

Manage up to 20 independent audio streams over a single WebSocket connection. Useful for multi-speaker conversations, pre-buffering, and interleaved audio. The conceptual guide is Multi-context streaming.

WSS/ws/tts/multi

Connection

Text
wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEY

Prepare a voice before sending text

Once connected, you can send the selected voice and model before the LLM produces text:

JSON
{"text": "", "voice_id": 1071, "model_id": "kugel-3.5"}

Omit context_id for this configuration-only message. It starts optional reference preparation without creating an audio context or charging for synthesis. There is no preparation acknowledgement to wait for; send text normally when it becomes available. Each generation still checks current voice permissions, and preparation does not create a missing voice encoding ahead of an admitted generation.

This also works on /ws/tts/stream. The top-level voice_id only warms preparation: it does not reach contexts. Each context must still receive voice_settings: {"voice_id": ...} on its first message, otherwise its first text fails with MISSING_VOICE_ID.

Client → Server Messages

MessageDescription
{"text": "", "voice_id": 1071, "model_id": "kugel-3.5"}Prepare the selected voice without creating a context; omit context_id
{"text": "", "context_id": "ctx1", "voice_settings": {"voice_id": 1071}}Create a context with its voice (replies context_created)
{"text": "Hello", "context_id": "ctx1"}Send text to context
{"text": "...", "context_id": "ctx1", "flush": true}Send text and flush buffer
{"flush": true, "context_id": "ctx1"}Flush context buffer
{"text": "", "context_id": "ctx1"}Keep-alive: an empty-text frame resets the context's inactivity timeout without generating audio
{"close_context": true, "context_id": "ctx1"}Close a context, letting queued sentences finish first
{"close_context": true, "context_id": "ctx1", "immediate": true}Barge-in: cancel the context's in-flight generation immediately and drop buffered text. See Barge-in
{"cancel": true, "context_id": "ctx1"}Barge-in, same as on the other endpoints: cancels that context, or every live context when context_id is omitted. Acknowledged with interrupted
{"update_settings": {"cfg_scale": 1.5, ...}}Change session generation parameters mid-connection (no context_id). See Updating settings
{"close_socket": true}Close all contexts and connection. {"close": true} and {"end_session": true} also end the whole session here (unlike on /ws/tts/stream)

Server → Client Messages

MessageDescription
{"context_created": true, "context_id": "ctx1"}Context created
{"generation_started": true, "context_id": "ctx1", "chunk_id": 0, "text": "..."}Generation started; text is the chunk's full text as spoken, after normalization
{"audio": "base64...", "enc": "pcm_s16le", "context_id": "ctx1", "idx": 0, "sr": 24000, "samples": 4800, "chunk_id": 0}Audio chunk (field reference)
{"chunk_complete": true, "context_id": "ctx1", "chunk_id": 0, "audio_seconds": 1.2, "gen_ms": 150}Chunk complete
{"word_timestamps": [...], "context_id": "ctx1", "chunk_id": 0, "text": "...", "original_text": "..."}Word-level time alignments (when enabled), with offsets into the spoken text and your original_text (payload)
{"settings_updated": true, "settings": {...}}Acknowledges an update_settings message; settings holds the generation parameters now in effect. See Updating settings
{"interrupted": true}Acknowledges a cancel
{"final": true, "context_id": "ctx1"}End of audio for a flush (ElevenLabs is_final equivalent): every audio frame for text sent before your {"flush": true} has been delivered. Also sent right before context_closed on a graceful close. Not sent on an immediate (barge-in) close
{"error": "context closed without producing audio", "error_code": "INTERNAL_ERROR", "code": 500, "context_id": "ctx1"}Sent instead of final, right before context_closed, when a context accepted speakable text but produced no audio. A failed turn: retry the text on a new context
{"context_closed": true, "context_id": "ctx1", "usage": {"audio_seconds": 4.1, "cost_cents": 0.37, "currency": "eur", "model_id": "kugel-3.5"}}Context closed (terminal: all audio sent). usage carries this conversation's audio time + amount charged (EUR cents; null + cost_unavailable if undetermined)
{"session_closed": true, "total_audio_seconds": 5.4}Session ended (all contexts). Per-conversation usage is on each context_closed, not here

The optional usage object on context_closed is present for organization- backed requests:

FieldTypeDescription
audio_secondsnumberAudio generated for this context.
charactersintegerInput characters submitted; omitted when none were recorded.
cost_centsnumber | nullActual charge in EUR cents; null when it could not be determined.
currencystringCurrency of cost_cents; present only when a charge is available.
cost_unavailablebooleantrue when the charge could not be determined; otherwise absent.
model_idstringModel billing identifier used for the context.

Voice Settings

When creating a context, pass voice settings as a nested object:

JSON
{
  "voice_settings": {
    "voice_id": 1071,
    "cfg_scale": 2.0,
    "max_new_tokens": 2048
  }
}
FieldTypeDefaultDescription
voice_idinteger | stringunsetNumeric voice ID or voice handle. An empty message can create a context without one, but it is required before the first non-empty text.
cfg_scalenumbersession value (2.0 initially)Per-context override, clamped to 1.2 to 2.5.
max_new_tokensintegersession value (2048 initially)Per-context override, 1 to 2048.
sample_rateintegersession valueAccepted here for compatibility, but updates the session format (sticky for contexts started later), not just this context. Options: 8000, 16000, 22050, 24000, 44100.
output_formatstringsession valueSame as sample_rate: updates the session format, not just this context.

Session-Level Config

These options can be set on any message and apply to the entire session:

ParameterTypeDefaultDescription
model_idstringkugel-3.5Model to use for generation: kugel-3 or kugel-3.5. See Models.
cfg_scalenumber2.0Session classifier-free guidance value, clamped to 1.2–2.5; a context's voice_settings can override it.
temperaturenumberunsetSampling variance, 0.0–1.0. Omission leaves the engine setting unset.
max_new_tokensinteger2048Session maximum tokens per generation, 1–2048; a context's voice_settings can override it.
sample_rateinteger24000Output sample rate in Hz. Options: 8000, 16000, 22050, 24000, 44100.
output_formatstringomittedCombined codec + rate token (for example ulaw_8000). See Audio formats. May be sent top-level or inside voice_settings.
normalizebooleantrueEnable text normalization.
languagestringunsetSupported language code for normalization.
word_timestampsbooleanfalseEnable word-level timestamp alignment.
project_idintegeromittedProject whose dictionaries should be loaded. When omitted, a project API key uses its own project; see project_id.
dictionary_idsinteger[]omittedPer-session dictionary selection. [] = none; a non-empty list = exactly those dictionaries of the session's project (project_id, or the API key's own project), including inactive ones, bypassing the language filter.
speednumber1.0Playback speed multiplier, 0.8–1.2.
flush_timeout_msinteger500Longest wait for the next token after a period only that token can explain (z. of z.B., 1299. of 1299.99). Every other sentence is synthesized as soon as it ends; see Chunking.
max_buffer_lengthinteger10000Maximum buffered characters before a forced flush.

Reuse the same context_id across turns to keep one context alive (recommended for a single conversation), or open new ids for parallel speakers:

JSON
// Create / address a context. Session-level fields (sample_rate,
// output_format, language, …) may be sent top-level or inside voice_settings.
{
  "context_id": "call-42",
  "text": "Hello, how can I help you today?",
  "output_format": "ulaw_8000",
  "voice_settings": { "voice_id": 1071, "cfg_scale": 2.0 }
}

Updating Settings

Change the session's generation parameters mid-connection with an update_settings message (no context_id: it is session-scoped). The server replies with settings_updated:

JSON
{
  "update_settings": {
    "cfg_scale": 1.5,
    "temperature": 0.3,
    "speed": 1.1,
    "max_new_tokens": 2048,
    "language": "de",
    "normalize": true
  }
}

Only these generation parameters are updatable; every field is optional. Identity, project, dictionary, and audio-format fields (voice_id, model_id, sample_rate, output_format, project_id, dictionary_ids) are rejected inside update_settings with a VALIDATION_ERROR frame. Ordinary context messages use the session-level fields in the table above.

Example

import asyncio
import websockets
import json
import base64

async def multi_speaker():
    uri = "wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEY"

    async with websockets.connect(uri) as ws:
        # Create narrator context
        await ws.send(json.dumps({
            "text": "",
            "context_id": "narrator",
            "voice_settings": {"voice_id": 1071},
        }))

        # Create character context
        await ws.send(json.dumps({
            "text": "",
            "context_id": "character",
            "voice_settings": {"voice_id": 1072},
        }))

        # Send text to different speakers
        await ws.send(json.dumps({
            "text": "The story begins.",
            "context_id": "narrator",
            "flush": True,
        }))

        await ws.send(json.dumps({
            "text": "Hello, I'm the main character!",
            "context_id": "character",
            "flush": True,
        }))

        # Receive audio from both contexts. After both flushed turns reach
        # their final frame, close the connection gracefully.
        completed = set()
        async for message in ws:
            data = json.loads(message)

            if "audio" in data:
                ctx = data["context_id"]
                audio_bytes = base64.b64decode(data["audio"])
                print(f"[{ctx}] Chunk {data['idx']}: {len(audio_bytes)} bytes")

            if data.get("context_closed"):
                usage = data.get("usage", {})
                # Per-context (per-conversation) usage: audio time + charge (EUR cents)
                print(f"[{data['context_id']}] usage: {usage.get('audio_seconds')}s, "
                      f"{usage.get('cost_cents')} ct")

            if data.get("final"):
                completed.add(data["context_id"])
                if completed == {"narrator", "character"}:
                    await ws.send(json.dumps({"close_socket": True}))

            if data.get("session_closed"):
                break

asyncio.run(multi_speaker())

Limits

  • Maximum 20 concurrent contexts per connection
  • Contexts auto-close after 20 seconds of inactivity (send the empty-text keep-alive to reset)
  • Opening a context beyond the limit returns a per-context error (error_code: "TOO_MANY_CONTEXTS", code: 429) without closing the connection. Close an existing context, or wait for an idle one to be released, then retry.

Errors

See Error Codes for the full TTS error lookup table. A refused connection is an HTTP status on the upgrade, not a close code (Refused WebSocket upgrades).