Multi-context streaming
Guide to multi-context streaming: run several independent text-to-speech audio streams over one WebSocket for multi-speaker dialogue and pre-buffering.
On this page
For advanced use cases like multi-speaker conversations or pre-buffering
audio, use the multi-context WebSocket endpoint (/ws/tts/multi). This allows
managing up to 20 independent audio streams over a single connection.
Use cases
- Multi-speaker conversations: Generate audio for different speakers concurrently
- Pre-buffering: Start generating the next response while the current one plays
- Interleaved audio: Dynamically switch between speakers in real-time
Example
Every SDK wraps /ws/tts/multi in a multi-context session. The raw WebSocket
version follows below.
import asyncio
from kugelaudio import KugelAudio
client = KugelAudio(api_key="YOUR_API_KEY")
async def multi_speaker_demo():
async with client.tts.multi_context_session(
model_id="kugel-3.5", language="en"
) as session:
await session.create_context("narrator", voice_id=1071)
await session.create_context("character", voice_id=1072)
async for chunk in session.send("narrator", "The story begins.", flush=True):
play_audio("narrator", chunk.audio)
async for chunk in session.send(
"character", "Hello, I'm the main character!", flush=True
):
play_audio("character", chunk.audio)
# Gracefully drain and close both contexts
for context_id in ("narrator", "character"):
async for chunk in session.close_context(context_id):
play_audio(context_id, chunk.audio)
print(f"[{context_id}] usage: {session.usage_for(context_id)}")
asyncio.run(multi_speaker_demo())import { KugelAudio } from 'kugelaudio';
const client = new KugelAudio({ apiKey: 'YOUR_API_KEY' });
const session = client.tts.createMultiContextSession({ language: 'en' });
const closed = new Set<string>();
await session.connect({
onChunk: (chunk) => playAudio(chunk.contextId, chunk.audio),
onContextClosed: (contextId, usage) => {
// Per-context usage for this conversation: audio time + charge
console.log(`[${contextId}] usage:`, usage);
closed.add(contextId);
if (closed.size === 2) session.close();
},
});
session.createContext('narrator', { voiceId: 1071 });
session.createContext('character', { voiceId: 1072 });
session.send('narrator', 'The story begins.', true);
session.send('character', "Hello, I'm the main character!", true);
// Gracefully drain and close both contexts
session.closeContext('narrator');
session.closeContext('character');CountDownLatch closed = new CountDownLatch(2);
MultiContextCallbacks callbacks = new MultiContextCallbacks() {
@Override
public void onChunk(String contextId, AudioChunk chunk) {
playAudio(contextId, chunk.getAudio());
}
@Override
public void onContextClosed(String contextId) {
closed.countDown();
}
};
MultiContextConfig config = MultiContextConfig.builder().language("en").build();
try (MultiContextSession session = client.multiContextSession(config)) {
session.connect(callbacks);
session.createContext("narrator", CreateContextOptions.builder().voiceId(1071).build());
session.createContext("character", CreateContextOptions.builder().voiceId(1072).build());
session.send("narrator", "The story begins.", true);
session.send("character", "Hello, I'm the main character!", true);
// Gracefully drain and close both contexts, then wait for them
session.closeContext("narrator");
session.closeContext("character");
closed.await(30, TimeUnit.SECONDS);
}Raw WebSocket
import asyncio
import websockets
import json
import base64
async def multi_speaker_demo():
async with websockets.connect(
"wss://api.kugelaudio.com/ws/tts/multi?api_key=YOUR_API_KEY"
) as ws:
# The first message for an ID creates its context.
await ws.send(json.dumps({
"text": "The story begins.",
"context_id": "narrator",
"voice_settings": {"voice_id": 1071},
"flush": True,
}))
await ws.send(json.dumps({
"text": "Hello, I'm the main character!",
"context_id": "character",
"voice_settings": {"voice_id": 1072},
"flush": True,
}))
# Gracefully drain and close both contexts.
await ws.send(json.dumps({"close_context": True, "context_id": "narrator"}))
await ws.send(json.dumps({"close_context": True, "context_id": "character"}))
# Receive audio from both contexts
closed = set()
async for message in ws:
data = json.loads(message)
if "audio" in data:
context_id = data["context_id"]
audio_bytes = base64.b64decode(data["audio"])
play_audio(context_id, audio_bytes)
if data.get("context_closed"):
# Per-context usage for this conversation: audio time + charge
print(f"[{data['context_id']}] usage: {data.get('usage')}")
closed.add(data["context_id"])
if len(closed) == 2:
await ws.send(json.dumps({"close_socket": True}))
if data.get("session_closed"):
break
asyncio.run(multi_speaker_demo())# Multi-context streaming requires a WebSocket client.
# Install websocat: https://github.com/vi/websocat
# Connect and send JSON messages interactively:
websocat "wss://api.kugelaudio.com/ws/tts/multi?api_key=$KUGELAUDIO_API_KEY"
# Then type each message and press Enter:
{"text": "The story begins.", "context_id": "narrator", "voice_settings": {"voice_id": 1071}, "flush": true}
{"text": "Hello, I'm the main character!", "context_id": "character", "voice_settings": {"voice_id": 1072}, "flush": true}
{"close_context": true, "context_id": "narrator"}
{"close_context": true, "context_id": "character"}
{"close_socket": true}Protocol summary
Each context message carries a context_id; the first message for a new ID
creates the context and can include voice_settings. An empty-text message can
therefore create or keep alive a context, but voice_id must be set no later
than the first non-empty text message. Per context you can send text, flush, and
close_context (with "immediate": true for
barge-in);
{"close_socket": true} ends everything. The server tags every
context-specific response frame (context_created, generation_started,
audio, chunk_complete, word_timestamps, final, context_closed, and
per-context errors) with the originating context_id. The final
session_closed frame is connection-wide and has no context_id; it carries
total_audio_seconds across the connection. After each flush, a final frame
(ElevenLabs is_final equivalent) signals that all audio for the flushed text
has been delivered.
If a context ends without producing any audio for text it accepted, the server
replaces that final with a per-context error frame
(error_code: "INTERNAL_ERROR", code: 500) before context_closed, so a
lost turn is never indistinguishable from a completed one. Treat it as a failed
turn and retry the text on a new context.
The full message tables, every field of every client→server and server→client frame, live in the Text-to-Speech API reference.
Usage is billed per context: a synthesized, billable context's
context_closed frame carries its usage block. A context closed before
synthesis starts has no usage block. See
Per-session usage.
Limits
- Maximum 20 concurrent contexts per connection
- Contexts auto-close after 20 seconds of inactivity
- Send empty text
{"text": "", "context_id": "..."}to reset the per-context inactivity timeout - Opening a context beyond the limit returns a per-context error
(
error_code: "TOO_MANY_CONTEXTS",code: 429) without closing the connection. Close an existing context, or wait for an idle one to be released, then retry.