Streaming

In the examples on this page, play_audio stands for your own playback function; the SDK does not ship one.

Streaming Audio

Receive audio chunks as they are generated for lower latency:

from kugelaudio import AudioChunk

# Synchronous streaming
for item in client.tts.stream(
    text="Hello, this is streaming audio.",
    model_id="kugel-3",
    voice_id=1071,
):
    if isinstance(item, AudioChunk):
        # Process audio chunk immediately
        print(f"Chunk {item.index}: {len(item.audio)} bytes, {item.samples} samples")
        play_audio(item.audio)
    elif isinstance(item, dict) and item.get('final'):
        # Final stats
        print(f"Total duration: {item.get('dur_ms', 0):.0f}ms")
        print(f"Generation time: {item.get('gen_ms', 0):.0f}ms")

Async Streaming

For async applications:

import asyncio

from kugelaudio import AudioChunk

async def generate_speech():
    async for item in client.tts.stream_async(
        text="Async streaming example.",
        model_id="kugel-3",
        voice_id=1071,
    ):
        if isinstance(item, AudioChunk):
            play_audio(item.audio)

asyncio.run(generate_speech())

stream_async() reuses the client’s pooled WebSocket by default; pass reuse_connection=False to open a fresh one.

LLM Integration: Streaming Sessions

For real-time TTS when streaming text from an LLM:

Async Streaming Session

import asyncio

async def stream_from_llm():
    # Simulate LLM token stream
    llm_tokens = ["Hello, ", "this ", "is ", "a ", "streamed ", "response."]

    async with client.tts.streaming_session(
        voice_id=1071,
        cfg_scale=2.0,
        flush_timeout_ms=500,  # Auto-flush after 500ms of no input
    ) as session:
        # Send tokens as they arrive from LLM
        for token in llm_tokens:
            async for chunk in session.send(token):
                # Play audio chunk immediately
                play_audio(chunk.audio)

        # Flush any remaining text
        async for chunk in session.flush():
            play_audio(chunk.audio)

asyncio.run(stream_from_llm())

Synchronous Streaming Session

with client.tts.streaming_session_sync(voice_id=1071) as session:
    for token in llm_tokens:
        for chunk in session.send(token):
            play_audio(chunk.audio)

    for chunk in session.flush():
        play_audio(chunk.audio)

Session Reuse

End a session without closing the WebSocket to avoid reconnection overhead when starting a new session (see Turn lifecycle). end_session() and close() discard audio that is still in flight, so call drain() first to receive the rest of the turn:

async with client.tts.streaming_session(voice_id=1071) as session:
    # Session 1
    async for chunk in session.send("Hello from voice one.", flush=True):
        play_audio(chunk.audio)
    async for chunk in session.drain():  # receive the tail audio
        play_audio(chunk.audio)
    await session.end_session()  # Keeps WebSocket open

    # Session 2: no reconnection needed
    session.update_config(voice_id=1072)
    async for chunk in session.send("Hello from voice two.", flush=True):
        play_audio(chunk.audio)
    async for chunk in session.drain():
        play_audio(chunk.audio)
# Leaving the block closes the session and the WebSocket

Session reuse needs the async session: StreamingSessionSync has no end_session() or update_config().

Barge-in (interrupt the current turn)

When the end user speaks over the agent, call cancel_current() to stop generating the current turn immediately and drop any buffered or queued text, without closing the WebSocket. Unlike end_session(), no remaining text is synthesized; the turn is abandoned. The socket stays open so the next send() starts the next turn right away.

session = client.tts.streaming_session(voice_id=1071)

async for chunk in session.send("This is a very long answer the user talks over"):
    play_audio(chunk.audio)

# VAD detected the user speaking: barge in
await session.cancel_current()

# Socket still open: next turn starts immediately
async for chunk in session.send("Sure, what would you like instead?", flush=True):
    play_audio(chunk.audio)

cancel_current() returns once the server acknowledges, or after a short quiet timeout if the server goes silent. Stop local playback as soon as you call it: a few in-flight frames may arrive before the acknowledgement. See Barge-in for the full protocol. The synchronous wrapper exposes cancel_current() too.

Updating settings mid-session

Change generation parameters on a live connection without reconnecting via update_settings(). It sends an explicit, acknowledged update and returns the parameters now in effect:

session = client.tts.streaming_session(voice_id=1071, cfg_scale=2.0)

async for chunk in session.send("Default voicing.", flush=True):
    play_audio(chunk.audio)

# Change cfg_scale / speed / temperature for the next turn:
effective = await session.update_settings(cfg_scale=1.5, speed=1.1)
# -> {"cfg_scale": 1.5, "temperature": ..., "speed": 1.1, ...}

async for chunk in session.send("Now a bit faster and softer.", flush=True):
    play_audio(chunk.audio)

Only generation parameters are updatable: cfg_scale, temperature, speed, max_new_tokens, language, normalize. Identity and audio-format settings (voice_id, model_id, sample_rate, output_format, project_id, dictionary_ids) are fixed for the connection; change those with update_config() after end_session(). The change applies to the next turn; call it between turns. The server rejects an out-of-range value or a non-updatable field with a typed error. StreamingSessionSync and MultiContextSession expose update_settings() too (on the multi session it is session-scoped and applies to contexts started after the update).

Streaming session reference

A session is created with streaming_session(...) (async) or streaming_session_sync(...) (sync). Both accept the same configuration: voice_id, model_id, cfg_scale, temperature, max_new_tokens, sample_rate, flush_timeout_ms, normalize, language, word_timestamps, speed, dictionary_ids, project_id, and an on_word_timestamps callback. Although voice_id defaults to None in the SDK signature, it must be set before the first synthesis; omission fails with MISSING_VOICE_ID. Dictionaries apply only when project_id is set (see Dictionaries). The async StreamingSession exposes:

Method	Returns	Description
`await session.connect()`	`None`	Open and authenticate the WebSocket. Called automatically by the first `send()` and by `async with`.
`session.send(text, flush=False)`	`AsyncIterator[AudioChunk]`	Buffer `text` and yield audio as it is generated. Without `flush`, it yields only what arrives within a short poll; `flush=True` forces synthesis of whatever is buffered and waits for its audio.
`session.flush()`	`AsyncIterator[AudioChunk]`	Flush the buffer and yield remaining audio for the current turn.
`session.drain()`	`AsyncIterator[AudioChunk]`	Signal end-of-input and yield every remaining chunk until the server closes the session. Call it before `end_session()` or `close()` to receive the tail audio.
`await session.end_session()`	`dict`	End the current session but keep the WebSocket open for reuse. The server flushes buffered text, but audio still in flight is discarded client-side; call `drain()` first to receive it.
`await session.cancel_current()`	`None`	Barge-in: abandon the current turn and drop buffered or queued text, keeping the socket open.
`session.update_config(config=None, **kwargs)`	`None`	Update configuration (e.g. `voice_id`) for the next session after `end_session()`. Keyword arguments change only those fields; passing a whole `StreamConfig` replaces every field.
`await session.update_settings(**params)`	`dict`	Change generation parameters (`cfg_scale`, `temperature`, `speed`, `max_new_tokens`, `language`, `normalize`) mid-connection and return the effective values. Applies to the next turn.
`await session.close()`	`dict`	Cancel a pending connection attempt, or end the session and close the WebSocket. Like `end_session()`, it discards audio still in flight.
`session.last_word_timestamps`	`list[WordTimestamp]`	The most recently received word timestamps.
`session.last_final`	`dict \| None`	End-of-audio stats from the most recently completed turn: the server’s `{"final": true, ...}` frame (ElevenLabs `isFinal` equivalent), sent after the turn’s last audio frame. `None` before the first turn completes; not updated on a barge-in cancel.
`session.last_usage`	`SessionUsage \| None`	Per-session usage (audio time + amount charged) from the most recently closed session, for billing your own customers per conversation. `None` before the first session closes. See SessionUsage.

StreamingSessionSync has a subset of this API without await/async for: send(), flush(), and drain() return list[AudioChunk], and cancel_current(), update_settings(), close(), and the last_word_timestamps / last_final / last_usage properties behave the same. It has no connect(), end_session(), or update_config(), so session reuse and voice switching are async-only.

Tuning streaming latency

By default the server accumulates LLM tokens and only begins generating at natural sentence boundaries. Tune how eagerly it starts with these session parameters:

Parameter	Type	Default	Description
`flush_timeout_ms`	`int`	`500`	Server-side auto-flush timeout: emit buffered text after this many milliseconds of no new input.
`chunk_length_schedule`	`list[int] \| None`	server default `[5, 80, 150, 250]`	Minimum buffer size (characters) before each successive auto-chunk is emitted. Entry `i` applies to chunk `i`; the last value repeats. Smaller values lower time-to-first-audio; larger values improve prosody.
`auto_mode`	`bool \| None`	`None`	Start generating at the very first clean sentence boundary (equivalent to ElevenLabs’ `auto_mode`). Lowest TTFA, slightly less prosody context.
`max_buffer_length`	`int`	`1000`	Maximum characters buffered before a forced flush.

flush_timeout_ms is a session factory argument. Set the other three with session.update_config(...) before the first send:

session = client.tts.streaming_session(voice_id=1071)
session.update_config(
    auto_mode=True,
    chunk_length_schedule=[50, 100, 150, 250],  # low-latency schedule
)

Multi-Context Sessions

A multi-context session manages up to 20 independent audio-generation contexts over a single WebSocket (see limits). Each context has its own text buffer, voice settings, and generation queue: useful for multi-speaker conversations, pre-buffering one stream while another plays, or interleaving audio for dynamic dialogue.

async with client.tts.multi_context_session(language="en") as session:
    # Create contexts, optionally with different voices
    await session.create_context("narrator", voice_id=1071)
    await session.create_context("character", voice_id=1072)

    # Send text to a specific context
    async for chunk in session.send("narrator", "The story begins."):
        play_audio(chunk.audio)

    async for chunk in session.send("character", "Hello there!", flush=True):
        play_audio(chunk.audio)

    # Drain remaining audio and close one context
    async for chunk in session.close_context("narrator"):
        play_audio(chunk.audio)

Create the session with multi_context_session(...):

Parameter	Type	Default	Description
`default_voice_id`	`int \| None`	`None`	Default voice for contexts that don’t override it.
`model_id`	`str \| None`	`None`	Model to use.
`sample_rate`	`int`	`24000`	Output sample rate.
`cfg_scale`	`float`	`2.0`	Guidance scale (1.2 to 2.5; values outside are clamped).
`temperature`	`float \| None`	`None`	Sampling variance.
`max_new_tokens`	`int`	`2048`	Maximum tokens per generation.
`normalize`	`bool`	`True`	Enable text normalization.
`language`	`str \| None`	`None`	Normalization language.
`dictionary_ids`	`list[int] \| None`	`None`	Exact dictionary selection; requires `project_id`. `[]` disables dictionaries.
`inactivity_timeout`	`float`	`20.0`	Seconds before an idle context auto-closes.
`word_timestamps`	`bool`	`False`	Request per-word time alignments for every context.
`on_word_timestamps`	`Callable[[str, list[WordTimestamp]], None] \| None`	`None`	Called with `(context_id, timestamps)` as alignments arrive.
`project_id`	`int \| None`	`None`	Project whose dictionaries apply; see Dictionaries.

Every context that synthesizes text must have an effective voice. Set default_voice_id on the session or pass voice_id to every create_context() call; otherwise the first text sent to that context fails with MISSING_VOICE_ID. MultiContextSession methods:

Method	Returns	Description
`await session.connect()`	`None`	Open the WebSocket. Called automatically by `async with`.
`await session.create_context(context_id, voice_id=None)`	`None`	Create a context with an optional voice override.
`session.send(context_id, text, flush=False, chunk_complete_idle_timeout=None)`	`AsyncIterator[AudioChunk]`	Send text to a context and yield its audio.
`session.flush(context_id)`	`AsyncIterator[AudioChunk]`	Flush a context’s buffer.
`session.close_context(context_id, immediate=False)`	`AsyncIterator[AudioChunk]`	Close a context and drain its audio. `immediate=True` barges in, discarding buffered or queued text.
`await session.keep_alive(context_id)`	`None`	Reset a context’s inactivity timeout.
`await session.update_settings(**params)`	`dict`	Change generation parameters for contexts started after the update; returns the effective values.
`session.get_word_timestamps(context_id)`	`list[WordTimestamp]`	The most recently received word timestamps for a context.
`await session.close()`	`dict`	Cancel a pending connection attempt, or close the established session and return stats.
`session.usage_for(context_id)`	`SessionUsage \| None`	Per-context usage (audio time + amount charged) for a closed context; each context is its own conversation. `None` until that context closes. See SessionUsage.
`session.context_usage`	`dict[str, SessionUsage]`	Map of `context_id` to usage for every context closed so far.
`session.active_contexts`	`set[str]`	The set of currently active context IDs.
`session.session_id`	`str \| None`	Server-assigned session ID.
`session.is_alive`	`bool`	Whether the underlying WebSocket is still usable for `send()`.

Word Timestamps in Streaming

Word timestamps work with one-shot streams and streaming sessions. During a one-shot stream, they are yielded as list[WordTimestamp] objects between audio chunks:

from kugelaudio import AudioChunk, WordTimestamp

for item in client.tts.stream(
    text="Hello, how are you today?",
    model_id="kugel-3",
    voice_id=1071,
    word_timestamps=True,
):
    if isinstance(item, AudioChunk):
        play_audio(item.audio)
    elif isinstance(item, list) and item and isinstance(item[0], WordTimestamp):
        for ts in item:
            print(f"{ts.word}: {ts.start_ms}-{ts.end_ms}ms")

Word Timestamps in Streaming Sessions

Request word-level time alignments alongside audio. Timestamps are delivered per chunk after the corresponding audio data:

async with client.tts.streaming_session(
    voice_id=1071,
    word_timestamps=True,
) as session:
    async for chunk in session.send("Hello, how are you today?"):
        play_audio(chunk.audio)

    async for chunk in session.flush():
        play_audio(chunk.audio)

    # Access the latest word timestamps
    timestamps = session.last_word_timestamps
    for ts in timestamps:
        print(f"{ts.word}: {ts.start_ms}ms - {ts.end_ms}ms (score: {ts.score:.2f})")

You can also register a callback to process timestamps as they arrive:

def on_timestamps(timestamps):
    for ts in timestamps:
        print(f"  {ts.word} [{ts.start_ms}-{ts.end_ms}ms]")

async with client.tts.streaming_session(
    voice_id=1071,
    on_word_timestamps=on_timestamps,
) as session:
    async for chunk in session.send("Hello world!"):
        play_audio(chunk.audio)
    async for chunk in session.flush():
        play_audio(chunk.audio)

Word timestamps add no extra audio latency. They arrive shortly after the corresponding audio chunk (see Latency) and are useful for barge-in handling, subtitle synchronization, and lip-sync.

Next steps

Types & Errors: AudioChunk, StreamConfig, SessionUsage, WordTimestamp
Text Normalization: languages and spell tags in streaming

Get started

Build a voice agent

Generate audio

Voices

Enhance audio

Hosting and deployment

SDK Reference

API Reference

Streaming Audio

Async Streaming

LLM Integration: Streaming Sessions

Async Streaming Session

Synchronous Streaming Session

Session Reuse

Barge-in (interrupt the current turn)

Updating settings mid-session

Streaming session reference

Tuning streaming latency

Multi-Context Sessions

Word Timestamps in Streaming

Word Timestamps in Streaming Sessions

Next steps

​Streaming Audio

​Async Streaming

​LLM Integration: Streaming Sessions

​Async Streaming Session

​Synchronous Streaming Session

​Session Reuse

​Barge-in (interrupt the current turn)

​Updating settings mid-session

​Streaming session reference

​Tuning streaming latency

​Multi-Context Sessions

​Word Timestamps in Streaming

​Word Timestamps in Streaming Sessions

​Next steps

Streaming Audio

Async Streaming

LLM Integration: Streaming Sessions

Async Streaming Session

Synchronous Streaming Session

Session Reuse

Barge-in (interrupt the current turn)

Updating settings mid-session

Streaming session reference

Tuning streaming latency

Multi-Context Sessions

Word Timestamps in Streaming

Word Timestamps in Streaming Sessions

Next steps