Chunking & per-segment latency

/ws/tts/stream is one logical TTS request per turn, regardless of how many send calls you make. The server’s text buffer accumulates tokens and hands each sentence to the model as soon as its closing ., ? or ! arrives. Text with no sentence end yet is synthesized after 2 s without new input, so end each sentence with punctuation or flush at the end of the reply. Inside a single turn, model state (KV cache, voice conditioning) is preserved across chunks so prosody stays natural. On /ws/tts/stream, a flush ends the turn. The server synthesizes what is buffered, sends final and then session_closed with its own usage block, and the next text starts a new turn on a fresh engine session. There is no mid-turn flush: flushing per sentence turns one reply into N turns, each with its own model prefill (the full model time-to-first-audio, see Latency), its own final/session_closed and its own usage record. See Turn lifecycle.

Flush granularity: flush once per reply

What decides time-to-first-audio per segment is how often you flush, not how you split the text you send. Raw LLM tokens sent without flush are fine: the server’s buffer reassembles them and only hands sentence-sized work to the model.

Flush granularity	Result
Once per reply	Best. One turn, one prefill, one usage record.
Per sentence	N turns per reply, each paying a fresh prefill.
Per clause or per word	The most turns and prefills. Worst time-to-first-audio.

If the full text is available before TTS starts, send it in one message with flush. We deliberately don’t publish exact ms figures here: they depend on region, voice, and deployment. To reproduce the comparison for your own deployment, time each chunking strategy against your endpoint as described in Measuring TTFA correctly.

Server-side chunking

Native streaming sessions use the server’s sentence-aware text buffer. Send tokens as they arrive and let that buffer decide when a complete unit is ready. Two message fields tune it (see the Stream Input reference):

flush_timeout_ms (default 500): the longest a period waits when only the next token can tell what it means: after a single letter (z. could become z.B.) or after a number (1299. could become 1299.99, 3. could become 3. Oktober); after a number it waits up to twice as long. The next token ends the wait. Every other sentence is synthesized as soon as it ends. Text with no sentence end yet waits 2 s.
max_buffer_length: the most characters the buffer holds before it is synthesized regardless of boundaries.

The native API has no diffusion-step or client-defined chunk-schedule controls. ElevenLabs-style fields such as chunk_length_schedule or auto_mode are unknown to it: the server ignores them and replies with a warning frame. The SDK code for this pattern is in the Streaming overview.

Handle backpressure

If audio arrives faster than you can play it, bound your buffer instead of letting it grow:

import asyncio

async def stream_with_backpressure():
    buffer = asyncio.Queue(maxsize=10)  # Limit buffer size

    async def producer():
        async for chunk in client.tts.stream_async(
            text=text,
            model_id="kugel-3",
            voice_id=1071,
        ):
            if hasattr(chunk, 'audio'):
                await buffer.put(chunk.audio)
        await buffer.put(None)  # Signal end

    async def consumer():
        while True:
            audio = await buffer.get()
            if audio is None:
                break
            play_audio(audio)
            await asyncio.sleep(len(audio) / 2 / 24000)  # Simulate playback time

    await asyncio.gather(producer(), consumer())

Common mistakes

Per-sentence flush=true. Every flush ends the turn, so the next text pays a fresh prefill and produces its own session_closed and usage record. Flush once per reply.
One session per sentence. A new WebSocket handshake plus a fresh model prefill, every sentence. Keep the same session open for the whole assistant reply and reuse it for the next one. See Turn lifecycle.
Client-side sentence buffering before send. Unnecessary: the server already buffers tokens and chunks at sentence boundaries. Pre-buffering on the client just adds latency.
Calling send(text, flush=true) per word “for lower latency.” It is the opposite: each flush is a separate turn and model call. Word-granular flushing produces the worst possible TTFA.

If you’re migrating from ElevenLabs, the flush semantics are the biggest behavioral difference. See the ElevenLabs migration guide.

Get started

Build a voice agent

Generate audio

Voices

Enhance audio

Hosting and deployment

SDK Reference

API Reference

Chunking & per-segment latency

Flush granularity: flush once per reply

Server-side chunking

Handle backpressure

Common mistakes

Next steps

Latency

Turn lifecycle

​Flush granularity: flush once per reply

​Server-side chunking

​Handle backpressure

​Common mistakes

​Next steps

Latency

Turn lifecycle

Flush granularity: flush once per reply

Server-side chunking

Handle backpressure

Common mistakes

Next steps