Skip to main content
tl;dr: pre-connect at startup, set language explicitly, end every turn with flush, and let the server chunk your text.
Latency depends on the model, voice, endpoint, region, network path, and load. Measure the deployment your users will call instead of relying on a single headline number. Before optimizing or comparing providers, measure your own deployment with a fixed model, voice, region, and endpoint.

The three factors

End-to-end latency decomposes into three parts; each has different levers.
  1. Inference: the model itself. You don’t tune this directly; you avoid paying the model prefill more often than necessary (see chunking: every client-side flush forces a fresh model prefill).
  2. Processing: what happens to your text before inference. Text normalization and output resampling (when you request a non-native sample rate) both add processing.
  3. Network: your RTT to the API, paid once per message exchange and several times during a connection handshake. Pick the closest region, and pre-connect so the handshake never lands in a user-visible request.
Word-timestamp messages arrive after their corresponding audio and do not delay it. See Word timestamps.

Levers

Pre-connect at startup

The single biggest fix. Without it, your first request pays the full WebSocket handshake; with it, the handshake happens at application startup where nobody is waiting.
Pre-connect the surface you will actually use. client.connect() (JavaScript and Java) and Python’s async KugelAudio.create() warm the pooled single-request WebSocket used by stream; a streamingSession owns a separate socket, so call that session’s connect() before the user interaction. Connections are reusable across turns; see Turn lifecycle. Python’s synchronous generate() / stream() wrappers create and close a connection on their own event-loop thread, so use stream_async() or streaming_session_sync() when connection reuse matters.

Set the language explicitly

When language is unset, the server normalizes in the voice’s primary language (English if it has none); it does not detect the language from the text. If the text is in another language, say so:

Let the server chunk; flush once per turn

Client-side per-sentence flushing forces a fresh model prefill per segment, the most common self-inflicted latency bug. Send tokens as they arrive and flush exactly once at the end of the turn; the server groups incoming text at natural sentence boundaries. Full guidance: Chunking & per-segment latency and Turn lifecycle.

Pick the right region and sample rate

Use the region closest to your servers. Keep the native 24000 Hz sample rate when you can; other rates require resampling and do not make inference faster.

Measuring TTFA correctly

Time-to-first-audio is the metric that matters for voice agents. Measure it correctly or you’ll chase the wrong bottleneck.

Pre-connect, then measure

Including the handshake in a TTFA measurement makes every other change look smaller than it is. Pre-connect first, start the clock after the connection is open:

What to measure

Report p50 and p95 over a repeated set of warm requests, not a single measurement. Keep the text, voice, model, language, endpoint, and region fixed.

Common reporting mistakes

  • Including the handshake in TTFA. Cold-start cost that has nothing to do with the model. Pre-connect first.
  • Measuring only against localhost. That omits the network path your production users experience.
  • Single-shot timings. Cold caches, GC pauses, JIT, and scheduler jitter can dominate one request. Compare distributions of repeated warm requests.
  • Mixing inference TTFA and end-to-end TTFA. Decide which one you’re reporting and label it. Comparing one to the other across vendors is how people end up with wrong “we’re slower than X” conclusions.

Next steps

Chunking & per-segment latency

Why per-sentence flushing increases TTFA and how server chunking works

Turn lifecycle

How turns start and end, session reuse, and the idle auto-flush
Last modified on September 22, 2026