Latency
What determines time-to-first-audio (TTFA) in KugelAudio text-to-speech, how to measure it correctly, and the levers that lower latency for voice agents.
On this page
Latency depends on the model, voice, endpoint, region, network path, and load. Measure the deployment your users will call instead of relying on a single headline number. Before optimizing or comparing providers, measure your own deployment with a fixed model, voice, region, and endpoint.
The three factors
End-to-end latency decomposes into three parts; each has different levers.
- Inference: the model itself. You don't tune this directly; you avoid paying the model prefill more often than necessary (see chunking: every client-side flush forces a fresh model prefill).
- Processing: what happens to your text before inference. Text normalization and output resampling (when you request a non-native sample rate) both add processing.
- Network: your RTT to the API, paid once per message exchange and several times during a connection handshake. Pick the closest region, and pre-connect so the handshake never lands in a user-visible request.
Word-timestamp messages arrive after their corresponding audio and do not delay it. See Word timestamps.
Levers
Pre-connect at startup
The single biggest fix. Without it, your first request pays the full WebSocket handshake; with it, the handshake happens at application startup where nobody is waiting.
import asyncio
from kugelaudio import KugelAudio
async def main():
# create() returns only after the pooled socket is open.
client = await KugelAudio.create(api_key="...")
try:
... # handle requests with client.tts.stream_async(...)
finally:
await client.aclose()
asyncio.run(main())const client = new KugelAudio({ apiKey: '...' });
await client.connect(); // pay the handshake here, onceKugelAudio client = KugelAudio.createConnected(
KugelAudioOptions.builder("...").build()
); // connects synchronously before returningPre-connect the surface you will actually use. client.connect() (JavaScript
and Java) and Python's async KugelAudio.create() warm the pooled
single-request WebSocket used by stream; a streamingSession owns a separate
socket, so call that session's connect() before the user interaction.
Connections are reusable across turns; see
Turn lifecycle. Python's
synchronous generate() / stream() wrappers create and close a connection
on their own event-loop thread, so use stream_async() or
streaming_session_sync() when connection reuse matters.
Set the language explicitly
When language is unset, the server recognizes each sentence's language from its words and falls back to the voice's
primary language when a sentence is too short to tell (Text normalization). When you know the language, say so:
audio = client.tts.generate(
text="Guten Tag!", model_id="kugel-3.5", voice_id=1071, language="de"
)Let the server chunk; flush once per turn
Client-side per-sentence flushing forces a fresh model prefill per segment, the most common self-inflicted latency bug. Send tokens as they arrive and flush exactly once at the end of the turn; the server groups incoming text at natural sentence boundaries. Full guidance: Chunking & per-segment latency and Turn lifecycle.
Pick the right region and sample rate
Use the region closest to your servers. Keep the native
24000 Hz sample rate when you can; other rates require resampling and do not
make inference faster.
Measuring TTFA correctly
Time-to-first-audio is the metric that matters for voice agents. Measure it correctly or you'll chase the wrong bottleneck.
Pre-connect, then measure
Including the handshake in a TTFA measurement makes every other change look smaller than it is. Pre-connect first, start the clock after the connection is open:
import asyncio
import time
from kugelaudio import KugelAudio
async def main():
client = await KugelAudio.create(api_key="...") # connects before returning
try:
start = time.perf_counter()
async for chunk in client.tts.stream_async(
text="Hello from KugelAudio.",
model_id="kugel-3.5",
voice_id=1071,
language="en",
):
if hasattr(chunk, "audio"):
ttfa_ms = (time.perf_counter() - start) * 1000
print(f"TTFA: {ttfa_ms:.1f} ms")
break
finally:
await client.aclose()
asyncio.run(main())import { KugelAudio } from 'kugelaudio';
const client = new KugelAudio({ apiKey: '...' });
await client.connect(); // handshake paid here, not measured
const start = performance.now();
let first = true;
await client.tts.stream(
{ text: 'Hello from KugelAudio.', modelId: 'kugel-3.5', voiceId: 1071, language: 'en' },
{
onChunk: () => {
if (first) {
console.log(`TTFA: ${(performance.now() - start).toFixed(1)} ms`);
first = false;
}
},
},
);What to measure
Report p50 and p95 over a repeated set of warm requests, not a single measurement. Keep the text, voice, model, language, endpoint, and region fixed.
| Metric | What it tells you |
|---|---|
| Inference TTFA | Server-side model time from a text chunk to the first audio chunk. Useful for comparing model, voice, or parameter changes against a fixed network. |
| End-to-end TTFA | What the user actually feels: network RTT, normalization, inference, and first-chunk delivery on an already-open connection. |
| Cold first request | End-to-end TTFA plus the TCP, TLS, and WebSocket handshake. Pre-connect to move the handshake out of the request path. |
| p50 / p95 / p99 | Distribution percentiles over repeated warm requests. |
| Chunk-to-chunk gap | After the first chunk, how long between subsequent chunks. Spikes here mean the network or playback buffer can't keep up, not the model. |
Common reporting mistakes
- Including the handshake in TTFA. Cold-start cost that has nothing to do with the model. Pre-connect first.
- Measuring only against
localhost. That omits the network path your production users experience. - Single-shot timings. Cold caches, GC pauses, JIT, and scheduler jitter can dominate one request. Compare distributions of repeated warm requests.
- Mixing inference TTFA and end-to-end TTFA. Decide which one you're reporting and label it. Comparing one to the other across vendors is how people end up with wrong "we're slower than X" conclusions.