Skip to content
KugelAudioDocs

Latency

What determines time-to-first-audio (TTFA) in KugelAudio text-to-speech, how to measure it correctly, and the levers that lower latency for voice agents.

On this page

Latency depends on the model, voice, endpoint, region, network path, and load. Measure the deployment your users will call instead of relying on a single headline number. Before optimizing or comparing providers, measure your own deployment with a fixed model, voice, region, and endpoint.

The three factors

End-to-end latency decomposes into three parts; each has different levers.

  1. Inference: the model itself. You don't tune this directly; you avoid paying the model prefill more often than necessary (see chunking: every client-side flush forces a fresh model prefill).
  2. Processing: what happens to your text before inference. Text normalization and output resampling (when you request a non-native sample rate) both add processing.
  3. Network: your RTT to the API, paid once per message exchange and several times during a connection handshake. Pick the closest region, and pre-connect so the handshake never lands in a user-visible request.

Word-timestamp messages arrive after their corresponding audio and do not delay it. See Word timestamps.

Levers

Pre-connect at startup

The single biggest fix. Without it, your first request pays the full WebSocket handshake; with it, the handshake happens at application startup where nobody is waiting.

Python
import asyncio
from kugelaudio import KugelAudio

async def main():
    # create() returns only after the pooled socket is open.
    client = await KugelAudio.create(api_key="...")
    try:
        ...  # handle requests with client.tts.stream_async(...)
    finally:
        await client.aclose()

asyncio.run(main())
TypeScript
const client = new KugelAudio({ apiKey: '...' });
await client.connect();   // pay the handshake here, once
Java
KugelAudio client = KugelAudio.createConnected(
    KugelAudioOptions.builder("...").build()
);   // connects synchronously before returning

Pre-connect the surface you will actually use. client.connect() (JavaScript and Java) and Python's async KugelAudio.create() warm the pooled single-request WebSocket used by stream; a streamingSession owns a separate socket, so call that session's connect() before the user interaction. Connections are reusable across turns; see Turn lifecycle. Python's synchronous generate() / stream() wrappers create and close a connection on their own event-loop thread, so use stream_async() or streaming_session_sync() when connection reuse matters.

Set the language explicitly

When language is unset, the server recognizes each sentence's language from its words and falls back to the voice's primary language when a sentence is too short to tell (Text normalization). When you know the language, say so:

Python
audio = client.tts.generate(
    text="Guten Tag!", model_id="kugel-3.5", voice_id=1071, language="de"
)

Let the server chunk; flush once per turn

Client-side per-sentence flushing forces a fresh model prefill per segment, the most common self-inflicted latency bug. Send tokens as they arrive and flush exactly once at the end of the turn; the server groups incoming text at natural sentence boundaries. Full guidance: Chunking & per-segment latency and Turn lifecycle.

Pick the right region and sample rate

Use the region closest to your servers. Keep the native 24000 Hz sample rate when you can; other rates require resampling and do not make inference faster.

Measuring TTFA correctly

Time-to-first-audio is the metric that matters for voice agents. Measure it correctly or you'll chase the wrong bottleneck.

Pre-connect, then measure

Including the handshake in a TTFA measurement makes every other change look smaller than it is. Pre-connect first, start the clock after the connection is open:

Python
import asyncio
import time
from kugelaudio import KugelAudio

async def main():
    client = await KugelAudio.create(api_key="...")  # connects before returning
    try:
        start = time.perf_counter()
        async for chunk in client.tts.stream_async(
            text="Hello from KugelAudio.",
            model_id="kugel-3.5",
            voice_id=1071,
            language="en",
        ):
            if hasattr(chunk, "audio"):
                ttfa_ms = (time.perf_counter() - start) * 1000
                print(f"TTFA: {ttfa_ms:.1f} ms")
                break
    finally:
        await client.aclose()

asyncio.run(main())
TypeScript
import { KugelAudio } from 'kugelaudio';

const client = new KugelAudio({ apiKey: '...' });
await client.connect();              // handshake paid here, not measured

const start = performance.now();
let first = true;
await client.tts.stream(
  { text: 'Hello from KugelAudio.', modelId: 'kugel-3.5', voiceId: 1071, language: 'en' },
  {
    onChunk: () => {
      if (first) {
        console.log(`TTFA: ${(performance.now() - start).toFixed(1)} ms`);
        first = false;
      }
    },
  },
);

What to measure

Report p50 and p95 over a repeated set of warm requests, not a single measurement. Keep the text, voice, model, language, endpoint, and region fixed.

MetricWhat it tells you
Inference TTFAServer-side model time from a text chunk to the first audio chunk. Useful for comparing model, voice, or parameter changes against a fixed network.
End-to-end TTFAWhat the user actually feels: network RTT, normalization, inference, and first-chunk delivery on an already-open connection.
Cold first requestEnd-to-end TTFA plus the TCP, TLS, and WebSocket handshake. Pre-connect to move the handshake out of the request path.
p50 / p95 / p99Distribution percentiles over repeated warm requests.
Chunk-to-chunk gapAfter the first chunk, how long between subsequent chunks. Spikes here mean the network or playback buffer can't keep up, not the model.

Common reporting mistakes

  • Including the handshake in TTFA. Cold-start cost that has nothing to do with the model. Pre-connect first.
  • Measuring only against localhost. That omits the network path your production users experience.
  • Single-shot timings. Cold caches, GC pauses, JIT, and scheduler jitter can dominate one request. Compare distributions of repeated warm requests.
  • Mixing inference TTFA and end-to-end TTFA. Decide which one you're reporting and label it. Comparing one to the other across vendors is how people end up with wrong "we're slower than X" conclusions.

Next steps