Latency

​

What to expect

​

The three factors

1. Inference — the model itself. You don’t tune this directly; you avoid paying the model prefill more often than necessary (see chunking — every client-side flush forces a fresh model prefill).
2. Processing — what happens to your text before inference. Language auto-detection adds work when language is unset. Output resampling also adds processing when you request a non-native sample rate.
3. Network — your RTT to the API, paid once per message exchange and several times during a connection handshake. Pick the closest region, and pre-connect so the handshake never lands in a user-visible request.

​

Levers

​

Pre-connect at startup

from kugelaudio import KugelAudio

# In async Python code, create() returns only after the pooled socket is open.
client = await KugelAudio.create(api_key="...")

const client = new KugelAudio({ apiKey: '...' });
await client.connect();   // pay the handshake here, once

KugelAudio client = KugelAudio.createConnected(
    KugelAudioOptions.builder("...").build()
);   // connects synchronously before returning

​

Set the language explicitly

client.tts.stream(
    text="Guten Tag!", model_id="kugel-3", voice_id=1071, language="de"
)

​

Let the server chunk; flush once per turn

​

Avoid extra model prefills

​

Pick the right region and sample rate

​

Measuring TTFA correctly

​

Pre-connect, then measure

import asyncio
import time
from kugelaudio import KugelAudio

async def main():
    client = await KugelAudio.create(api_key="...")  # connects before returning
    try:
        start = time.perf_counter()
        async for chunk in client.tts.stream_async(
            text="Hello from KugelAudio.",
            model_id="kugel-3",
            voice_id=1071,
            language="en",
        ):
            if hasattr(chunk, "audio"):
                ttfa_ms = (time.perf_counter() - start) * 1000
                print(f"TTFA: {ttfa_ms:.1f} ms")
                break
    finally:
        await client.aclose()

asyncio.run(main())

import { KugelAudio } from 'kugelaudio';

const client = new KugelAudio({ apiKey: '...' });
await client.connect();              // handshake paid here, not measured

const start = performance.now();
let first = true;
await client.tts.stream(
  { text: 'Hello from KugelAudio.', modelId: 'kugel-3', voiceId: 1071, language: 'en' },
  {
    onChunk: () => {
      if (first) {
        console.log(`TTFA: ${(performance.now() - start).toFixed(1)} ms`);
        first = false;
      }
    },
  },
);

​

What to measure

​

Reference benchmark

cd packages/public/java-sdk/benchmark
mvn compile exec:java

• Cold TTFA (first request, includes handshake) vs pooled TTFA (subsequent requests, connection reused) — quantifies what pre-connecting saves on your network.
• TTFA across chunking strategies (full-text, sentence, ≥20-char, clause, word) — the cost of small flushes.
• RTF on long-form text.

​

Common reporting mistakes

• Including the handshake in TTFA. Cold-start cost that has nothing to do with the model. Pre-connect first.
• Measuring only against localhost. That omits the network path your production users experience.
• Single-shot timings. Cold caches, GC pauses, JIT, and scheduler jitter can dominate one request. Compare distributions of repeated warm requests.
• Mixing inference TTFA and end-to-end TTFA. Decide which one you’re reporting and label it. Comparing one to the other across vendors is how people end up with wrong “we’re slower than X” conclusions.

​

Next steps