LLM Integration: Streaming Sessions
For real-time TTS when streaming text from an LLM (like GPT-4, Claude, etc.):
Session Reuse
End a session without closing the WebSocket to avoid reconnection overhead (see Turn lifecycle):
Barge-in (interrupt the current turn)
When the end user speaks over the agent, call cancelCurrent() to stop
generating the current turn immediately and drop any buffered/queued text,
without closing the WebSocket. Unlike endSession(), no remaining text is
flushed; the turn is abandoned. The socket stays open so the next send()
starts the next turn right away. Override onInterrupted() on your
StreamCallbacks to stop local playback at the cancellation point.
cancelCurrent() blocks until the server acknowledges (onInterrupted fires),
or up to ~5 seconds if the server goes silent. Stop local playback as soon as
you call it: a few in-flight frames may arrive before the acknowledgement. See
Barge-in for the
full protocol.
Updating settings mid-session
Change generation parameters on a live connection without reconnecting via
updateSettings(). It sends an explicit, acknowledged update and returns the
parameters now in effect as an EffectiveSettings:
Only generation parameters are updatable: cfgScale, temperature, speed,
maxNewTokens, language, normalize. Identity / audio-format settings
(voiceId, modelId, sampleRate, outputFormat, projectId,
dictionaryIds) are fixed for the connection. Create a new StreamingSession with a fresh StreamConfig
to change them. The change applies to the next turn; call it between turns.
updateSettings() throws a KugelAudioException if the server rejects an
out-of-range value. MultiContextSession exposes updateSettings() too
(session-scoped; applies to contexts started after the update).
Tuning latency with StreamConfig
StreamConfig.builder() exposes the same generation knobs as GenerateRequest
plus the server-side auto-flush timeout:
.voiceId(int) takes a numeric voice ID. It is optional in the Java builder but
required before the first synthesis; omission fails with MISSING_VOICE_ID.
StreamCallbacks reference
StreamCallbacks is used by both tts().stream(...) and streamingSession(...).
Only onChunk is required; the rest are default no-ops you override as needed:
tts().stream(request, callbacks, reuseConnection) takes an optional third
argument: pass true to reuse the client’s pooled WebSocket connection
instead of opening a fresh one for the request.
Per-session usage
For billing your own customers per conversation, every closed session and
request reports a SessionUsage (audio time + the actual amount charged):
session.getLastUsage() on a StreamingSession: usage from the most
recently closed session (null before the first close).
response.getUsage() on the AudioResponse from a one-shot
generate() / stream() request: per-request usage.
session.getUsageFor(contextId) on a MultiContextSession: per-context
usage (see Multi-Context Sessions).
SessionUsage getters: getAudioSeconds() (double, always present),
getCostCents() (Double, the EUR-cents charge or null), getCurrency()
(String), getCharacters() (Integer), getModelId() (String), and
isCostAvailable() (boolean).
getCostCents() is null (and isCostAvailable() is false) when the
charge cannot be determined at session end, e.g. a transient billing error
or an internal session. It is never a misleading 0; getAudioSeconds() is
always reported.
Multi-Context Sessions
Generate audio for multiple speakers or contexts concurrently over a single WebSocket connection:
To barge in on one speaker (the user talks over speaker A), cancel only that
context. Its in-flight generation and queued text are dropped; other contexts and
the connection stay open:
MultiContextConfig.builder() accepts .sampleRate(int), .outputFormat(String),
.normalize(boolean), .language(String), .wordTimestamps(boolean),
.temperature(double), .projectId(long), and .dictionaryIds(List<Integer>). Supported
outputFormat tokens are pcm_8000,
pcm_16000, pcm_22050, pcm_24000, ulaw_8000, and alaw_8000.
Per-context overrides are set with CreateContextOptions.builder():
.voiceId(int), .cfgScale(double), and .maxNewTokens(int).
Every Java multi-context context that synthesizes text must set
.voiceId(int) in its CreateContextOptions. MultiContextConfig has no
default-voice field, so createContext(id) without options creates a voiceless
context whose first text fails with MISSING_VOICE_ID.
MultiContextSession methods: connect(MultiContextCallbacks),
createContext(id[, options]), send(id, text[, flush]), flush(id),
closeContext(id[, immediate]), keepAlive(id), getSessionId(),
getActiveContexts(), isConnected(), and close().
For per-conversation billing, getUsageFor(contextId) returns the
SessionUsage (audio time + amount charged) for a closed
context (each context is its own conversation), or null until it closes;
getContextUsage() returns a snapshot map of contextId to usage for all
closed contexts.
MultiContextCallbacks (only onChunk is required):
Reconnecting after a server restart
During a rolling deploy the server closes sockets with code 1012 or 1013. The
Java SDK treats any close code outside 4xxx as a normal end of stream: no error
callback fires, and it does not reconnect or replay the turn. The next send() on
that session fails. Open a new session and resend the current turn. See
Errors for the close codes.
Next steps
- Voices: list, create, and manage voices
- Types:
AudioChunk, StreamConfig, SessionUsage, WordTimestamp
- Barge-in: the full interruption protocol