Skip to content
KugelAudioDocs

Stream Input

WebSocket reference for /ws/tts/stream: send LLM text token by token in turn-based sessions, with config, flush, cancel and every response message.

Stream text input token-by-token for LLM integration. This is the endpoint behind every SDK streaming session; the conceptual guide is Streaming overview and the turn semantics are on Turn lifecycle.

WSS/ws/tts/stream

Connection

Text
wss://api.kugelaudio.com/ws/tts/stream?api_key=YOUR_API_KEY

Protocol

  1. Send config (once): Initial configuration message. voice_id, audio format, and the other settings are sticky for the connection; you do not re-send them on later turns.
  2. Send text: Text chunks for the current turn as they arrive.
  3. Receive audio: Audio chunks arrive as they're generated, while you keep sending text.
  4. Send flush: Ends the turn: emits any trailing buffered text, streams its audio, then closes the turn's session (session_closed). The socket stays open.
  5. Next turn: Send the next turn's text (a fresh config is optional). Repeat. To end the whole connection, send close_socket.

Messages

Config Message

Sending the selected voice before text arrives also starts optional voice preparation.

JSON
{
  "voice_id": 1071,
  "model_id": "kugel-3.5",
  "cfg_scale": 2.0,
  "temperature": 0.4,
  "sample_rate": 24000,
  "normalize": true,
  "language": "en",
  "word_timestamps": false,
  "flush_timeout_ms": 500,
  "max_buffer_length": 1000,
  "speed": 1.0
}
FieldTypeDefaultDescription
temperaturenumberunsetSampling variance (0.0 to 1.0). When omitted, uses the engine default, consistently with REST and /ws/tts; see temperature guidance.
flush_timeout_msinteger500Longest wait for the next token after a period only that token can explain (z. of z.B., 1299. of 1299.99). Every other sentence is synthesized as soon as it ends; see Chunking.
max_buffer_lengthinteger10000Maximum characters buffered before a forced flush.
project_idintegeromittedProject whose dictionaries should be loaded. When omitted, a project API key uses its own project; see project_id.
dictionary_idsinteger[]omittedPer-request dictionary selection, sticky for the session. [] = none; a non-empty list selects exactly those dictionaries of the session's project (project_id, or the API key's own project), including inactive ones, bypassing the language filter.

All other fields share the meaning and defaults of the Generate Speech parameters.

To change generation parameters (cfg_scale, temperature, speed, max_new_tokens, language, normalize) part-way through a connection, send an Update Settings message; the change applies to the next turn.

Text Message

JSON
{
  "text": "chunk of text"
}

Flush Message

JSON
{
  "flush": true
}

Close Message

End the current session; the WebSocket stays open and the server starts a fresh session on the next config / text message:

JSON
{
  "close": true
}

{"end_session": true} is accepted as an alias. To end the session and close the WebSocket connection, send {"close_socket": true} instead.

Cancel Message (barge-in)

JSON
{
  "cancel": true
}

Abandons the current turn immediately: in-flight generation is cancelled and buffered text dropped. The server acknowledges with {"interrupted": true}; the socket stays open for the next turn. See Barge-in.

Update Settings Message

Change generation parameters mid-connection without reconnecting. Send an update_settings message; the server validates it and replies with a settings_updated acknowledgement carrying the parameters now in effect.

JSON
{
  "update_settings": {
    "cfg_scale": 1.5,
    "temperature": 0.3,
    "speed": 1.1,
    "max_new_tokens": 2048,
    "language": "de",
    "normalize": true
  }
}

Only these generation parameters are updatable: every field is optional, and a message updates only the fields it carries:

FieldTypeDescription
cfg_scalenumberClassifier-free guidance scale; values are clamped to 1.2–2.5.
temperaturenumberSampling variance (0.0 to 1.0).
max_new_tokensintegerMaximum tokens per generation (1–2048).
languagestringLanguage code (e.g. en, de).
normalizebooleanEnable text normalization.
speednumberPlayback speed multiplier (0.8–1.2).

Identity, project, dictionary, and audio-format fields (voice_id, model_id, sample_rate, output_format, project_id, dictionary_ids) are not accepted inside update_settings. Including one is rejected with a VALIDATION_ERROR frame (the socket stays open) so an unsupported change is never silently dropped. To change one between turns, send it as an ordinary config message before starting the next turn.

Response Messages

Generation Started

JSON
{
  "generation_started": true,
  "chunk_id": 0,
  "text": "Hello, this is streaming."
}

text is the chunk's full text as it is spoken, after normalization.

Audio Chunk

JSON
{
  "audio": "base64_encoded_pcm16_data",
  "enc": "pcm_s16le",
  "idx": 0,
  "sr": 24000,
  "samples": 4800,
  "chunk_id": 0
}

Field-by-field reference: Audio formats.

Word Timestamps (when word_timestamps: true)

JSON
{
  "word_timestamps": [
    {"word": "Hello", "start_ms": 0, "end_ms": 320, "char_start": 0, "char_end": 5, "original_char_start": 0, "original_char_end": 5, "score": 1.0}
  ],
  "chunk_id": 0,
  "text": "Hello world.",
  "original_text": "Hello world."
}

char_* offsets index text, the spoken text; original_char_* offsets index original_text, the text you sent. See Word timestamps for every field.

Chunk Complete

JSON
{
  "chunk_complete": true,
  "chunk_id": 0,
  "audio_seconds": 1.2,
  "gen_ms": 150
}

Interrupted

Sent only in response to {"cancel": true}: the turn was cancelled and the session is ready for the next turn:

JSON
{
  "interrupted": true
}

Settings Updated

Acknowledges an update_settings message. settings holds the generation parameters now in effect for subsequent turns:

JSON
{
  "settings_updated": true,
  "settings": {
    "cfg_scale": 1.5,
    "temperature": 0.3,
    "max_new_tokens": 2048,
    "language": "de",
    "normalize": true,
    "speed": 1.1
  }
}

Warning

Non-fatal advisory; the socket stays open. Currently emitted when a turn is auto-ended after the idle timeout because no flush was sent:

JSON
{
  "warning": "Turn ended after 5s of inactivity. Send {\"flush\": true} to end a turn explicitly — it lowers latency and avoids this auto-flush."
}

Final (End of Audio)

Sent after the last audio frame of every gracefully completed turn (explicit flush, close, or idle auto-flush), right before session_closed. Once you receive it, no further audio for the turn will arrive, the equivalent of ElevenLabs' isFinal. It is not sent after a cancel (barge-in); that path acknowledges with interrupted instead.

JSON
{
  "final": true,
  "total_audio_seconds": 5.4,
  "total_text_chunks": 3,
  "total_audio_chunks": 15
}

Use final to stop waiting for audio (e.g. to end playback or hang up a call); use the session_closed frame that follows for usage/billing data.

Session Closed

Sent at the end of every turn (on flush, idle auto-flush, or close). The socket stays open for the next turn.

JSON
{
  "session_closed": true,
  "total_audio_seconds": 5.4,
  "total_text_chunks": 3,
  "total_audio_chunks": 15,
  "usage": {
    "audio_seconds": 5.4,
    "characters": 142,
    "cost_cents": 0.49,
    "currency": "eur",
    "model_id": "kugel-3.5"
  }
}

The usage object reports the session's consumed audio time and the actual amount charged (EUR cents) so you can bill per conversation, same fields as the /ws/tts final message. cost_cents is null with cost_unavailable: true if the charge can't be determined (never a silent 0).

Example

import asyncio
import websockets
import json
import base64

async def stream_from_llm(llm_tokens):
    uri = "wss://api.kugelaudio.com/ws/tts/stream?api_key=YOUR_API_KEY"

    async with websockets.connect(uri) as ws:
        # Send config
        await ws.send(json.dumps({
            "voice_id": 1071,
            "model_id": "kugel-3.5",
            "cfg_scale": 2.0,
        }))

        # Stream tokens
        for token in llm_tokens:
            await ws.send(json.dumps({"text": token}))

            # Check for audio (non-blocking)
            try:
                message = await asyncio.wait_for(ws.recv(), timeout=0.01)
                data = json.loads(message)
                if "audio" in data:
                    audio_bytes = base64.b64decode(data["audio"])
                    play_audio(audio_bytes)
            except asyncio.TimeoutError:
                pass

        # Flush ends the turn (emits session_closed); close_socket ends the connection.
        # For a multi-turn conversation, skip close_socket and just send the next
        # turn's text after session_closed: the config above stays in effect.
        await ws.send(json.dumps({"flush": True}))
        await ws.send(json.dumps({"close_socket": True}))

        # Receive remaining audio
        async for message in ws:
            data = json.loads(message)
            if "audio" in data:
                audio_bytes = base64.b64decode(data["audio"])
                play_audio(audio_bytes)
            if data.get("session_closed"):
                usage = data.get("usage", {})
                # Per-session usage: audio time + actual charge (EUR cents)
                print(f"Usage: {usage.get('audio_seconds')}s, {usage.get('cost_cents')} ct")
                break

# Example usage
tokens = ["Hello, ", "this ", "is ", "streaming ", "from ", "an ", "LLM."]
asyncio.run(stream_from_llm(tokens))

Errors

See Error Codes for the full TTS error lookup table. A refused connection is an HTTP status on the upgrade, not a close code (Refused WebSocket upgrades).