LiveKit Integration

KugelAudio ships an official TTS plugin for LiveKit Agents in Python and in TypeScript. It plugs into AgentSession as the tts provider and keeps one WebSocket open per agent, so every turn streams over a warm connection.

1. Install

Python
TypeScript

pip install "kugelaudio[livekit]" \
  livekit-plugins-deepgram livekit-plugins-openai livekit-plugins-silero

The livekit extra installs livekit-agents. The Deepgram, OpenAI and Silero plugins are only needed for the STT, LLM and VAD in the example below; swap them for the providers you use.

npm install kugelaudio @livekit/agents @livekit/rtc-node \
  @livekit/agents-plugin-deepgram @livekit/agents-plugin-openai @livekit/agents-plugin-silero

@livekit/agents and @livekit/rtc-node are optional peer dependencies; the core kugelaudio client never loads them. The three @livekit/agents-plugin-* packages are only needed for the STT, LLM and VAD in the example below.

2. Add it to your agent

Pass the plugin as tts and call prewarm() while the call is being set up, so the first response does not pay for the connection. Save the Python version as voice_agent.py or the TypeScript version as voice_agent.ts.

Python
TypeScript

from livekit.agents import Agent, AgentSession, JobContext
from livekit.agents import WorkerOptions, cli
from livekit.plugins import deepgram, openai, silero
from kugelaudio.livekit import TTS as KugelAudioTTS

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    tts = KugelAudioTTS(model="kugel-3", voice_id=1071, language="en")
    tts.prewarm()

    session = AgentSession(
        stt=deepgram.STT(),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=tts,
        vad=silero.VAD.load(),
    )
    await session.start(
        room=ctx.room,
        agent=Agent(instructions="You are a helpful voice assistant."),
    )
    await session.say("Hello! How can I help you?")

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

python voice_agent.py console   # local test
python voice_agent.py start     # production worker

import { fileURLToPath } from 'node:url';
import { WorkerOptions, cli, defineAgent, voice } from '@livekit/agents';
import * as deepgram from '@livekit/agents-plugin-deepgram';
import * as openai from '@livekit/agents-plugin-openai';
import * as silero from '@livekit/agents-plugin-silero';
import { TTS as KugelAudioTTS } from 'kugelaudio/livekit';

export default defineAgent({
  entry: async (ctx) => {
    await ctx.connect();

    const tts = new KugelAudioTTS({
      model: 'kugel-3',
      voiceId: 1071,
      language: 'en',
    });
    tts.prewarm();

    const session = new voice.AgentSession({
      stt: new deepgram.STT(),
      llm: new openai.LLM({ model: 'gpt-4o-mini' }),
      tts,
      vad: await silero.VAD.load(),
    });
    await session.start({
      agent: new voice.Agent({
        instructions: 'You are a helpful voice assistant.',
      }),
      room: ctx.room,
    });
    await session.say('Hello! How can I help you?');
  },
});

cli.runApp(new WorkerOptions({ agent: fileURLToPath(import.meta.url) }));

npx tsx voice_agent.ts console   # local test
npx tsx voice_agent.ts start     # production worker

The worker reads KUGELAUDIO_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY and LIVEKIT_API_SECRET from the environment (full list under Environment variables). Pick a voice in Voices; the plugin has no default voice.

Three settings decide most of the result: the voice, the model (kugel-3, see Models) and the language. Set language when the text is not in the voice’s primary language; it is not detected from the text.

3. Tune the sound

speed, temperature and cfg_scale shape the audio. Set them in the constructor or change them mid-call; either way the next turn picks up the new values.

Python
TypeScript

tts = KugelAudioTTS(
    model="kugel-3",
    voice_id=1071,
    language="de",
    speed=1.1,          # 0.8 to 1.2
    temperature=0.3,    # 0.0 (most stable) to 1.0 (most varied)
    cfg_scale=2.0,      # 1.2 to 2.5
)

# Later, e.g. from a tool call: switch voice, slow down
tts.update_options(voice_id=300, speed=0.9)

const tts = new KugelAudioTTS({
  model: 'kugel-3',
  voiceId: 1071,
  language: 'de',
  speed: 1.1,          // 0.8 to 1.2
  temperature: 0.3,    // 0.0 (most stable) to 1.0 (most varied)
  cfgScale: 2.0,       // 1.2 to 2.5
});

// Later, e.g. from a tool call: switch voice, slow down
tts.updateOptions({ voiceId: 300, speed: 0.9 });

What each parameter does is documented in Generation parameters. Two rules specific to the plugin:

Out-of-range values raise instead of being clamped: speed outside 0.8 to 1.2 or temperature outside 0.0 to 1.0 is rejected before anything is applied, so a bad update leaves the current settings untouched.
speed and temperature are session-wide. All turns share one socket and the server applies the last value it received. A change therefore takes effect for turns started after it; a turn already playing keeps the old value.

4. Word timestamps (optional)

Turn on the word-timestamps option (off by default, see Options) and the server aligns each audio chunk and delivers per-word timing alongside it. LiveKit uses this for barge-in and transcript sync. Timestamps arrive after their audio chunk, so playback never waits for them. Details in Word timestamps.

5. Remove background noise (optional)

enhancement() puts KugelAudio speech enhancement (Clarity, clarity-1) on the agent’s audio input, the slot LiveKit uses for noise cancellation. It removes background noise before VAD and STT hear the user. Pass speaker= with a clean 2 to 8 second sample of one person and it keeps only that voice. Python only; it needs livekit-agents 1.3.10 or later. On an older release, importing enhancement raises an ImportError that says to upgrade:

pip install -U "kugelaudio[livekit]" "livekit-agents>=1.3.10" \
  livekit-plugins-deepgram livekit-plugins-openai livekit-plugins-silero

The agent from step 2, with enhancement on the participant’s audio:

from livekit.agents import Agent, AgentSession, JobContext
from livekit.agents import WorkerOptions, cli, room_io
from livekit.plugins import deepgram, openai, silero
from kugelaudio import load_audio
from kugelaudio.livekit import TTS as KugelAudioTTS
from kugelaudio.livekit import enhancement

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    tts = KugelAudioTTS(model="kugel-3", voice_id=1071, language="en")
    tts.prewarm()

    session = AgentSession(
        stt=deepgram.STT(),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=tts,
        vad=silero.VAD.load(),
    )
    await session.start(
        room=ctx.room,
        agent=Agent(instructions="You are a helpful voice assistant."),
        room_options=room_io.RoomOptions(
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=enhancement(),  # reads KUGELAUDIO_API_KEY
                # keep one voice: enhancement(speaker=load_audio("caller.wav"))
            ),
        ),
    )
    await session.say("Hello! How can I help you?")

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

LiveKit releases that still take room_input_options= accept the same processor as room_io.RoomInputOptions(noise_cancellation=enhancement()). The input must be mono, which is LiveKit’s default; on a multi-channel input enhancement turns itself off with an error log. Before you turn it on:

Latency. Enhancement runs on KugelAudio’s servers, not on the device, so the audio makes a round trip over the network and the agent hears the user somewhat later than with an on-device filter. Frames keep their size; they arrive delayed.
Plan and limits. Enhancement uses your organization’s Clarity plan: the free month, then paid. Every second of the participant’s audio is enhanced, silence included, and each running agent session holds one stream open, so on the free month its concurrency limit caps how many sessions can be enhanced at once. See Free month and paid plan.
Connection. The processor opens its connection when the agent starts, before the participant speaks, and keeps it for the participant’s later tracks and after a sample-rate change. After 60 seconds without audio the server closes it and the next audio reconnects.
Failures. While the stream connects, or reconnects after a rate limit or a network error, the agent hears the original audio and a warning is logged. A rejected API key, spent credits, an ended free month or an invalid request turn enhancement off for the rest of the session with one error log; pass on_error= to be told.

Option	Default	Description
`api_key`	`KUGELAUDIO_API_KEY` env	API key.
`client`	`None`	An existing `KugelAudio` client, instead of `api_key` and `base_url`.
`model`	`"clarity-1"`	Enhancement model.
`speaker`	`None`	`load_audio(...)` sample of the one voice to keep. Unset: remove noise, keep every voice.
`max_backlog_s`	`2.0`	Seconds the server may fall behind before the original audio is used and the stream reconnects.
`on_error`	`None`	Called once with the error that turned enhancement off.
`base_url`	`https://api.kugelaudio.com`	API base URL.

Reference

Options

Python
TypeScript

Keyword arguments of kugelaudio.livekit.TTS(...); all of them except api_key, region, base_url, http_session and sample_rate can be changed later with update_options(...).

Option	Default	Description
`api_key`	`KUGELAUDIO_API_KEY` env	API key. An `eu-` prefix selects the EU endpoint.
`voice_id`	`None`	Voice ID. Required before the first turn.
`model`	`"kugel-3"`	TTS model.
`language`	`None`	ISO 639-1 code such as `"de"`. Unset: the voice’s primary language.
`speed`	`None`	Playback speed, `0.8` to `1.2`. Unset means the server default `1.0`.
`temperature`	`None`	Sampling variance, `0.0` to `1.0`. Unset means the engine default.
`dictionary_ids`	`None`	Pronunciation dictionaries by ID; `[]` disables them. A non-empty list requires `project_id`.
`project_id`	`None`	Project whose dictionaries apply. Unset: no dictionary applies.
`cfg_scale`	`2.0`	Voice adherence, clamped to `1.2` to `2.5`.
`max_new_tokens`	`2048`	Maximum tokens per turn, `1` to `2048`.
`sample_rate`	`24000`	Output rate in Hz, see Sample rates.
`normalize`	`True`	Text normalization, see Text processing.
`word_timestamps`	`False`	Per-word timing, see above.
`region`	`None`	`"eu"` for the direct EU endpoint. Overridden by `base_url`.
`base_url`	`https://api.kugelaudio.com`	API base URL.
`http_session`	`None`	aiohttp `ClientSession` to reuse.

The plugin can also be registered under LiveKit’s plugin namespace:

from kugelaudio.livekit import register_plugin
register_plugin()

from livekit.plugins import kugelaudio
tts = kugelaudio.TTS(model="kugel-3", voice_id=1071)

Fields of the object passed to new TTS({...}) from kugelaudio/livekit; all of them except apiKey, baseURL and sampleRate can be changed later with updateOptions({...}) (a sampleRate passed there is ignored).

Option	Default	Description
`apiKey`	`KUGELAUDIO_API_KEY` env	API key. An `eu-` prefix selects the EU endpoint.
`voiceId`	`null`	Voice ID. Required before the first turn.
`model`	`'kugel-3'`	TTS model.
`language`	unset	ISO 639-1 code such as `'de'`. Unset: the voice’s primary language.
`speed`	unset	Playback speed, `0.8` to `1.2`. Unset means the server default `1.0`.
`temperature`	unset	Sampling variance, `0.0` to `1.0`. Unset means the engine default.
`cfgScale`	`2.0`	Voice adherence, clamped to `1.2` to `2.5`.
`maxNewTokens`	`2048`	Maximum tokens per turn, `1` to `2048`.
`sampleRate`	`24000`	Output rate in Hz, see Sample rates.
`normalize`	`true`	Text normalization, see Text processing.
`wordTimestamps`	`false`	Per-word timing, see above.
`dictionaryIds`	unset	Pronunciation dictionaries by ID. Requires `projectId`.
`projectId`	unset	Project the dictionaries belong to.
`baseURL`	`https://api.kugelaudio.com`	API base URL.

Sample rates

24000 Hz is the native rate and the right choice unless the transport needs another one. 22050, 16000, 8000 and 44100 are resampled server-side; use 8000 or 16000 for telephony.

Environment variables

Variable	Required	Description
`KUGELAUDIO_API_KEY`	Yes	Your KugelAudio API key
`LIVEKIT_URL`	Yes	Your LiveKit server URL
`LIVEKIT_API_KEY`	Yes	LiveKit API key
`LIVEKIT_API_SECRET`	Yes	LiveKit API secret
`DEEPGRAM_API_KEY`	If using Deepgram STT	Deepgram key
`OPENAI_API_KEY`	If using OpenAI LLM	OpenAI key

Using the plugin without AgentSession

Direct synthesize() and stream() calls

AgentSession drives these for you. Call them directly only when you consume the audio frames yourself.

Python
TypeScript

# One-shot
async for event in tts.synthesize("Hello, this is a complete sentence."):
    ...  # event.frame is a LiveKit AudioFrame

# Streaming, e.g. from an LLM token stream
stream = tts.stream()
stream.push_text("Hello, ")
stream.push_text("how are you today?")
stream.flush()
stream.end_input()
async for event in stream:
    ...  # event.frame

// One-shot
for await (const event of tts.synthesize('Hello, this is a complete sentence.')) {
  // event.frame is a LiveKit AudioFrame
}

// Streaming, e.g. from an LLM token stream
const stream = tts.stream();
stream.pushText('Hello, ');
stream.pushText('how are you today?');
stream.flush();
stream.endInput();
for await (const event of stream) {
  // event.frame
}

Troubleshooting

MISSING_VOICE_ID

The plugin has no default voice. Pass a voice ID in the constructor or set it at runtime before the first turn. Browse voices in Voices.

API key not found

Set KUGELAUDIO_API_KEY in the environment or pass the API key option to the constructor.

First response is slow

Call prewarm() during call setup, not right before the first turn
Create one TTS instance per worker and reuse it; do not construct one per phrase
Measure from the same region as the API before changing synthesis parameters, see Latency

Speed or temperature change did not apply

Both are session-wide: a change binds for turns started after it, and a turn already playing finishes with the old value. Values outside the accepted range raise instead of being clamped, so check for an exception at the call site.

WebSocket connection fails

Check the base URL option and that the API is reachable from the worker over wss://. After aclose() (Python) or close() (TypeScript), create a new instance for the next session.

No word timestamps arrive

Word timestamps are off by default, so check that the option is on. If it is, alignment failed server-side for that chunk: the server logs it and skips the timestamps, and the audio is unaffected.

Next steps

Voice prompting

Prompt patterns that make an LLM sound good when spoken

Streaming best practices

Chunking, flushing and latency for LLM-driven agents

Barge-in

What happens to the current turn when the caller interrupts

Pipecat integration

The same plugin idea for Pipecat pipelines

Get started

Build a voice agent

Generate audio

Voices

Enhance audio

Hosting and deployment

SDK Reference

API Reference

1. Install

2. Add it to your agent

3. Tune the sound

4. Word timestamps (optional)

5. Remove background noise (optional)

Reference

Options

Sample rates

Environment variables

Using the plugin without AgentSession

Troubleshooting

Next steps

Voice prompting

Streaming best practices

Barge-in

Pipecat integration

​1. Install

​2. Add it to your agent

​3. Tune the sound

​4. Word timestamps (optional)

​5. Remove background noise (optional)

​Reference

​Options

​Sample rates

​Environment variables

​Using the plugin without AgentSession

​Troubleshooting

​Next steps

Voice prompting

Streaming best practices

Barge-in

Pipecat integration

1. Install

2. Add it to your agent

3. Tune the sound

4. Word timestamps (optional)

5. Remove background noise (optional)

Reference

Options

Sample rates

Environment variables

Using the plugin without AgentSession

Troubleshooting

Next steps