Speech to text

Speech to text turns speech in nine European languages into text with Kugel Ear 1 (kugel-ear-1). Send a finished recording and get its transcript back, or stream a live call and read the words while the person is still speaking.

Choose how to send audio

A stream sends revisable partial transcripts while the person speaks, then the final transcript of each turn, and shortly after it a more accurate revision of the same turn; Stream a live call shows how to read them.

Quick start

Every request needs an API key allowed to use Speech to text.

Transcribe a recording

from kugelaudio import KugelAudio

client = KugelAudio(api_key="YOUR_API_KEY")

with open("call.wav", "rb") as f:
    result = client.asr.transcribe(f.read())

print(result.text, result.language)

import { readFile } from 'node:fs/promises';
import { KugelAudio } from 'kugelaudio';

const client = new KugelAudio({ apiKey: 'YOUR_API_KEY' });

const audio = new Blob([await readFile('call.wav')], { type: 'audio/wav' });
const result = await client.asr.transcribe({ audio, filename: 'call.wav' });
console.log(result.text, result.language);

KugelAudio client = KugelAudio.fromEnv(); // reads KUGELAUDIO_API_KEY
TranscriptionResponse result = client.asr().transcribe(Files.readAllBytes(Path.of("call.wav")));
System.out.println(result.getText() + " " + result.getLanguage());

curl https://api.kugelaudio.com/v1/audio/transcriptions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "file=@call.wav"

Uploads can be WAV, FLAC, Ogg or most other common container formats, at 8 to 48 kHz. Every argument and result field is in the SDK reference.

Stream a live call

client.asr.stream opens the stream, sends your audio as it goes and yields every transcript the server sends back. This example streams a file, so you can try it without a live source:

# pip install -U kugelaudio
import asyncio
from kugelaudio import KugelAudio, load_audio_stream

client = KugelAudio(api_key="YOUR_API_KEY")

async def main():
    audio = load_audio_stream("call.wav")  # mono, 16-bit
    async for event in client.asr.stream(audio):  # ends after the last turn
        if event.revises_final:
            print("revised:", event.partial_text, event.language)
        elif event.is_final:
            print("final:", event.partial_text, event.turn_end_reason)
        elif event.type == "partial":
            print("partial:", event.partial_text)

asyncio.run(main())

// npm install kugelaudio
import { KugelAudio, loadAudioStream } from 'kugelaudio';

const client = new KugelAudio({ apiKey: 'YOUR_API_KEY' });

const audio = await loadAudioStream('call.wav'); // mono, 16-bit
for await (const event of client.asr.stream(audio)) { // ends after the last turn
  if (event.revises_final) console.log('revised:', event.partial_text, event.language);
  else if (event.is_final) console.log('final:', event.partial_text, event.turn_end_reason);
  else if (event.type === 'partial') console.log('partial:', event.partial_text);
}

// No streaming client in the Java SDK yet: open the
// /v1/audio/transcriptions/stream WebSocket as the SDK reference shows.

Each turn of the call reaches you as a sequence of events. Check them in the order of this table, as the example does:

Each partial_text is the whole turn so far, not the words added since the last event. Word alternatives for the turn arrive in an extra alternatives event after the final; see Using word alternatives.

The connection stays open across turns, so one stream covers the whole call. To transcribe live audio, such as a microphone or a phone call, pass its mono 16-bit PCM chunks as they arrive, as an iterable or async iterable, together with their sample rate:

async for event in client.asr.stream(
    microphone_chunks(),  # any iterable or async iterable of PCM16 bytes
    sample_rate=16000,
    language=["de", "en"],
):
    ...

for await (const event of client.asr.stream(microphoneChunks(), { // any (async) iterable of PCM16 chunks
  sampleRate: 16000,
  language: ['de', 'en'],
})) {
  // ...
}

// No streaming client in the Java SDK yet: send the sample rate and language
// in the config frame of the /v1/audio/transcriptions/stream WebSocket.

When your source ends, the SDK tells the server so; the last turn ends and the iteration finishes with it. The other stream options, such as turn detection and boosted phrases, are in the SDK reference; without an SDK, the WebSocket protocol describes the same frames.

Languages

Kugel Ear 1 transcribes these languages:

Choose the language with language, the same two-letter code the text to speech API takes:

• One code, such as "fr": the audio is transcribed in that language.
• A list, such as ["de", "fr", "it"]: the model detects which of these is spoken.
• "auto" or nothing: the model detects whether German or English is spoken, and only those two.

So for any language other than German and English, send its code or include it in the list; otherwise the audio is transcribed as German or English. Detection runs once per recording for an upload, and per turn on a stream, so a conversation that switches language between turns is followed. An unknown code such as "ja", or a form such as "de-DE" or "German", is refused rather than ignored. The result reports the language it was transcribed in as its code, such as "fr".

On a stream, the live partial transcripts are most accurate in English, German, Spanish and French. The revision of each turn, the more accurate transcript that follows its final, is transcribed in the language you chose.

KugelTurn ends turns in every one of these languages. On LiveKit's open end-of-turn benchmark it takes fewer of a speaker's pauses for the end of the turn than any published model in each language the benchmark shares with Kugel Ear 1: German, English, French, Italian, Spanish, Portuguese and Dutch. Polish and Romanian are not in the benchmark. The numbers per language are on the turn detection benchmark.

Word alternatives

Names often have several spellings that sound the same (Catherine, Katherine, Kathryn). For a word like that, the result lists the likely spellings with their probabilities, most likely first. Pass them to the language model behind your agent along with the transcript, so it can ask the caller about the one word in doubt instead of asking them to spell everything. The list is conservative and never delays or replaces the transcript; see Using word alternatives.

End of turn with KugelTurn

On a stream, you send the audio as it is, pauses included, and the server ends each turn when the speaker has finished. By default KugelTurn, our end-of-turn model, decides from a short pause, and a pause it still holds after a few seconds ends the turn anyway. The config frame can choose a conservative level that holds longer through a pause mid-thought, a fixed pause instead of KugelTurn, or turns that end only when you say so. The options are in Client frames.

Custom vocabulary

Tell the recognizer which names, brands and terms to expect, and it writes them your way where the audio fits them:

• Boosted phrases you send with a request or a stream's config frame.
• Your dictionaries, the same ones that shape text to speech: the written word of every entry applies to transcription too, without sending anything. Choose dictionaries per request, or none.

A phrase is a hint, not a replacement rule. List what speakers in this audio may actually say: the closer the list is to the call, the more it helps. Limits and examples are in Custom vocabulary.

Access, billing and limits

• Speech to text has to be enabled for your organization; until it is, requests are refused with 403 and Transcription is not enabled for this organization.
• The API key needs the Speech to text permission.
• Usage is billed from your organization's credits; see pricing. A spent balance is refused with 402 INSUFFICIENT_CREDITS.
• An upload, and a turn of a stream, can be up to 300 seconds long.
• No word timestamps and no speaker separation (diarization) today.

Every status and close code is in the API reference; the shared error format is in Error codes.

Next steps