> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kugelaudio.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Speech to Text in the SDKs
> SDK reference for speech to text with kugel-ear-1: transcribe a recording with client.asr in Python, TypeScript and Java, and stream live audio with client.asr.stream in Python and TypeScript.
`client.asr.transcribe` sends a complete recording to `kugel-ear-1` and returns
its transcript, its language and spelling alternatives for uncertain words;
`client.asr.stream` streams live audio and yields the transcript as it is heard.
What the model does is in the [Speech to text guide](/features/speech-to-text);
the wire protocol, the limits and every error are in the
[API reference](/api-reference/endpoints/speech-to-text).
`client.asr.transcribe` is a blocking call; there is no async variant yet.
`client.asr.stream` is an async iterator, and `client.asr.stream_sync` its
blocking twin.
`client.asr.transcribe` takes the recording as a `Blob` (a `File` is one).
`client.asr.stream` is an async generator, in Node.js and in the browser.
`client.asr().transcribe` is a blocking call. There is no streaming client
yet; open the WebSocket yourself, as [Stream live audio](#stream-live-audio)
shows.
Not available in the Rust SDK yet; use the REST and WebSocket endpoints in the
[Speech to text API reference](/api-reference/endpoints/speech-to-text).
Not available in the C++ SDK yet; use the REST and WebSocket endpoints in the
[Speech to text API reference](/api-reference/endpoints/speech-to-text).
Not available in the Dart SDK yet; use the REST and WebSocket endpoints in the
[Speech to text API reference](/api-reference/endpoints/speech-to-text).
## Transcribe a file
```python Python
from kugelaudio import KugelAudio
client = KugelAudio(api_key="YOUR_API_KEY")
with open("call.wav", "rb") as f:
result = client.asr.transcribe(f.read())
print(result.text) # the transcript
print(result.language) # language code, such as "de" or "fr"
print(result.duration_s) # audio duration in seconds
for word in result.word_alternatives:
print(word.word, [alt.spelling for alt in word.alternatives])
```
```typescript TypeScript
import { readFile } from 'node:fs/promises';
import { KugelAudio } from 'kugelaudio';
const client = new KugelAudio({ apiKey: 'YOUR_API_KEY' });
const audio = new Blob([await readFile('call.wav')], { type: 'audio/wav' });
const result = await client.asr.transcribe({ audio, filename: 'call.wav' });
console.log(result.text); // the transcript
console.log(result.language); // language code, such as 'de' or 'fr'
console.log(result.duration_s); // audio duration in seconds
for (const word of result.word_alternatives) {
console.log(word.word, word.alternatives.map((alt) => alt.spelling));
}
```
```java Java
import com.kugelaudio.sdk.KugelAudio;
import com.kugelaudio.sdk.TranscriptionResponse;
import java.nio.file.Files;
import java.nio.file.Path;
KugelAudio client = KugelAudio.fromEnv(); // reads KUGELAUDIO_API_KEY
byte[] audio = Files.readAllBytes(Path.of("call.wav"));
TranscriptionResponse result = client.asr().transcribe(audio);
System.out.println(result.getText()); // the transcript
System.out.println(result.getLanguage()); // language code, such as "de" or "fr"
System.out.println(result.getDurationSeconds()); // audio duration in seconds
for (TranscriptionResponse.WordAlternatives word : result.getWordAlternatives()) {
System.out.println(word.word() + " " + word.alternatives());
}
client.close();
```
```rust Rust
// Not available in the Rust SDK yet: POST /v1/audio/transcriptions
// (see the Speech to text API reference).
```
```cpp C++
// Not available in the C++ SDK yet: POST /v1/audio/transcriptions
// (see the Speech to text API reference).
```
```dart Dart
// Not available in the Dart SDK yet: POST /v1/audio/transcriptions
// (see the Speech to text API reference).
```
The recording can be WAV, FLAC, Ogg or most other common container formats, at
8 to 48 kHz, up to 300 seconds; see
[Audio limits](/api-reference/endpoints/speech-to-text#audio-limits). Without a
`language`, the model detects German or English. For any other
[supported language](/features/speech-to-text#languages), pass its code, such as
`language="fr"` in Python, `language: 'fr'` in TypeScript, or `"fr"` as the
`language` argument in Java. The SDKs send one code; to let the model detect
the language among several, send a list to the
[REST endpoint](/api-reference/endpoints/speech-to-text#supported-languages)
directly.
### Your vocabulary
Names, brands and domain terms you pass as boosted phrases come out in your
spelling. The words of your project's [dictionaries](/features/dictionaries)
are added after them without sending anything; choose dictionaries per request,
or none. How phrases work and their limits are in
[Custom vocabulary](/api-reference/endpoints/speech-to-text#custom-vocabulary).
```python Python
result = client.asr.transcribe(
audio_bytes,
filename="call.flac",
content_type="audio/flac",
boosted_phrases=["Kratzenstein", "KugelAudio", "Mayrhofer"],
dictionary_ids=[12], # only this dictionary; [] applies none
)
```
```typescript TypeScript
const result = await client.asr.transcribe({
audio,
filename: 'call.flac',
boostedPhrases: ['Kratzenstein', 'KugelAudio', 'Mayrhofer'],
dictionaryIds: [12], // only this dictionary; [] applies none
});
```
```java Java
TranscriptionResponse result = client.asr().transcribe(
audio, "call.flac", "audio/flac", null, null,
List.of("Kratzenstein", "KugelAudio", "Mayrhofer"),
null, // projectId: the API key's project
List.of(12)); // only this dictionary; List.of() applies none
```
```rust Rust
// Not available in the Rust SDK yet: send boosted_phrases and dictionary_ids
// to POST /v1/audio/transcriptions.
```
```cpp C++
// Not available in the C++ SDK yet: send boosted_phrases and dictionary_ids
// to POST /v1/audio/transcriptions.
```
```dart Dart
// Not available in the Dart SDK yet: send boosted_phrases and dictionary_ids
// to POST /v1/audio/transcriptions.
```
### `transcribe`
```python
def transcribe(
audio: bytes,
*,
filename: str = "audio.wav",
content_type: str = "audio/wav",
language: str | None = None,
model: str | None = None,
boosted_phrases: list[str] | None = None,
project_id: int | None = None,
dictionary_ids: list[int] | None = None,
) -> TranscriptionResponse
```
| Argument | Type | Description |
|---|---|---|
| `audio` | `bytes` | The complete recording file. Empty audio raises `ValidationError` before any request. |
| `filename`, `content_type` | `str` | Sent with the upload. Set them for a container other than WAV. |
| `language` | `str \| None` | A two-letter code such as `"fr"` to transcribe in that language, or `"auto"`. Omitted: German or English, detected. See [Supported languages](/api-reference/endpoints/speech-to-text#supported-languages). |
| `model` | `str \| None` | Omit it, or pass `"kugel-ear-1"`. |
| `boosted_phrases` | `list[str] \| None` | Your vocabulary for this request. |
| `project_id` | `int \| None` | The project whose dictionaries apply. Omitted: the API key's project. |
| `dictionary_ids` | `list[int] \| None` | Omitted: every active dictionary of the project. `[]`: none. |
Returns a [`TranscriptionResponse`](#transcriptionresponse).
```typescript
transcribe(options: TranscribeOptions): Promise
```
| Option | Type | Description |
|---|---|---|
| `audio` | `Blob` | Required. The complete recording file. |
| `filename` | `string` | Sent with the upload (`'audio.wav'` when omitted). |
| `language` | `string` | A two-letter code such as `'fr'` to transcribe in that language, or `'auto'`. Omitted: German or English, detected. See [Supported languages](/api-reference/endpoints/speech-to-text#supported-languages). |
| `model` | `string` | Omit it, or pass `'kugel-ear-1'`. |
| `boostedPhrases` | `string[]` | Your vocabulary for this request. |
| `projectId` | `number` | The project whose dictionaries apply. Omitted: the API key's project. |
| `dictionaryIds` | `number[]` | Omitted: every active dictionary of the project. `[]`: none. |
Returns a [`TranscriptionResponse`](#transcriptionresponse).
```java
TranscriptionResponse transcribe(byte[] audio)
TranscriptionResponse transcribe(byte[] audio, String filename, String contentType,
String language, String model)
TranscriptionResponse transcribe(byte[] audio, String filename, String contentType,
String language, String model, List boostedPhrases)
TranscriptionResponse transcribe(byte[] audio, String filename, String contentType,
String language, String model, List boostedPhrases,
Long projectId, List dictionaryIds)
```
| Argument | Type | Description |
|---|---|---|
| `audio` | `byte[]` | The complete recording file. Empty audio throws `ValidationException` before any request. |
| `filename`, `contentType` | `String` | Sent with the upload; the one-argument form sends `"audio.wav"`, `"audio/wav"`. |
| `language` | `String` | A two-letter code such as `"fr"` to transcribe in that language, or `"auto"`. `null`: German or English, detected; the one-argument form sends `null`. See [Supported languages](/api-reference/endpoints/speech-to-text#supported-languages). |
| `model` | `String` | `null` to leave it out, or `"kugel-ear-1"`. |
| `boostedPhrases` | `List` | Your vocabulary for this request. Must not be `null`; pass `List.of()` for none. |
| `projectId` | `Long` | The project whose dictionaries apply. `null`: the API key's project. |
| `dictionaryIds` | `List` | `null`: every active dictionary of the project. An empty list: none. |
Returns a [`TranscriptionResponse`](#transcriptionresponse).
Not available in the Rust SDK yet; see
[Upload a complete recording](/api-reference/endpoints/speech-to-text#upload-a-complete-recording)
in the API reference.
Not available in the C++ SDK yet; see
[Upload a complete recording](/api-reference/endpoints/speech-to-text#upload-a-complete-recording)
in the API reference.
Not available in the Dart SDK yet; see
[Upload a complete recording](/api-reference/endpoints/speech-to-text#upload-a-complete-recording)
in the API reference.
## Stream live audio
`client.asr.stream` opens the
[WebSocket](/api-reference/endpoints/speech-to-text#stream-live-audio), sends
your audio as mono 16-bit PCM while it reads, and yields every frame the server
sends back: the partials, the final of each turn and its revision. Each example
streams a file; a live source such as a microphone passes its chunks the same way
as they arrive, as an iterable or async iterable with the sample rate. The Java
SDK has no streaming client yet, so its example opens the socket itself and
parses the frames with the SDK's types.
```python Python
import asyncio
from kugelaudio import KugelAudio, load_audio_stream
client = KugelAudio(api_key="YOUR_API_KEY")
async def main():
audio = load_audio_stream("call.wav") # mono, 16-bit
async for event in client.asr.stream(audio): # ends after the last turn
if event.revises_final:
print("revised:", event.partial_text, event.language)
elif event.is_final:
print("final:", event.partial_text, event.turn_end_reason)
elif event.type == "partial":
print("partial:", event.partial_text)
asyncio.run(main())
```
```typescript TypeScript
import { KugelAudio, loadAudioStream } from 'kugelaudio';
const client = new KugelAudio({ apiKey: 'YOUR_API_KEY' });
const audio = await loadAudioStream('call.wav'); // mono, 16-bit
for await (const event of client.asr.stream(audio)) { // ends after the last turn
if (event.revises_final) console.log('revised:', event.partial_text, event.language);
else if (event.is_final) console.log('final:', event.partial_text, event.turn_end_reason);
else if (event.type === 'partial') console.log('partial:', event.partial_text);
}
```
```java Java
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kugelaudio.sdk.StreamingTranscriptionResult;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.WebSocket;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.Base64;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
ObjectMapper json = new ObjectMapper();
CompletableFuture closed = new CompletableFuture<>();
WebSocket.Listener listener = new WebSocket.Listener() {
private final StringBuilder message = new StringBuilder();
@Override
public CompletionStage onText(WebSocket ws, CharSequence data, boolean last) {
message.append(data);
if (last) {
try {
JsonNode frame = json.readTree(message.toString());
if (frame.has("error")) {
System.err.println(frame.path("error_code").asText() + " " + frame.path("error").asText());
} else {
StreamingTranscriptionResult event = json.treeToValue(frame, StreamingTranscriptionResult.class);
if (frame.path("revises_final").asBoolean()) {
System.out.println("revised: " + event.partialText() + " " + event.language());
} else if (event.isFinal()) {
System.out.println("final: " + event.partialText() + " " + event.turnEndReason());
} else if ("partial".equals(event.type())) {
System.out.println("partial: " + event.partialText());
}
}
} catch (Exception e) {
closed.completeExceptionally(e);
}
message.setLength(0);
}
ws.request(1);
return null;
}
@Override
public CompletionStage onClose(WebSocket ws, int statusCode, String reason) {
closed.complete(null); // after the last turn
return null;
}
@Override
public void onError(WebSocket ws, Throwable error) {
closed.completeExceptionally(error);
}
};
WebSocket ws = HttpClient.newHttpClient().newWebSocketBuilder()
.header("Authorization", "Bearer " + System.getenv("KUGELAUDIO_API_KEY"))
.buildAsync(URI.create("wss://api.kugelaudio.com/v1/audio/transcriptions/stream"), listener)
.join();
ws.sendText(json.writeValueAsString(Map.of("type", "config", "sample_rate", 16000)), true).join();
byte[] pcm = Files.readAllBytes(Path.of("call.pcm")); // raw mono PCM16 at 16 kHz
for (int i = 0; i < pcm.length; i += 3200) { // 0.1 s per chunk
byte[] chunk = Arrays.copyOfRange(pcm, i, Math.min(i + 3200, pcm.length));
ws.sendText(json.writeValueAsString(Map.of(
"type", "audio_chunk",
"audio_b64", Base64.getEncoder().encodeToString(chunk))), true).join();
}
ws.sendText("{\"type\": \"end_of_speech\"}", true).join();
closed.join();
```
```rust Rust
// Not available in the Rust SDK yet: open the /v1/audio/transcriptions/stream
// WebSocket (see the Speech to text API reference).
```
```cpp C++
// Not available in the C++ SDK yet: open the /v1/audio/transcriptions/stream
// WebSocket (see the Speech to text API reference).
```
```dart Dart
// Not available in the Dart SDK yet: open the /v1/audio/transcriptions/stream
// WebSocket (see the Speech to text API reference).
```
The server ends each turn when the speaker finishes, as KugelTurn hears it, and
the connection stays open for the next turn; after your last chunk the SDK sends
`end_of_speech`, which ends the last one, and the iteration ends with it. A
failure raises the same typed errors as every other call (see [Errors](#errors)).
Pass `language`, `turn_detection`, `turn_detection_level`, `boosted_phrases`,
`dictionary_ids` or `project_id` to `stream` as the
[client frames](/api-reference/endpoints/speech-to-text#client-frames) describe;
on a stream, `language` can be one code or a list of codes.
```python Python
async for event in client.asr.stream(
microphone_chunks(), # any iterable or async iterable of PCM16 bytes
sample_rate=16000,
language=["de", "fr"],
boosted_phrases=["Kratzenstein", "KugelAudio"],
dictionary_ids=[12],
):
...
```
```typescript TypeScript
for await (const event of client.asr.stream(microphoneChunks(), { // any (async) iterable of PCM16 chunks
sampleRate: 16000,
language: ['de', 'fr'],
boostedPhrases: ['Kratzenstein', 'KugelAudio'],
dictionaryIds: [12],
})) {
// ...
}
```
```java Java
// No streaming client in the Java SDK yet: add the options to the config frame
// of the example above.
```
```rust Rust
// Not available in the Rust SDK yet: add the options to the config frame of the
// /v1/audio/transcriptions/stream WebSocket.
```
```cpp C++
// Not available in the C++ SDK yet: add the options to the config frame of the
// /v1/audio/transcriptions/stream WebSocket.
```
```dart Dart
// Not available in the Dart SDK yet: add the options to the config frame of the
// /v1/audio/transcriptions/stream WebSocket.
```
A partial is a guess to replace, the final is the turn's result, and the revision that may
follow it is a more accurate transcript of the same turn that also carries the
code of the language it was transcribed in.
## `TranscriptionResponse`
What `transcribe` returns. Field by field it is the
[upload response](/api-reference/endpoints/speech-to-text#response-fields).
```python
class TranscriptionResponse:
text: str # the transcript (OpenAI-compatible field)
transcript: str # the same text
language: str # language code, such as "de" or "fr"
duration_s: float # audio duration in seconds
model: str # "kugel-ear-1"
model_revision: str | None # immutable revision of the model build
word_alternatives: list[WordAlternatives] # spellings for uncertain words
raw: dict # the response body as received
class WordAlternatives:
raw_word_index: int # position of the word in transcript
word: str # the word as written
alternatives: list[SpellingAlternative] # most likely first
class SpellingAlternative:
spelling: str
probability: float # the alternatives of one word sum to 1
```
The fields keep the API's snake_case names.
```typescript
interface TranscriptionResponse {
text: string; // the transcript (OpenAI-compatible field)
transcript: string; // the same text
language: string; // language code, such as 'de' or 'fr'
duration_s: number; // audio duration in seconds
model: string; // 'kugel-ear-1'
model_revision?: string; // immutable revision of the model build
word_alternatives: WordAlternatives[]; // spellings for uncertain words
[key: string]: unknown; // any other field of the response
}
interface WordAlternatives {
raw_word_index: number; // position of the word in transcript
word: string; // the word as written
alternatives: SpellingAlternative[]; // most likely first
}
interface SpellingAlternative {
spelling: string;
probability: number; // the alternatives of one word sum to 1
}
```
```java
TranscriptionResponse result;
String text = result.getText(); // the transcript (OpenAI-compatible field)
String transcript = result.getTranscript(); // the same text
String language = result.getLanguage(); // language code, such as "de" or "fr"
double seconds = result.getDurationSeconds(); // audio duration
String model = result.getModel(); // "kugel-ear-1"
String modelRevision = result.getModelRevision(); // immutable revision of the model build
List words = result.getWordAlternatives();
record WordAlternatives(int rawWordIndex, String word, List alternatives) {}
record SpellingAlternative(String spelling, double probability) {}
```
Not available in the Rust SDK yet.
Not available in the C++ SDK yet.
Not available in the Dart SDK yet.
### Streaming frames
One parsed frame of the [stream](#stream-live-audio): a partial, a final, or an
`alternatives` frame after the final.
`client.asr.stream` yields one `StreamingTranscriptionEvent` per frame;
`StreamingTranscriptionEvent.from_dict(frame)` parses a frame you read yourself:
```python
class StreamingTranscriptionEvent:
partial_text: str # complete rolling hypothesis; replace the previous one
is_final: bool # True on the turn's final
type: str # "partial" or "alternatives"
turn_end_reason: str | None # on the final: why the turn ended
turn_end_confidence: float | None
turn_end_inference_ms: float | None
word_alternatives: list[WordAlternatives]
model: str | None # on the final and alternatives frames
model_revision: str | None
language: str | None # on the turn's revision; None on partials
revises_final: bool # True on the turn's revision
```
```typescript
interface StreamingTranscriptionResult {
type: 'partial' | 'alternatives';
partial_text: string; // complete rolling hypothesis; replace the previous one
is_final: boolean; // true on the turn's final
model?: string; // on the final and alternatives frames
model_revision?: string;
language?: string; // on the turn's revision; absent on partials
turn_end_reason?: 'client_end_of_speech' | 'model_end_of_turn' | 'silence_timeout' | 'max_duration';
turn_end_confidence?: number;
turn_end_inference_ms?: number;
word_alternatives: WordAlternatives[];
revises_final?: boolean; // true on the turn's revision
[key: string]: unknown; // any other field of the frame
}
```
`client.asr.stream` yields one `StreamingTranscriptionResult` per frame. Its
options are `TranscriptionStreamOptions`; `StreamingTranscriptionConfig` and
`StreamingAudioChunk` type the frames on the wire.
Jackson reads a frame into the record:
```java
record StreamingTranscriptionResult(
String type, // "partial" or "alternatives"
String partialText, // complete rolling hypothesis; replace the previous one
boolean isFinal, // true on the turn's final
String model, // on the final and alternatives frames
String modelRevision,
String turnEndReason, // on the final: why the turn ended
Double turnEndConfidence,
Double turnEndInferenceMs,
List wordAlternatives,
String language) {} // on the turn's revision; null on partials
```
`revises_final` is not a component; read it from the frame itself, as the
example above does.
Not available in the Rust SDK yet.
Not available in the C++ SDK yet.
Not available in the Dart SDK yet.
## Errors
`transcribe` and `stream` raise the same errors as every other SDK call, for example
`AuthenticationError` for a rejected key, `InsufficientCreditsError` for a spent
balance and `RateLimitError` for `429` (in Java `AuthenticationException`,
`InsufficientCreditsException` and `RateLimitException`); see
[Types and errors](/sdks/types#errors).
What each status means for speech to text is in the
[error table](/api-reference/endpoints/speech-to-text#errors).
## Next steps
- [Speech to text guide](/features/speech-to-text): what the model does, turn detection, vocabulary
- [Speech to text API](/api-reference/endpoints/speech-to-text): the wire protocol, limits and close codes
- [Dictionaries](/sdks/dictionaries): manage the dictionaries that also teach the recognizer