Limits and usage

Limits

Rate limits

Speech to speech has the same per-organization rate limits and plan tiers as text to speech, counted separately: a session takes nothing from your speech generation limits, and a speech generation takes nothing from your sessions. A session holds one concurrent slot and one open connection until it ends, and takes one request from the per-minute limit when it starts.

Retry a refused session

A refused session gets an error event with the code rate_limit_exceeded, then close code 1013. Wait and open a new session. Depending on when the refusal arrives, the SDK raises it from session.start or from the event loop, so catch both:

import asyncio
import os
import random

from openai import AsyncOpenAI, WebSocketConnectionClosedError
from websockets.exceptions import ConnectionClosedError

CLOSED = (ConnectionClosedError, WebSocketConnectionClosedError)


def asked_to_retry(error: Exception) -> bool:
    closed = error if isinstance(error, ConnectionClosedError) else error.__cause__
    return isinstance(closed, ConnectionClosedError) and closed.rcvd is not None and closed.rcvd.code == 1013


async def run_session(client: AsyncOpenAI) -> None:
    async with client.live.connect() as connection:
        await connection.session.start(session={"model": "fluid-1"})
        async for event in connection:
            if event.type == "session.started":
                print("Session started")
                await connection.session.close()
            elif event.type == "session.closed":
                return


async def main() -> None:
    async with AsyncOpenAI(
        api_key=os.environ["KUGELAUDIO_API_KEY"],
        base_url="https://api.kugelaudio.com/v1",
    ) as client:
        for attempt in range(6):
            try:
                return await run_session(client)
            except CLOSED as error:
                if not asked_to_retry(error):
                    raise
                delay = min(2**attempt, 30) + random.random()
                print(f"Refused, retrying in {delay:.1f} s")
                await asyncio.sleep(delay)
        raise SystemExit("Still refused after 6 attempts")


if __name__ == "__main__":
    asyncio.run(main())

The same close code also means the server is restarting, and a retry is right for that too.

Usage

Usage starts with the first audio in either direction and runs until the session ends, in exact seconds. A socket waiting for session.start costs nothing. Time counts whether anyone is speaking or not. session.usage.updated and session.closed report the seconds so far. A delegated model you run yourself is yours to pay for. In hosted delegation the service runs the model, and the tokens its calls use (input, cached input and output) are charged in addition to the seconds, at that model's token price. The events report seconds only. For rates, see pricing.

When the credits run out during a call, you get an error event with the code billing_limit_reached, and the socket closes with 1008.

Recordings during the beta

During the beta, KugelAudio records calls made with an API key or from the dashboard so it can debug the model. A recording holds both sides of the audio, your session configuration including its instructions, and the session's events and transcripts. Recordings are not used for training. They are deleted after 90 days, and backups keep them for up to about 60 more days.

Next steps