> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kugelaudio.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Answer from your backend

> Look up what the caller asks for in your own systems and have the agent tell them the answer.

When a turn needs your systems, you get `session.delegation.created`. It carries an id and no
text, so keep the transcript yourself. Do the work in a task of its own so audio keeps flowing,
then send the result under that id. The examples on this page replace functions in `talk.py`
from the [quickstart](/speech-to-speech/quickstart).

## Send the result

Find the order in what the caller said and send the facts back as commentary. The agent says
them in its own words. Callers say numbers as words, so this lookup reads both.

```python
import re

ORDERS = {"4711": "Order 4711 shipped yesterday and arrives tomorrow."}
DIGITS = {"zero": "0", "one": "1", "two": "2", "three": "3", "four": "4",
          "five": "5", "six": "6", "seven": "7", "eight": "8", "nine": "9"}


def look_up_order(caller_said: str) -> str:
    words = re.findall(r"[a-z0-9]+", caller_said.lower())
    number = "".join(DIGITS.get(word, word) for word in words if word in DIGITS or word.isdigit())
    return ORDERS.get(number, "There is no order with that number. Ask the caller to repeat it.")
```

Send facts, not finished sentences. From "Order 4711 arrives tomorrow", the agent makes a reply
that fits the conversation.

## Choose how the agent uses what you send

| Send | The agent |
|---|---|
| `connection.session.commentary.append` | Tells the caller. Send results here. |
| `connection.session.thinking.append` | Knows it and does not say it. Send progress here. |
| `connection.session.instructions.append` | Changes how it behaves from now on. |

Each takes `content` of at most 500 tokens and a `delegation_id`, the id of the delegation it
belongs to. These are [OpenAI's append events](https://developers.openai.com/api/docs/guides/live-delegation).

## Use the caller's whole turn

`session.input_transcript.delta` streams words while the caller speaks. For a lookup, use the
final text of the turn instead. Add `"kugel": {}` to the session and keep the last
`kugel.input_transcript.committed`, which arrives before the delegation:

```python
turn = ""

async for event in connection:
    if event.type == "kugel.input_transcript.committed":
        turn = event.text
    elif event.type == "session.delegation.created":
        tasks.add(asyncio.create_task(answer(connection, event.delegation.id, turn)))
```

## Report progress on slow work

The agent tells the caller it is checking, then waits for your result. While a slow lookup runs,
send progress as thinking so the agent knows what is happening without reading it out:

```python
async def answer(connection, delegation_id: str, caller_said: str) -> None:
    await connection.session.thinking.append(
        delegation_id=delegation_id,
        content="Checking the order now. Nothing is confirmed yet.",
    )
    await asyncio.sleep(5)  # your slow lookup
    result = await asyncio.to_thread(look_up_order, caller_said)
    await connection.session.commentary.append(delegation_id=delegation_id, content=result)
```

## Ask the caller a question

To ask something and carry on with the answer, send a commentary with `expects_reply`. The agent
says it word for word, and the caller's next finished turn opens a new delegation. The SDK's
`append` has no argument for it, so send the event yourself.

This `answer` confirms before it cancels an order:

```python
confirming = False


async def answer(connection, delegation_id: str, caller_said: str) -> None:
    global confirming
    if not confirming:
        confirming = True
        await connection.send(
            {
                "type": "session.commentary.append",
                "delegation_id": delegation_id,
                "content": "Shall I cancel order 4711?",
                "expects_reply": True,
            }
        )
        return
    confirming = False
    said_yes = "yes" in re.findall(r"[a-z]+", caller_said.lower())[-6:]
    await connection.session.commentary.append(
        delegation_id=delegation_id,
        content="Order 4711 is cancelled." if said_yes else "Order 4711 stays as it is.",
    )
```

## Set the greeting and what the agent knows

The agent speaks first. Tell it what to open with, and what you already know about the caller,
in `instructions`:

```python
session = {
    "model": "fluid-1",
    "instructions": (
        "You answer the phone for a parcel service. Keep replies short. "
        "Open every call with exactly: Good morning, parcel service, this is Mia. "
        "The caller has one open order, number 4711."
    ),
    "delegation": {"type": "client"},
}
```

## Say something at any time

Send a commentary with no delegation, and the agent says it word for word at the next
opportunity:

```python
await connection.session.commentary.append(
    delegation_id=None,
    content="A quick note: our depot closes at six today.",
)
```

If the caller finishes a turn before the agent has said it, the line is dropped, because that
turn may have changed the subject.

## Change how the agent behaves

Add to the agent's instructions during the call, for example when your system learns something
that changes the rules:

```python
await connection.session.instructions.append(
    delegation_id=None,
    content="The caller's identity is confirmed. You can now give delivery details.",
)
```

## Choose what the agent delegates

By default the agent delegates the work your instructions list for a back office or tools, or,
when they list none, anything it needs your backend for. To choose exactly, describe your tools
in `session.kugel.tools`. With `[]` the agent never delegates. Your code still runs the work.

```python
session = {
    "model": "fluid-1",
    "instructions": "You answer the phone for a parcel service. Keep replies short.",
    "delegation": {"type": "client"},
    "kugel": {
        "tools": [
            {
                "name": "find_order",
                "description": "Find an order and where it is by its number.",
                "parameters": {"type": "object", "properties": {"number": {"type": "string"}}},
            }
        ]
    },
}
```

The agent decides when a turn needs delegating, so test your calls with the instructions and
tools you plan to use.

## Next steps

<CardGroup cols={2}>
  <Card title="Let KugelAudio run the back office" icon="robot" href="/speech-to-speech/back-office">
    List your tools and have a model use them
  </Card>
  <Card title="Control a call" icon="sliders" href="/speech-to-speech/during-a-call">
    Handle interruptions, pick a voice, mute and restart
  </Card>
</CardGroup>