> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kugelaudio.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Goals and A/B experiments
> Declare in your agent's code what counts as a successful call and which variants to compare, read a call's variant with useVariant, try each variant locally, and see per call which goals it reached.
A goal says what counts as a successful call, for example "the caller booked a slot". An
experiment splits calls between variants of your agent, for example a formal and a casual
greeting, so you can see which variant reaches your goals more often. You declare both in
your agent's code, next to the agent itself, and they go live with your next
[deployment](/hosted-agents/deploy).
## Declare goals and experiments
Add `goals` and `experiments` to `defineAgent` in `src/agent/agent.tsx`:
```tsx
import { defineAgent } from "curb";
export const agent = defineAgent({
app: BookingAgent,
goals: {
booked: { description: "The caller booked a webinar slot", tool: "book_slot" },
felt_easy: {
description: "Booking felt easy",
judge: "The caller says booking with the assistant was easy or pleasant.",
},
},
experiments: {
greeting: {
description: "Formal vs casual opening",
variants: { formal: 1, casual: 1 },
fromClient: "greeting",
},
},
});
```
A goal is one of two kinds:
- **Tool goal** (`tool`): reached when the agent ran that tool during the call and the
tool succeeded. Name one of your agent's tools. `npx curb check` warns when no tool of
your agent has that name.
- **Judge goal** (`judge`): a sentence describing what should be true of the conversation,
1 to 1000 characters. Judge goals are not judged yet: every call shows them as
**not judged** for now, so declare them today and their results follow once judging is
available.
An experiment lists 2 to 6 variants, each with a weight from 1 to 100. The weights are
relative: `{ formal: 1, casual: 1 }` gives each variant half the calls, and
`{ formal: 1, casual: 3 }` gives `casual` three calls in four. Each call gets one variant
of every experiment when it starts and keeps it until it ends. The same call always gets
the same variant, so you can trace any call back to the variant it had.
The limits:
- Goal, experiment and variant names start with a lowercase letter and have at most 40
lowercase letters, digits and underscores, such as `booked` or `casual_v2`.
- At most 20 goals and 5 experiments per agent.
- A `description` is optional and holds at most 200 characters. It is for the people
reading the results; the agent never sees it.
`npx curb check` reports goals and experiments outside these limits and names every
problem, and a deployment with them fails to build, so a mistake never reaches a live call.
## Use the variant in your agent
`useVariant` returns the variant the current call got, so the agent can behave
differently per variant:
```tsx
import { Instruction, useVariant } from "curb";
function Greeting() {
return useVariant("greeting") === "casual" ? (
Greet the caller casually and use their first name when you know it.
) : (
Greet the caller formally.
);
}
```
Render it inside your ``, like any other instruction. The variant is set before the
agent's first words, so the greeting can depend on it. `useVariant` returns `undefined` for
an experiment your agent does not declare.
## Choose a variant from the page
`fromClient` names a key of the context your web page passes to the call. When the call's
context has that key and its value is one of the experiment's variants, the call gets that
variant instead of a random one. Any other value is ignored and the call gets a variant by
weight as usual.
With `fromClient: "greeting"`, a widget tag with `data-context-params="greeting"` lets a link
choose the variant: `https://example.com/call?greeting=casual` always starts a call with the
casual greeting. Use this to send one audience to one variant, or to try a variant yourself
on the live agent. How a page passes context is in
[Tell the agent where a call comes from](/hosted-agents/web-widget#tell-the-agent-where-a-call-comes-from).
## Try each variant locally
Every conversation in `npx curb dev` gets its own variants, chosen the same way as on a live
call, so a new conversation can get another variant. To try one variant, add the
experiment's `fromClient` key to the preview's address:
```text
http://127.0.0.1:4000/?client.greeting=casual
```
In a test scenario, `experiments` fixes the variant for that conversation. An experiment the
scenario leaves out gets a fixed variant chosen from the scenario's file and name, so the same
scenario runs with the same variants every time:
```ts
import { scenario } from "curb/testing";
export const casualGreeting = scenario({
experiments: { greeting: "casual" },
turns: ["Hallo, ich möchte einen Termin buchen."],
expect: ({ called }) => [called("book_slot")],
});
```
Keep one scenario per variant for every conversation that has to work in each of them, and
run them with `npx curb test`.
## See the results
The dashboard shows goals and experiments; you change them only in code.
- Each call on your agent's **Calls** page shows the variant it got in every experiment
and, shortly after the call ends, the result of every goal. The calls table has a
**Goals** column and an **A/B test** filter that lists only the calls of one variant.
- The agent's **Goals** page shows each goal of the live version with how it is checked and
how often calls reached it over the last 30 days, the same rates for every version, and
for each experiment the share of calls, the number of calls and the goal rates of every
variant.
Each rate comes with a 95% range. While the ranges of two variants or versions overlap,
the page says there are too few calls to tell them apart rather than naming a winner.
Calls whose goals are not judged or not evaluable are left out of the rates and counted
beside them.
A goal's result is one of:
| Result | Meaning |
|---|---|
| Reached | A tool goal's tool ran and succeeded during the call. |
| Not reached | The tool did not run, or every run of it failed. |
| Not judged | A judge goal. Judging is not available yet. |
| Not evaluable | The call's conversation is not stored, so there is nothing to evaluate it against. |
Changing a goal or an experiment takes effect with the next deployment. Calls keep the
results and variants they had, so the calls of an earlier deployment still show what they
were measured against.
## Next steps
Ship the goals and experiments with `curb deploy`
Common integrations, step by step