ObservabilityBeta
Evals and quality checks
Turn the conversations that matter into repeatable test cases and catch regressions before your callers do.
Quality cases
A quality case is a conversation you want your agent to handle correctly every time: the input, the expected behavior, and guardrails it must never break. Cases are specific to a channel (voice or chat) and grounded in your agent's real configuration.
Running checks
- Run checks on demand after every prompt, model or knowledge change.
- An evaluator model judges each response against the expected behavior and guardrails, and records the verdict and reasoning.
- History is kept, so you can see when a change made things better or worse.
- Each run shows its estimated credit cost before you start.
What to test first
- 1
The five most common reasons customers contact you.
- 2
Your highest-value outcome, such as a booking or a qualified lead.
- 3
Things the agent must never do: invent prices, give medical or legal advice, promise refunds.
- 4
Unhappy paths: angry customers, requests for a human, off-topic questions.
Note
Quality checks are in Beta. Agents that use flows or live tools are partly covered while a read-only execution mode is completed.
Last updated November 2026 · Found a problem? Email support@katexs.com
