choice is null and abstain is true:
"What is the weather like on Mars?" with three banking intents. None of them applies, the probabilities are almost flat, and confidence is close to 0.
When the model abstains
The model abstains whenconfidence, the margin between the two best options, is below the threshold configured for that kind of question. Typical reasons:
- None of the options applies. The request is out of scope for your option set.
- Two options are hard to tell apart. The context fits both, or the options overlap.
- The context is too vague to pick an option.
Handling abstention
probabilities is still returned when the model abstains, so you can see its best guess. Krun and the SDKs never replace null with that guess: what to do is your decision.
- send the request to a human or a general-purpose flow,
- ask the user a clarifying question,
- pass the request and the best guess to an LLM,
- add a missing option to your option set, if abstentions show a gap.
Abstention status
Every answer includesabstention_status. It tells you how much to trust abstain for that kind of question:
Calibrated
For intent questions with label-only options, the abstention threshold was validated on held-out data at a target of 98% accuracy on the answers the model does not abstain on. On data similar to Krun One’s evaluation data, answers withabstain: false should be correct at about that rate. Your traffic can differ, so monitor it with feedback.
Advisory
Advisory status applies to:- Tool routing. Abstention was measured on real “no tool applies” requests and does not yet meet the bar for calibrated. See Benchmarks.
- Intents with descriptions. This setup has not been calibrated yet.
calibrated and advisory answers.