Read the trace, not the chat — find the broken turn before reading the user's complaint — step 7 of 9
The triage code is hardcoded to blame "true_hallucination" no matter what the trace shows. But the trace says the agent's search tool ran AND returned data — so this is a retrieval problem (wrong data came back), not a hallucination (model inventing from nothing).
Fix the classifier to check trace[-1]["tools_called"]. If it's
non-empty, this is "retrieval". Otherwise it's
"true_hallucination".
Expected output:
class: retrieval
The triage code is hardcoded to blame "true_hallucination" no matter what the trace shows. But the trace says the agent's search tool ran AND returned data — so this is a retrieval problem (wrong data came back), not a hallucination (model inventing from nothing).
Fix the classifier to check trace[-1]["tools_called"]. If it's
non-empty, this is "retrieval". Otherwise it's
"true_hallucination".
Expected output:
class: retrieval
this step needs the editor
on desktop today; in the app (coming soon). save your spot and we'll bring you back here when you're ready.