promptdojo_

The classify_failure(trace) below is the lazy default: it returns "hallucination" for every failure. That's the worst habit on a debugging team — "the model hallucinated" is what you say when you don't want to read the trace.

Fix the function to check the trace fields in priority order:

  1. If retrieved_chunks is non-empty AND retrieved_chunks_match_query is False, return "retrieval".
  2. Else if raw_output != output_after_postprocess, return "parse".
  3. Else if retrieved_chunks is empty, return "hallucination".
  4. Else return "prompt".

The trace in the editor is a class-1 retrieval bug (retrieval returned the wrong account's ticket). The fixed function should return "retrieval".

Expected output:

class: retrieval
The break is on line 10 — but read the whole snippet first.

this step needs the editor

on desktop today; in the app (coming soon). save your spot and we'll bring you back here when you're ready.

open this same url on a laptop to keep going today.