promptdojo_

Triage the bad prediction — reproduce with the logged input, then route by class — step 7 of 7

ch 24 · debugging broken ai output7/7
Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

Final drill: the whole runbook in one function. Three tickets from this morning, each with a trace and a repro flag. Write triage(report):

  1. Gate. If report["reproduced_with_logged_input"] is falsy, return "blocked: reproduce with the logged input" — no classification, no routing.
  2. Classify. Apply the lesson-02 priority order to report["trace"]:
    • retrieved_chunks non-empty AND retrieved_chunks_match_query is False → "retrieval"
    • raw_output != output_after_postprocess → "parse"
    • retrieved_chunks empty → "hallucination"
    • otherwise → "prompt"
  3. Route. Return "route to <destination>" via ROUTES.

Print one line per ticket as ticket_<id>: <result>.

Expected output:

ticket_9101: blocked: reproduce with the logged input
ticket_9102: route to retrieval pipeline
ticket_9103: route to post-processing code

full-screen editor opens — close anytime to keep reading.

open the editor to run it.