promptdojo_

Evals before it works — golden cases and the ratchet — step 5 of 8

Same router, same golden set, same red run: G4 FAIL, 3/4 — hold. Now do the TDD move — change the router until the run is green. The golden set is the spec; it doesn't move.

A registry 503 is transient infrastructure noise, same class as a timeout: the flake branch needs to recognize it.

Expected output:

G1 pass (got rerun)
G2 pass (got escalate)
G3 pass (got open_issue)
G4 pass (got rerun)
4/4 — ship
The break is on line 5 — but read the whole snippet first.

full-screen editor opens — close anytime to keep reading.