One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.
Checkpoint: the incident drill, end to end.
The return-window macro is eval-passed. Its shipped logic escalates
when days > 35 — but the actual policy window is 30 days. Nobody
noticed, because both golden cases (day 10, day 45) land on the same
side of both numbers — it got promoted on a thin two-case set. The
lesson-2 ladder catches an empty golden set, not a thin one; nobody
enforced the five-case entry ticket, and this drill is exactly why it
exists. Today it
told a day-31 customer they were still inside the window. A human
caught it in review of an adjacent ticket.
Write run_drill(macro, fn, incident) that runs the response, printing
four lines:
- Pull the macro: set its status to
"pulled"and printkill switch: NAME pulled - Print
correction: sent by a named human— the customer gets a person, not a corrected template. - Bank the incident: append it to the macro's golden set and print
golden set: BEFORE -> AFTER cases - Re-run the evals over the grown set (a case passes when
fn(ticket) == want). Set status to"eval-passed"only if every case passes, else"review-required", and printre-eval: PASSED/TOTAL — STATUS
The buggy fn will now fail its own incident case — that's the
ratchet working. Expected output:
kill switch: return-window pulled
correction: sent by a named human
golden set: 2 -> 3 cases
re-eval: 2/3 — review-required