promptdojo_

Every number has an address — sample checks and the traceable memo — step 7 of 7

Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

Checkpoint: the sample-first gate — no AI-drafted query touches the full export until it has matched a hand-computed answer on the sample.

Three drafted queries arrived, with hand expectations written down before anything ran (they're in HAND). Write sample_gate(queries, sample, hand, full):

  • For each query name in dict order: run it on sample and compare to hand[name].
  • Match → print {name}: sample {got} vs hand {want} — PASS and remember the query as cleared.
  • No match → print {name}: sample {got} vs hand {want} — FAIL, fix before full run.
  • Then print cleared for full run: {n} of {total}.
  • Finally, run only the cleared queries on full, in order, printing {name} on full: {result}.

Expected output:

july_mrr: sample 138.0 vs hand 138.0 — PASS
july_active: sample 5 vs hand 3 — FAIL, fix before full run
mrr_per_seat: sample 12.37 vs hand 12.55 — FAIL, fix before full run
cleared for full run: 1 of 3
july_mrr on full: 512.0

(Two of the AI's three drafts are wrong, and the sample says exactly how: july_active forgot the month filter; mrr_per_seat averaged the per-account ratios instead of pooling.)

full-screen editor opens — close anytime to keep reading.