promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_

Eval receipt — step 2 of 2

ch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialistch 58 · fine-tune a small specialist2/2
Checkpoint

One last thing before we move on. pass this to mark the lesson done, or skip and keep moving. hop to the next when you're ready.

Checkpoint. Write grade_eval_receipt(receipt) that returns a verdict string. Checks run in this order; first failure wins:

  1. receipt["split_clean"] is not True → "hold: split audit failed"
  2. any score in prompt/rag/ft format or label is outside 0..1 → "hold: hallucinated scores"
  3. receipt["ft"]["format"] < receipt["prompt"]["format"] → "hold: FT did not beat prompt-only format"
  4. receipt.get("knowledge_job") is True → "hold: RAG owns a knowledge failure"
  5. otherwise → "ship"

Print each named receipt as name verdict.

Expected output:

honest-ft ship
leaky-split hold: split audit failed
invented hold: hallucinated scores
worse-than-prompt hold: FT did not beat prompt-only format
handbook hold: RAG owns a knowledge failure

full-screen editor opens — close anytime to keep reading.

open the editor to run it.