lesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receiptlesson 6 of 8 · eval receipt
Same cases, three systems, one table
An adapter without an eval receipt is a vibe. The receipt compares prompt-only, RAG, and FT on the same held-out companies (or docs). You do not get to pick a friendlier set for the adapter.
eval-receipt.md in the artifact is a table plus a pass/fail call:
| system | format (exact label shape) | label (correct bucket) | notes |
|---|---|---|---|
| prompt-only | 0.62 | 0.58 | schema in the prompt, no adapter |
| RAG | 0.64 | 0.61 | retrieved runbook; should not "win" a knowledge job we already rejected |
| FT | 0.86 | 0.84 | adapter on Qwen2.5-1.5B-Instruct |
Ship only when all of these hold:
- Split audit is clean — no company/doc overlap, reviewer present.
- FT format (consistency) is ≥ prompt-only on the named job.
- RAG does not silently own a knowledge failure. If the leftover miss was "the model did not know a changing fact," you should have held at the judgment gate, not arrived here with a RAG column that excuses the adapter.
Hold when scores look invented (1.00 across the board with five eval rows and no case list), when train companies appear in eval, or when someone reports loss and calls it an eval. Loss is a training curve. The receipt is labels on held-out logs.
The fixture table in specialist-adapter-receipt/eval-receipt.md is a legal starting point if you did not run T4. Replace the numbers if you did. Do not edit them upward to force a pass.