A report is a draft over data
Launch week is done. The relaunch email went to 18,200 people, and the platform export — the CSV with the actual counts — is sitting in your downloads folder. You paste it into the model and ask for a performance summary, because writing "open rate improved meaningfully week over week" by hand is nobody's favorite job and the model's narrative will be smoother than yours.
The summary comes back in eight seconds and it reads great. That's the problem. It reads equally great whether the numbers in it are real or not. A language model producing a results paragraph is doing sentence-completion over your data, and when a rounder, more flattering number makes a better sentence, you sometimes get the better sentence. A claimed 3.6% when the export says 3.4%. An open count that appeared nowhere in the file. The failure is quiet, plausible, and wearing your formatting.
So the studio rule: every figure in an AI-drafted report is either traced to the export or it doesn't ship. Traced means one of exactly two things:
- A lookup. The report says 618 clicks; the export's
clickscell says 618. Match. - A shown calculation. The report says 3.4% CTR; you can point at
618 / 18200. The formula is visible, the inputs are export cells.
Anything else — however confident, however nicely phrased — is invented, and invented numbers in a campaign report are how teams make confident wrong decisions. Nobody re-checks a number that's already in a deck. The verification pass happens before the deck, or it never happens — which is why it runs as a scheduled script, not a heroic one-off: end of launch week, every campaign, pointed at that campaign's fresh export. Campaigns recur; so does the check.
Run the editor
Two computed rates, and look closely at the parentheses: open rate is opens over delivered, CTR here is clicks over sent. Same export, different denominators, and each printed line says which one it used. Worth knowing before you compare against a benchmark: most email platforms compute CTR over delivered (and click-to-open over opens), so an unlabeled "CTR" can mean three different numbers. This platform's export reports it over sent — the stated denominator is what keeps that from being a trap. A percentage that doesn't state its denominator isn't a fact yet — hold that thought, because the next read step shows the marketing industry doing this to itself with its own AI-adoption statistics.
Why this is your check to run, not the model's
You can ask the model to double-check its own summary, and you should — it catches some slips. But the model checking the model is the intern proofreading their own essay. The pass that counts is mechanical: your script walks every claim, compares it to the export, and prints TRACED or INVENTED with no opinion about narrative flow. You'll build exactly that in the next two steps — first by eye, then as verify_numbers.