Sample first. Full run second. In that order, forever.
The discipline fits in three sentences. Before an AI-drafted query touches the full dataset, pull a sample small enough to check with a pencil. Write down what the answer should be before you run anything — five rows, 151 signups, 900 visits, so pooled conversion must be 16.8%. Then run the drafted query on those same five rows and compare.
Run the code — that's the whole ceremony, mechanized. Two details carry all the weight:
The expectation comes first, in writing. Compute-then-peek feels identical and is worthless: once the query's answer is on screen, your brain will happily rationalize its way to agreeing with it ("oh right, I guess it rounds there"). An expectation written before the run is a commitment; one written after is a caption. This is the analyst's version of calling the coin before it lands.
A sample match is a license, not a proof. Five rows won't exercise every null, every duplicate key, every timezone edge. That's fine — the sample check is a cheap filter that catches the category errors: wrong aggregation shape (average-of-rates vs pooled announces itself instantly), wrong filter, wrong column, off-by-one windows. The expensive checks — join row counts, recomputing one group by hand — layer on top for the numbers that matter most. You'll drill both this lesson.
One mechanical footnote before the habit sets: the == in this gate is safe only because both sides were rounded to three places first. Round to a declared precision, then compare — never == two raw floats, or the gate will one day fail on numbers that are equal everywhere except the fifteenth decimal.
Every survivor gets an address
The second half of the discipline: when a number passes verification, it gets parked under a ref — RESULTS["q2_pooled_rate"] = 0.151 — and from that moment the memo is only allowed to cite refs. Not "conversion was 15.1%" floating free; "conversion was 15.1% [q2_pooled_rate]", where the ref resolves to the verified result, which resolves to the query, which ran on the cleaned rows from lesson 1's log. One chain, no gaps: memo → ref → result → rows → receipts. The drills ahead grade the first links — memo → ref → result — because the back half is already built: result → rows → receipts is lesson 1's cleaning log doing its job underneath the RESULTS store.
This sounds bureaucratic until the first time someone senior asks "where does 15.1% come from?" in a room. The answer stops being twenty minutes of scrolling through final_v2_REAL.ipynb, and becomes a lookup that takes four seconds and visibly cannot be wrong. Stakeholders remember which analysts can do that on the spot. So does the CFO. The rest of this lesson makes the lookup — and the audit that runs over all of it — code you've written.