Mission: the statistical sanity check
Before any model touches a dataset, someone should interrogate it — and that someone should be a function. Write the smallest report that would catch this chapter's failure modes, aim it at data with planted problems, then sharpen whatever it missed.
The job: a sanity_report(data) you run on any labeled dataset
before any model — the statistical twin of chapter 36's SQL
quality gate.
The report must state:
- n and base rate — with the majority-class baseline accuracy printed right next to it (the number to beat).
- Per-feature center and spread — mean and median (their gap flags skew), min/max (range sanity).
- A sampling-noise reminder — compute the report on a random half of the data and print both versions side by side. If a number swings hard between halves, it's too noisy to bet on.
- Stress case — inject one absurd outlier and one label-correlated "cheat feature" (e.g., a copy of the label plus noise); your report should make both visible (max blows up; the cheat feature's per-class means separate suspiciously).
- One improvement — a train/test split by index with the report run on each side, or a correlation-with-label ranking: pick one, note why.
Run the editor for stages 1–2 on synthetic data. Everything funnels to one screen: hand the function a dataset and get back n, base rate, the baseline to beat, spread, and anything suspicious — before a single model is trained.