promptdojo_

Bias, variance, and leakage in plain code — step 1 of 7

Overfitting, underfitting, and leakage: the three ways to fool yourself

A model's job is to work on data it has never seen. Every methodological rule in ML exists to protect one measurement: an honest estimate of that.

The memorizer, live

Run the editor. The "model" is a lookup table of the training set: 100% on training data, 50% (coin-flip) on new data. That gap is overfitting — the model learned the noise and specifics of its sample instead of the pattern. The opposite failure, underfitting, is a model too simple to capture the pattern at all: bad on training and test.

The classic mental model (bias/variance): simple models err by rigidity (high bias), flexible models err by mirroring their particular sample (high variance). You navigate between the two — and the only instrument that can tell you where you are is held-out data. Train score alone cannot distinguish brilliance from memorization; the editor just proved it.

The split is sacred

So: split before anything else. Train on one part, evaluate on a part the model — and you — never touched while making decisions. Two subtleties that bite:

  • Tuning against the test set (trying ten ideas, keeping what helps on test) slowly overfits the test set through your own choices. Use a third slice (validation) for tuning; open the test set rarely.
  • Time matters: for anything predictive, split by time (train on past, test on later) — a random split lets the model peek across time in both directions.

Leakage: overfitting's evil twin

Leakage is when information unavailable at prediction time sneaks into training. You've met it twice: features summed across the label window (chapter 36) and imputing with full-dataset statistics before splitting (chapter 35). Add: duplicates straddling the split, and "features" that are the label wearing a hat (account_closed_date predicting churn). Leakage produces amazing offline scores and production faceplants — the exact confident-but-wrong failure this course keeps warning about.

Where AI specifically gets this wrong

  • Preprocessing before splitting. Generated sklearn code loves to scale/impute on the full dataset, then split. Fit preprocessing on train only (chapter 39 shows the pipeline shape that enforces this).
  • Reporting train accuracy. If a script prints one accuracy, ask which set it came from. The flattering one, usually.
  • Suspiciously perfect scores. 99.8% on first try isn't genius; it's a leak until proven otherwise. Hunt the feature that knows the future.