promptdojo_

The ML-system option — six boring stages that get a repo taken seriously — step 1 of 7

ch 25 · capstone1/7

The other capstone — a tiny ML system, all six stages present

You've shipped the agent. Loop, tools, validation, evals, MCP — lessons 01 through 05, done. This last lesson is the fork in the road: if the ML track is where you're headed — model training, pipelines, the "ML engineer" job description — here's the second capstone to build, and the last thing this course will ask of you.

It is not "train a model." Anyone can train a model; a tutorial notebook trains a model. A system is a model wearing a pipeline, and the pipeline has exactly six stages:

  1. Data pull — a script that fetches the raw data and snapshots it. Re-runnable, dated, boring.
  2. Validation — a schema check that refuses rows instead of hoping. Chapter 24 taught you where bad predictions come from; most are data problems, and this stage is where you catch them.
  3. Train + eval — fit the model, then score it on data it never saw. The held-out split is the whole point; the training score is a vanity metric.
  4. Gate — one comparison: eval score against a threshold you wrote down before training. Below the bar, the model does not ship. No exceptions, not even for demo day.
  5. Serving stub — one function, features in, prediction out. Not a Kubernetes cluster. A function you could put behind a route in an afternoon.
  6. Monitoring hook — every prediction logged with its input. When a bad prediction lands next Tuesday, this is the trace you'll triage from.

How this fits in a weekend

Scope is the trap, so pre-shrink everything. One public dataset you can download in under a minute. One boring model — logistic regression is a fine choice and a better signal than a half-understood transformer. One metric. Saturday: stages 1-3. Sunday morning: the gate and the stub. Sunday afternoon: the monitoring hook and a README with a single run command. If a stage is threatening to eat the weekend, shrink the data or the model — never delete the stage.

What reviewers actually look at

Here's the part nobody tells you: the reviewer skimming your repo spends about ninety seconds and never reads your model code. They've seen a thousand model.fit() calls. What they look for is judgment — does the eval use a held-out set, does a gate exist, does anything log at serving time, does the README have one command that runs the whole thing. A mediocre model inside all six stages beats a clever model inside none, every single time. The clever model says you can follow a tutorial. The gate says you can be trusted with a pipeline that touches real users.

Run the editor. Six lines of table-of-contents now; by Sunday night, six working stages with your name on the repo.