Continuous training and careful rollout
Models age (chapter 47 shows the mechanisms), so retraining is a recurring event — which means deployment is a recurring event, and recurring events need to be boring. Two disciplines make them boring: automated retraining with gates, and rollouts that limit blast radius.
Continuous training, scoped to reality
"Continuous" rarely means literally always. The standard shape is
scheduled or triggered retraining: chapter 37's pipeline
rebuilds the dataset, chapter 43's loop retrains, chapter 46.4's
gate evaluates, and only a passing candidate reaches the registry
as staging. Everything you've built is the CT system; the new
part is that no human is in the loop until something fails a gate.
The absolute rule carried over: an ungated automatic retrain is
an automatic way to ship a regression. The gate is not optional
just because the pipeline is automatic.
Rollout: never 0% → 100%
Run the editor. The candidate looks fine in offline evals but is secretly worse live. A canary rollout routes a small share of traffic (here 10%) to it, live metrics diverge — 88.9% vs 85.2% good outcomes — and you caught the problem having exposed a tenth of your users. Promote only when the canary matches or beats the incumbent at meaningful volume (chapter 38's minimum-n rule applies: 223 canary requests is a hint, not a verdict — let it soak).
The gentler sibling is shadow mode: the candidate receives
real traffic and its predictions are logged but not acted on.
Zero user risk, perfect for validating chapter 45.2's skew and
infrastructure under real load — but it can't measure outcome
differences, since its decisions never happen. Shadow first,
canary second, then percentage ramps; and rollback (chapter 45's
registry promote(previous)) rehearsed, one command, no meeting.
Where AI specifically gets this wrong
- Retraining cron with no gate. The generated scheduler faithfully ships whatever the loop produced, including the run where upstream data broke (chapter 37's drift, weaponized).
- All-at-once deploys. The default generated deploy replaces 100% of traffic — the canary demo you just ran is the argument against.
- Canaries judged on vibes. "Looks fine" is not a promote condition; write it as code, like everything else in this chapter.
- Canaries judged on tiny n. Two hundred requests of wiggle is sampling noise (chapter 38) — the promote condition needs a minimum n baked in.