# The diff is the change control You just watched a diff catch something a rewrite would have smuggled through: the deletion of the one-at-a-time requeue rule. That's the whole argument for change control in one `-` line. When a process changes as an in-place edit, the old truth just evaporates — nobody approves the deletion because nobody sees it. When it changes as a diff, the deletion has to stand there and defend itself. So every SOP change in this chapter ships the same way: 1. **The version bumps.** Same version, different steps is a lie waiting for an incident review. If the content moved, the number moves. 2. **A named reviewer approves — and it isn't the drafter.** Self-review is no review; the person who proposed dropping step 3 is the person least likely to notice why step 3 mattered. One name in the `reviewer` field, different from the name in `drafter`. The lint can check that mechanically, and yours will. 3. **Safety-touching steps get a safety review, explicitly.** That new step 5 — flagging hazmat orders for the safety owner — isn't decoration. Under the OSH Act's General Duty Clause, the employer must keep the workplace free of recognized hazards, and that duty sits with humans no matter what drafted the procedure. An AI-written SOP that omits a safety step moves nothing off the SOP's owner. So the review question is asked out loud every time: *does any changed step touch a hazard or a recordkeeping duty?* Writing down "no" counts. Skipping the question doesn't. ## Why bother, in numbers you can take to your boss The evidence on AI in operations points one direction, and it isn't "buy a tool." A July 2025 Gartner survey of 1,973 managers found business units that **redesign how work gets done** with AI are twice as likely to exceed revenue goals. Meanwhile MIT's 2025 GenAI Divide study found 95% of enterprise genAI pilots delivered no measurable P&L impact — with the value that did materialize concentrated where workflows were actually redesigned. (That study counts P&L only, so efficiency gains are undercounted; the direction survives the caveat.) Redesign means changing SOPs. Changing SOPs without change control means every improvement is also a chance to silently delete the step that was load-bearing. The diff-plus-named-reviewer loop is what lets you change processes *fast* without changing them *blind* — which is the actual competitive skill, per the only large-sample evidence anyone has. Next two steps: you build the lint, then the full release gate that puts all of this — provenance, verification, version bump, reviewer independence, safety review — in front of every v-next.2b:T43b,SOPS = [ {"id": "sop-receiving", "owner": "miguel", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, ]}, {"id": "sop-stuck-orders", "owner": "vera", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, {"n": 3, "source": "model-filled", "verified": False, "safety": False}, ]}, {"id": "sop-hazmat-intake", "owner": None, "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": True, "safety_reviewed": True}, {"n": 2, "source": "model-filled", "verified": True, "safety": True, "safety_reviewed": False}, ]}, ] def lint_sop(sop): reasons = [] # 1. no owner? append "no owner" # 2. for each step: unverified model fill? unreviewed safety? return reasons for sop in SOPS: r = lint_sop(sop) print(f"{sop['id']}: {'READY' if not r else 'BLOCKED — ' + '; '.join(r)}") 2c:T52b,SOPS = [ {"id": "sop-receiving", "owner": "miguel", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, ]}, {"id": "sop-stuck-orders", "owner": "vera", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, {"n": 3, "source": "model-filled", "verified": False, "safety": False}, ]}, {"id": "sop-hazmat-intake", "owner": None, "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": True, "safety_reviewed": True}, {"n": 2, "source": "model-filled", "verified": True, "safety": True, "safety_reviewed": False}, ]}, ] def lint_sop(sop): reasons = [] if not sop["owner"]: reasons.append("no owner") for s in sop["steps"]: if s["source"] == "model-filled" and not s["verified"]: reasons.append(f"step {s['n']} unverified model fill") if s["safety"] and not s.get("safety_reviewed"): reasons.append(f"step {s['n']} safety step unreviewed") return reasons for sop in SOPS: r = lint_sop(sop) print(f"{sop['id']}: {'READY' if not r else 'BLOCKED — ' + '; '.join(r)}") 2d:T475,CHANGES = [ {"id": "sop-receiving", "drafter": "vera", "reviewer": "miguel", "current": {"version": "1.0"}, "proposed": {"version": "1.1", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, ]}}, {"id": "sop-stuck-orders", "drafter": "vera", "reviewer": "vera", "current": {"version": "1.0"}, "proposed": {"version": "2.0", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, ]}}, {"id": "sop-hazmat-intake", "drafter": "sam", "reviewer": "miguel", "current": {"version": "2.0"}, "proposed": {"version": "2.0", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": True, "safety_reviewed": True}, {"n": 2, "source": "model-filled", "verified": False, "safety": False}, ]}}, ] # define release_sop(change) below for c in CHANGES: r = release_sop(c) if r: print(f"{c['id']}: BLOCKED — {'; '.join(r)}") else: print(f"{c['id']}: RELEASED v{c['proposed']['version']}") 2e:T6dd,CHANGES = [ {"id": "sop-receiving", "drafter": "vera", "reviewer": "miguel", "current": {"version": "1.0"}, "proposed": {"version": "1.1", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, {"n": 2, "source": "transcript", "verified": True, "safety": False}, ]}}, {"id": "sop-stuck-orders", "drafter": "vera", "reviewer": "vera", "current": {"version": "1.0"}, "proposed": {"version": "2.0", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": False}, ]}}, {"id": "sop-hazmat-intake", "drafter": "sam", "reviewer": "miguel", "current": {"version": "2.0"}, "proposed": {"version": "2.0", "steps": [ {"n": 1, "source": "transcript", "verified": True, "safety": True, "safety_reviewed": True}, {"n": 2, "source": "model-filled", "verified": False, "safety": False}, ]}}, ] def release_sop(change): reasons = [] if change["proposed"]["version"] == change["current"]["version"]: reasons.append("version not bumped") if not change.get("reviewer"): reasons.append("no reviewer") elif change["reviewer"] == change["drafter"]: reasons.append("reviewer is the drafter") for s in change["proposed"]["steps"]: if s["source"] == "model-filled" and not s["verified"]: reasons.append(f"step {s['n']} unverified model fill") if s.get("safety") and not s.get("safety_reviewed"): reasons.append(f"step {s['n']} safety step unreviewed") return reasons for c in CHANGES: r = release_sop(c) if r: print(f"{c['id']}: BLOCKED — {'; '.join(r)}") else: print(f"{c['id']}: RELEASED v{c['proposed']['version']}") 2f:Tbaf,# The checklist that lied You're in the ops studio. **What you're doing this lesson:** decide which checklist items a computer may finish, and which ones a named person must sign. A green tick that nobody did is worse than no checklist. You do not need the SOP lesson to start. A warehouse team wires their recurring checklist into an automation. It's a good automation — it knows the cadences, it fires on schedule, it never forgets. Six months later an auditor asks who performed the forklift daily inspections in March. The checklist says they happened. Green ticks, every day. Nobody inspected the forklift. The bot marked the item done because the item was due, and "due" and "done" had been wired to the same wire. The checklist didn't just fail to catch the miss — it *manufactured evidence that there was no miss*. That's strictly worse than having no checklist at all, and it's the failure this whole lesson exists to make impossible. ## Some sign-offs are laws For an ops lead, this isn't a philosophy question. The floor is statutory: - Under **OSHA** (Occupational Safety and Health Administration — the US workplace-safety regulator) **recordkeeping rule (29 CFR 1904)**, covered employers must maintain the Form 300 log of work-related injuries and illnesses, post the 300A annual summary where employees can see it, and — depending on size and industry — submit the data to OSHA electronically, with annual submissions due March 2. Employees have the right to review the log. These are duties held by the employer and its named people. - Under the **General Duty Clause**, the workplace must be kept free of recognized hazards — a duty that doesn't move onto whatever tool drafted or scheduled the procedure. A script that auto-completes "post the 300A summary" hasn't discharged a duty. It has hidden the fact that nobody did. When the gap surfaces — audit, incident, lawsuit — the record your own automation faked is the first exhibit. ## The line, drawn explicitly So every recurring item gets classified, on purpose, before automation touches it: - **Auto-run** — reversible, no duty, no dollar: regenerate the report draft, refresh the due list, compile the reorder suggestions. If it runs wrong, you rerun it. The machine may do these end to end. - **Named approval** — a named person must mark it done. Anything with a legal duty, a signature, a safety act, or spend behind it: inspections, compliance postings, invoice approvals, anything that binds the company. The machine computes that it's *due* and routes it to that person. Only that person marks it *done*. Here's the part that should annoy you: most tool tours leave the threshold implicit, as Lean tradition. The question is concrete — what a system may execute versus what needs a human, and who sets that line. You're about to encode it as a field in a data structure, which is where a threshold actually holds. Next: the checklist as schedule data, with the line built in.30:T967,# The schedule is data Run it. Four recurring duties, one computation, and the whole lesson's argument in the output: the machine decided what's *due*; it decided nothing about what's *done*. The fields carry the policy: - **`every` + `last_done`** — the cadence and the last completion, both as day numbers. "Due" is arithmetic: `TODAY - last_done >= every`. No one remembers the vendor scorecards are monthly; the data does. (Day numbers instead of datetimes keep the drill deterministic — the arithmetic is identical when you swap in real dates. And a negative `last_done`, like the 300A's `-200`, just means the last completion was *last year*, counted back past day zero — which is how an annual item ever manages to come due.) - **`approval`** — the line from the last step, as a field. `"auto"` means the machine may run it end to end. A *name* means the machine's authority ends at routing: compute that it's due, put it in front of `miguel`, stop. Note what the field's type buys you: there is no way to write "approved by somebody." It's `"auto"` or it's a person. - **`anchor`** — *why* the sign-off exists, written where the next person can find it. `"29 CFR 1904"` on the 300A item means the approval isn't a formality someone can optimize away in next quarter's efficiency push. An unanchored sign-off eventually gets deleted by someone reasonable; an anchored one survives, because deleting it means arguing with the anchor. ## What the run just showed Three items came up due. The low-stock report routes to auto-run — it's a draft over an export; if it's wrong, you regenerate it. The forklift inspection and the 300A both route to named humans, and look at *who*: the item's own owner. The person who holds the duty is the person the automation asks. Meanwhile `vendor-scorecards` — fifteen days since last done, cadence thirty — correctly stays quiet. A checklist that computes "not due" is as valuable as one that computes "due": it's the difference between a system and a nag. One thing this loop deliberately cannot do: write `last_done`. Completing an item is what updates `last_done`, and for named-approval items completion happens when the named human says so — which means the machine can never make an item *stop being due* by itself. That property is the entire fix for the checklist that lied, and you're about to enforce it against code that breaks it.31:T9c8,# Exceptions are a queue, not a shrug The worst possible output for an automated checklist run isn't a page of errors. It's `all clear` — printed over a failure. The export wasn't there, so the low-stock report silently regenerated from *last week's* file; the vendor scorecard source hadn't refreshed in 41 days, so the "current" scores are stale fiction. Nothing crashed. Everything lied. You've spent this chapter making the machine unable to fake a signature. This step makes it unable to fake a clean run. The rule is one sentence: **a failed or doubtful item never disappears — it lands in the exception queue with a reason, and the queue is part of the run report.** Three properties make an exception queue worth having: - **Every entry carries a reason, in operational language.** `export missing` and `source stale: 41 days old` are things the 8am human can act on without archaeology. An exception without a reason is just a fancier silent failure — it tells you something's wrong and makes you re-derive what. - **The queue is visible where the results are.** Not a log file nobody tails. The run report prints successes, then the count, then the queue. If the count isn't zero, nobody can read the report and come away thinking the night was clean — the format itself refuses to let a failure hide. - **Carried items carry their reason too.** Sometimes the exception is human: the cycle count didn't happen because aisle 3 was blocked by an inbound load. Fine — real operations have Tuesdays. The item gets carried *with the reason written down*, not quietly marked done to keep the streak green. A checklist with three honest carries is a record; a checklist with a perfect completion rate is a suspect. There's a trust argument here that ops people feel in their bones. Your team already lives in a world of traceability — lot numbers, chain of custody, calibration stickers. An automation that swallows failures breaks that culture from the inside: the *first* silently stale report someone catches is the last time anyone trusts the automated ones, and then you're back to everyone rebuilding the numbers by hand, except now there's also a bot to babysit. The exception queue is what lets the automation keep its credibility on the nights it fails — which is the only kind of credibility that counts. Next: you build the full router — due list in, three-way routing out. Then the checkpoint bolts the exception queue on and produces the complete nightly run report.32:T527,TODAY = 200 ITEMS = [ {"task": "pull-wms-export", "every": 1, "last_done": 199, "approval": "auto", "result": "export missing"}, {"task": "regenerate-low-stock-report", "every": 7, "last_done": 192, "approval": "auto", "result": "ok"}, {"task": "forklift-inspection-log", "every": 1, "last_done": 199, "approval": "miguel"}, {"task": "post-300a-summary", "every": 365, "last_done": -200, "approval": "vera"}, {"task": "vendor-scorecards", "every": 30, "last_done": 185, "approval": "auto", "result": "ok"}, ] def nightly_run(items, today): ran = exceptions = waiting = 0 for it in items: if today - it["last_done"] < it["every"]: print(f"{it['task']}: not due") elif it["approval"] != "auto": waiting += 1 print(f"{it['task']}: awaiting sign-off ({it['approval']})") elif it["result"] == "ok": ran += 1 print(f"{it['task']}: ran") else: exceptions += 1 print(f"{it['task']}: EXCEPTION — {it['result']}") print(f"ran: {ran} exceptions: {exceptions} awaiting sign-off: {waiting}") if exceptions: print(f"verdict: NEEDS ATTENTION — {exceptions} exception(s)") else: print("verdict: CLEAN") nightly_run(ITEMS, TODAY) 33:Tc37,# The number that moves money You're in the ops studio. **What you're doing this lesson:** treat every number in the weekly ops report as a claim that must point back to the export it came from. A pretty number with no **receipt** (the export cell or the shown calculation that proves it) can move real money. You do not need the SOP or approval lessons to start. The weekly ops report isn't reading material. Someone sees "average days to ship: 4.2" and *doesn't* add a second picker to the Thursday shift. Someone sees "two late orders" and decides the carrier contract survives another quarter. Every number in that report is a lever connected to real staffing, real inventory, real vendor money — which is what makes an invented number in an ops report categorically worse than an invented sentence in a memo. The memo embarrasses you. The number misallocates payroll. Now let a model draft the report, because it's genuinely good at it: hand it the export and it returns clean prose, sensible sections, plausible figures, in seconds. Here's the discipline this lesson installs: **an AI-drafted ops report is a draft over data. Every figure in it either traces to the export or it is fiction with good formatting.** There is no third category. The model doesn't flag which of its numbers it computed and which it pattern-completed — a fabricated average sits in the same font as a real one, and reads just as confident. Your job is not to sense which is which. Your job is to recompute. ## The uncomfortable stat, and what it actually says Published exposure analyses are blunt about roles like this one: they warn of "role compression" for operations administrators and middle managers whose job is primarily process compliance, data collection, and report generation. If your value is *producing* the Friday number, you are competing with a token stream that produces numbers for free. But hold that against the labor data before drawing the gloomy conclusion: Vanguard research covering mid-2023 to mid-2025 found employment in high-AI-exposure occupations *grew* 1.7% versus 0.8% elsewhere, with real wages up 3.8% versus 0.7%. Exposure is not displacement. The reconciliation is the whole career thesis of this lesson: the compressible part of the job is generating the report; the growing part is **standing behind it**. The person who can say "every number in this report traces to the export, and I caught the one that didn't" is doing something the model structurally cannot do for itself — because a model checking its own output against a source is one prompt-injection or one context-slip away from grading its own homework. Ops, of all functions, should find this natural. You already refuse to accept an inventory count without a cycle count, a shipment without a BOL, a calibration without a sticker. The export is the receipt; the report is the claim. This lesson just makes you apply chain-of-custody thinking to the numbers the model hands you — with a verification pass you run in code, not vibes. First move, next step: recompute one claimed average against five rows, and watch it fail.34:T84b,# Recompute, don't trust Run it. The report claims average days-to-ship is 4.2. The export — all five rows of it — says 6.0. The draft was off by almost two days on the one number that decides whether this week's shipping performance was fine or a problem. Notice how the invented number *feels*: 4.2 is exactly the kind of average this data could plausibly have. It's not absurd. It's not flagged. If you eyeballed the rows — 2, 9, 3, 4, 12 — would you have caught it? The 12 drags the mean up more than intuition expects; that's precisely the kind of arithmetic models fumble and humans don't re-do by hand. Which is why the check is code, not squinting. Three details in this tiny verifier that carry the whole method: - **The actual comes from the export, computed in front of you.** `sum(...) / len(...)` over the rows. Not the model's stated methodology, not a second model asked to double-check the first — the same file the report claims as its source, re-aggregated. The receipt is only a receipt if you pull it yourself. - **The comparison uses a tolerance, not `==`.** `abs(claimed - actual) < 0.05` accepts honest rounding — a report that says 6.0 when the raw mean is 5.98 is fine. Float equality would flag legitimate rounding as fraud and bury the real catches in noise; a sloppy tolerance like ±0.5 would wave the fabrications through. The tolerance is a policy choice, so it's visible, in one place, on purpose. - **The mismatch line names both numbers.** `report says 4.2, export says 6.0` is a sentence you can put in front of anyone — your boss, the model in a retry prompt, yourself in three weeks — and the correction is unambiguous. "This number is wrong" starts an argument; "the export says 6.0" ends one. The pattern generalizes to every figure a report can claim: counts, maxima, percentages, rates. Each claim gets a recomputation; each recomputation is a few tokens of stdlib. The rest of the lesson scales this from one number to the whole report — and then puts a SEND/HOLD gate on the result, because a report with one unverified number isn't 95% done. It's not done.35:T988,# The anomaly is a question, not a finding Verification catches numbers that don't match the export. But there's a second class of model output in every AI-drafted ops report: the *anomaly flag*. "Shipping times spiked mid-week." "Damage rate is trending up." These aren't numbers you can tolerance-check — they're interpretations, and they fail differently. An anomaly flag from a model is a hypothesis wearing a conclusion's clothes. Sometimes it's real: the 12-day order in your export is genuinely worth a conversation with the carrier. Sometimes it's an artifact: "spiked mid-week" because two orders happened to land on a Wednesday in a five-row sample, which is noise narrated as signal. The model can't tell you which, because telling the difference requires the thing it doesn't have — knowledge of what Wednesday was actually like on your dock. So anomaly flags get the same treatment as numbers, adapted: - **Every flag gets checked against the export before it survives into the report.** Which rows support it? If the "spike" is one order that a forklift sat on while aisle 3 was blocked, the flag dies and the reason gets a sentence instead. - **A checked flag either becomes a finding with rows attached, or it gets cut.** No flag passes through on the model's say-so. "Damage rate trending up" with no prior-week data behind it isn't a trend — it's a vibe with a percentage. ## The last line of the report One more discipline, and it's the one that makes the whole document worth sending: **the report names the decision it informs.** "Avg days-to-ship 6.0, driven by two orders over 9 days — decide whether Thursday gets a second picker." A report that informs no decision is a status ritual; the compressible kind of ops work, per every exposure analysis that's looked. A report that ends in a named decision is the opposite: it's the artifact that makes you the person the decision waits for. This is also where your field's existing culture does the arguing for you. Operations already runs on traceability — lot numbers, audit trails, ISO paperwork with signatures on it. Numbers-with-receipts isn't a new discipline you're importing from software; it's your discipline, applied to a new supplier. The model is a vendor that delivers reports. You'd never onboard a vendor without incoming inspection. Don't onboard this one that way either. Two steps left: build the full verifier, then gate the send.36:T488,```python def report_gate(name, claims, export): actuals = { "orders_shipped": len(export), "avg_days": sum(r["days"] for r in export) / len(export), "late_orders": sum(1 for r in export if r["days"] > 5), "damage_rate_pct": 100 * sum(1 for r in export if r["damaged"]) / len(export), } failing = [] print(f"{name}:") for metric, claimed in claims: actual = actuals[metric] if abs(claimed - actual) <= 0.05: print(f" {metric}: VERIFIED ({claimed})") else: print(f" {metric}: MISMATCH — report {claimed}, export {actual}") failing.append(metric) if failing: print(f" verdict: HOLD — {', '.join(failing)}") else: print(" verdict: SEND") ``` That's three artifacts so far: the SOP that survives a resignation, the checklist that can't forge a signature, and the report that can't ship a fiction. One property — a stranger can trust them, because the checking is built in, and the humans are exactly where the duties are. Next lesson adds a fourth: a ticket that cannot leave for the model still carrying the member. 37:T42f,Checkpoint: the send gate. Two versions of the weekly report — the model's first draft and the corrected resubmission — go through the full verification, and each gets a verdict. Write `report_gate(name, claims, export)`: 1. Print `{name}:` 2. Recompute the four metrics (same formulas as the last step: row count, mean of `days`, count of `days > 5`, percent `damaged`) and verify each claim in order with tolerance 0.05, printing indented lines: ` {metric}: VERIFIED ({claimed})` or ` {metric}: MISMATCH — report {claimed}, export {actual}` 3. Collect the mismatched metric names. Print ` verdict: HOLD — {names comma-joined}` if any, else ` verdict: SEND`. Expected output: ``` weekly-ops-w30: orders_shipped: VERIFIED (5) avg_days: MISMATCH — report 4.2, export 6.0 late_orders: VERIFIED (2) damage_rate_pct: VERIFIED (20.0) verdict: HOLD — avg_days weekly-ops-w30-corrected: orders_shipped: VERIFIED (5) avg_days: VERIFIED (6.0) late_orders: VERIFIED (2) damage_rate_pct: VERIFIED (20.0) verdict: SEND ``` 38:T5a9,EXPORT = [ {"order": "A-101", "days": 2, "damaged": False}, {"order": "A-102", "days": 9, "damaged": False}, {"order": "A-103", "days": 3, "damaged": True}, {"order": "A-104", "days": 4, "damaged": False}, {"order": "A-105", "days": 12, "damaged": False}, ] REPORTS = [ ("weekly-ops-w30", [ ("orders_shipped", 5), ("avg_days", 4.2), ("late_orders", 2), ("damage_rate_pct", 20.0), ]), ("weekly-ops-w30-corrected", [ ("orders_shipped", 5), ("avg_days", 6.0), ("late_orders", 2), ("damage_rate_pct", 20.0), ]), ] def report_gate(name, claims, export): actuals = { "orders_shipped": len(export), "avg_days": sum(r["days"] for r in export) / len(export), "late_orders": sum(1 for r in export if r["days"] > 5), "damage_rate_pct": 100 * sum(1 for r in export if r["damaged"]) / len(export), } failing = [] print(f"{name}:") for metric, claimed in claims: actual = actuals[metric] if abs(claimed - actual) <= 0.05: print(f" {metric}: VERIFIED ({claimed})") else: print(f" {metric}: MISMATCH — report {claimed}, export {actual}") failing.append(metric) if failing: print(f" verdict: HOLD — {', '.join(failing)}") else: print(" verdict: SEND") for name, claims in REPORTS: report_gate(name, claims, EXPORT) 39:Tabd,# The ticket that left the building You're in the ops studio. **What you're doing this lesson:** hide member names and phone numbers before a vendor chatbot sees a ticket, then put them back only for the person who will send the reply. You do not need the earlier ops lessons to start. Vera's exceptions desk drafts replies to members whose orders failed. Tonight's ticket: member `MBR-104221`, clinic email, a 555 callback number, a one-line note that the refill is late. Someone pastes the whole thing into the vendor model because the model is genuinely good at the draft — warm, specific, done in ten seconds. The draft is usable. The leak is not a vibe. The raw member ID, email, and phone are now in a vendor log. The model did exactly what it is for. It has no duty to refuse identifiers. Nothing in the send path asked whether those values were allowed to leave. ## Why this exists (and what this lesson is not) **HIPAA** (Health Insurance Portability and Accountability Act — the US health-privacy law) and **SOC 2** (a common vendor security audit) are why the vendor review keeps asking what customer data a model sees. **PII** (personally identifiable information — a name, email, phone, or member ID that points at a real person) is the payload this lesson hides. A vendor model is outside the building. Raw member identifiers don't go with it. That is the *reason*. This is not a compliance class: no BAA walkthrough, no Trust Services Criteria, no advice about what your lawyer would file. The control you can actually write is a proxy. ## The three rules of the proxy 1. **Mask before the model.** Every known identifier in the ticket becomes a token — `[MEMBER_1]`, `[EMAIL_2]`, `[PHONE_3]`. The outbound payload is the ticket with those values gone. If a known value is still in the outbound text, the send is refused. The model drafts against tokens. 2. **Unmask after, for a named human.** The draft comes back with tokens in it. A token map restores the real values. Restoration is not "the system." It is Vera, or Miguel — a person who will send the reply and can be asked about it later. 3. **`auto` does not get the real values.** The same line you already drew for checklists: the machine may compute and draft; it may not hold the identifiers. A proxy that unmasks into a log, a vendor payload, or an `approval: auto` path is the paste bug with extra steps. A proxy that satisfies all three has the property this lesson is built on: **the model can draft the exception reply without ever seeing the member, and the named human can send it without ever typing the identifiers back in by hand.** The next step turns these rules into a data structure — because a rule you can't lint is a rule that erodes.3a:T90f,# Mask, then unmask Run the code. That's tonight's exception ticket as a string — and the loop just took every known identifier out of the outbound payload. Walk the fields: - **`PII`** — the values this ticket is allowed to tokenize, in order. `kind` becomes the token stem (`member` → `MEMBER`). The number is the 1-based index in *this* list, not a global counter. `[MEMBER_1]` on this ticket is not a promise about any other ticket. - **`token_map`** — the receipt. Token to real value. Without it, unmasking is guessing. With it, restoration is a replace loop, same as the mask loop run backwards. - **`KNOWN`** — what must never appear in the outbound text, even if someone forgot to put it on `PII`. The token map is what you think you masked. The leftover check is what you already know is an identifier. Two lists, on purpose — the same split you used for `source` vs `verified` on an SOP step. | layer | sees | |---|---| | outbound, to the model | tokens only — `[MEMBER_1]`, `[EMAIL_2]`, `[PHONE_3]` | | leftover check | any `KNOWN` value still sitting in that outbound string | | restore, for the named human | the real member ID, email, and phone, written back from the map | Now look at what printed. The outbound line has no clinic email, no callback number, no member ID. The leftover line says `none`. The map still holds the real values — they did not disappear; they moved to a side channel the model never gets. ## Why a proxy and not a memo Because a memo can only be remembered, and remembering is exactly the review that fails at 7pm on a desk with twenty open exceptions. Data can be linted: - known value still in the outbound string → **blocked**, mechanically, every time, including the ticket someone "just needed a quick draft on." - No named reviewer → blocked. `auto` is not a reviewer. - Later: unmask into a vendor log → blocked. Same lint, one more destination. The one thing the leftover check can't do is discover identifiers you never listed. `KNOWN` is only as good as the catalog you maintain — emails, phones, member IDs, the shapes that already live in your member file. The machine enforces that those values did not leave. A human still owns the catalog. Next: you play the send-path reviewer, reading four payloads against the leftover rule.3b:T84c,# Unmask is a named-human act You just watched a draft come back as tokens and leave as a member email, a phone, and a member ID. That restore step is the whole second half of the proxy, and it has the same shape as the checklist line from earlier in the chapter: the machine may compute; a named human holds what binds the company. Here the thing that binds is not a forklift inspection or an OSHA posting. It is the identifiers themselves. Once they are written back, they are in someone's hands — on a screen, in a send box, in a thread a lawyer can ask about. `auto` is not a pair of hands. A vendor log is not a pair of hands. Unmasking into either is the original paste, with a token map as an alibi. So the restore path gets classified, on purpose, before it runs: - **Named human** — Vera, Miguel, whoever owns the exception reply. The map runs. The real values appear in front of that person. They send, or they don't. - **Anyone else** — `auto`, a log line, a "send the draft back to the vendor for a second pass." The map does not run. The draft stays tokenized, or the whole ticket blocks. Tokens can travel. Real values stop at a name. ## The leftover check is not optional just because the reviewer is named A named human on a *raw* outbound is not a proxy. That is someone reading the leak after it already left. The leftover check runs on what the model would see, and it runs first. Only a clean outbound gets as far as the reviewer question. Two gates, in that order: 1. Any known identifier still in the outbound string → refuse the send. The model never sees it. 2. No named reviewer (missing, or `auto`) → refuse the restore. The real values never come back. You have seen this division all chapter: the machine enforces that the check happened; the human is the check. The leftover list is the machine. The name on the restore is the human. HIPAA and SOC 2 are still just the reason anyone funded this. They do not write the function. Next two steps: you build the proxy, then the gate that prints both sides — what left for the model, and who (if anyone) got the real values back.3c:T447,KNOWN = [ "ada@harbor-clinic.org", "415-555-0142", "MBR-104221", "samir@harbor-clinic.org", ] MODEL_DRAFT = { "exc-441": "We'll call [PHONE_3] about [MEMBER_1] and email [EMAIL_2].", "exc-442": "We'll call [PHONE_2] about [MEMBER_1].", "exc-443": "We'll email [EMAIL_1].", } TICKETS = [ {"id": "exc-441", "reviewer": "vera", "text": "Member MBR-104221 at ada@harbor-clinic.org — late refill. Call 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, {"kind": "email", "value": "ada@harbor-clinic.org"}, {"kind": "phone", "value": "415-555-0142"}, ]}, {"id": "exc-442", "reviewer": "miguel", "text": "Member MBR-104221 called from 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, ]}, {"id": "exc-443", "reviewer": "auto", "text": "Email samir@harbor-clinic.org about the missed window.", "pii": [ {"kind": "email", "value": "samir@harbor-clinic.org"}, ]}, ] # define run_proxy(ticket) below for t in TICKETS: run_proxy(t) 3d:T6f0,KNOWN = [ "ada@harbor-clinic.org", "415-555-0142", "MBR-104221", "samir@harbor-clinic.org", ] MODEL_DRAFT = { "exc-441": "We'll call [PHONE_3] about [MEMBER_1] and email [EMAIL_2].", "exc-442": "We'll call [PHONE_2] about [MEMBER_1].", "exc-443": "We'll email [EMAIL_1].", } TICKETS = [ {"id": "exc-441", "reviewer": "vera", "text": "Member MBR-104221 at ada@harbor-clinic.org — late refill. Call 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, {"kind": "email", "value": "ada@harbor-clinic.org"}, {"kind": "phone", "value": "415-555-0142"}, ]}, {"id": "exc-442", "reviewer": "miguel", "text": "Member MBR-104221 called from 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, ]}, {"id": "exc-443", "reviewer": "auto", "text": "Email samir@harbor-clinic.org about the missed window.", "pii": [ {"kind": "email", "value": "samir@harbor-clinic.org"}, ]}, ] def run_proxy(ticket): masked = ticket["text"] token_map = {} for i, item in enumerate(ticket["pii"], 1): token = f"[{item['kind'].upper()}_{i}]" token_map[token] = item["value"] masked = masked.replace(item["value"], token) if any(v in masked for v in KNOWN): print(f"{ticket['id']}: BLOCKED — raw PII") return if not ticket.get("reviewer") or ticket["reviewer"] == "auto": print(f"{ticket['id']}: BLOCKED — no named human") return restored = MODEL_DRAFT[ticket["id"]] for token, value in token_map.items(): restored = restored.replace(token, value) print(f"{ticket['id']}: FOR {ticket['reviewer']}: {restored}") for t in TICKETS: run_proxy(t) 3e:T493,```python def proxy_gate(ticket): masked = ticket["text"] token_map = {} for i, item in enumerate(ticket["pii"], 1): token = f"[{item['kind'].upper()}_{i}]" token_map[token] = item["value"] masked = masked.replace(item["value"], token) leftover = [v for v in KNOWN if v in masked] print(f"{ticket['id']}:") print(f" outbound: {masked}") if leftover: print(f" leftover: {', '.join(leftover)}") print(" verdict: BLOCKED — raw PII") return print(" leftover: none") if not ticket.get("reviewer") or ticket["reviewer"] == "auto": print(" verdict: BLOCKED — no named human") return restored = MODEL_DRAFT[ticket["id"]] for token, value in token_map.items(): restored = restored.replace(token, value) print(f" for {ticket['reviewer']}: {restored}") ``` Read it like the 8am human will: one ticket left as tokens and came back as a member, in front of Vera; one never left, because a known phone was still in the outbound; one was clean and still stopped, because `auto` is not a pair of hands. The model drafted nothing it was not allowed to see. 3f:T488,Checkpoint: the send gate. Same three tickets, both sides of the proxy printed — what would leave for the model, and who (if anyone) gets the real values back. Write `proxy_gate(ticket)`: 1. Print `{id}:` 2. Mask as in the last step (`[KIND_n]`, `n` from 1 in `pii` order). Print ` outbound: {masked}` 3. Collect leftover `KNOWN` values still in the masked text, in `KNOWN` order. If any → print ` leftover: {values comma-joined}` and ` verdict: BLOCKED — raw PII` 4. Elif reviewer is falsy or `"auto"` → print ` leftover: none` and ` verdict: BLOCKED — no named human` 5. Else unmask `MODEL_DRAFT[id]` and print ` leftover: none` and ` for {reviewer}: {restored}` Expected output: ``` exc-441: outbound: Member [MEMBER_1] at [EMAIL_2] — late refill. Call [PHONE_3]. leftover: none for vera: We'll call 415-555-0142 about MBR-104221 and email ada@harbor-clinic.org. exc-442: outbound: Member [MEMBER_1] called from 415-555-0142. leftover: 415-555-0142 verdict: BLOCKED — raw PII exc-443: outbound: Email [EMAIL_1] about the missed window. leftover: none verdict: BLOCKED — no named human ``` 40:T449,KNOWN = [ "ada@harbor-clinic.org", "415-555-0142", "MBR-104221", "samir@harbor-clinic.org", ] MODEL_DRAFT = { "exc-441": "We'll call [PHONE_3] about [MEMBER_1] and email [EMAIL_2].", "exc-442": "We'll call [PHONE_2] about [MEMBER_1].", "exc-443": "We'll email [EMAIL_1].", } TICKETS = [ {"id": "exc-441", "reviewer": "vera", "text": "Member MBR-104221 at ada@harbor-clinic.org — late refill. Call 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, {"kind": "email", "value": "ada@harbor-clinic.org"}, {"kind": "phone", "value": "415-555-0142"}, ]}, {"id": "exc-442", "reviewer": "miguel", "text": "Member MBR-104221 called from 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, ]}, {"id": "exc-443", "reviewer": "auto", "text": "Email samir@harbor-clinic.org about the missed window.", "pii": [ {"kind": "email", "value": "samir@harbor-clinic.org"}, ]}, ] # define proxy_gate(ticket) below for t in TICKETS: proxy_gate(t) 41:T785,KNOWN = [ "ada@harbor-clinic.org", "415-555-0142", "MBR-104221", "samir@harbor-clinic.org", ] MODEL_DRAFT = { "exc-441": "We'll call [PHONE_3] about [MEMBER_1] and email [EMAIL_2].", "exc-442": "We'll call [PHONE_2] about [MEMBER_1].", "exc-443": "We'll email [EMAIL_1].", } TICKETS = [ {"id": "exc-441", "reviewer": "vera", "text": "Member MBR-104221 at ada@harbor-clinic.org — late refill. Call 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, {"kind": "email", "value": "ada@harbor-clinic.org"}, {"kind": "phone", "value": "415-555-0142"}, ]}, {"id": "exc-442", "reviewer": "miguel", "text": "Member MBR-104221 called from 415-555-0142.", "pii": [ {"kind": "member", "value": "MBR-104221"}, ]}, {"id": "exc-443", "reviewer": "auto", "text": "Email samir@harbor-clinic.org about the missed window.", "pii": [ {"kind": "email", "value": "samir@harbor-clinic.org"}, ]}, ] def proxy_gate(ticket): masked = ticket["text"] token_map = {} for i, item in enumerate(ticket["pii"], 1): token = f"[{item['kind'].upper()}_{i}]" token_map[token] = item["value"] masked = masked.replace(item["value"], token) leftover = [v for v in KNOWN if v in masked] print(f"{ticket['id']}:") print(f" outbound: {masked}") if leftover: print(f" leftover: {', '.join(leftover)}") print(" verdict: BLOCKED — raw PII") return print(" leftover: none") if not ticket.get("reviewer") or ticket["reviewer"] == "auto": print(" verdict: BLOCKED — no named human") return restored = MODEL_DRAFT[ticket["id"]] for token, value in token_map.items(): restored = restored.replace(token, value) print(f" for {ticket['reviewer']}: {restored}") for t in TICKETS: proxy_gate(t) 1c:[["$","$L24",null,{"payload":{"catalog":"The SOP is data — get the process out of Vera's head without letting the model invent it (step 3/8) · ops studio · promptdojo","byPath":{"operations":"The SOP is data — get the process out of Vera's head without letting the model invent it (step 3/8) · ops studio · promptdojo"}}}],["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"$25"}}],["$","$L26",null,{"tree":{"toc":{"version":"0.1.0","generatedAt":"1970-01-01T00:00:00.000Z","chapters":[{"number":0,"slug":"before-you-build","title":"from worker to builder","blurb":"stop doing repeated work by hand. this chapter is the map: the builder loop, old workplace deliverables turned into site artifacts, and the reusable tool card. if you are starting here, those moves are the lesson. if you already ran a studio, you already used the loop.","status":"live","lessonCount":10,"stepCount":14,"liveLessonCount":10,"liveStepCount":14,"estMinutes":74,"xpTotal":36},{"number":1,"slug":"read-the-script","title":"read the script — your first code-reading lab","blurb":"ai hands you small scripts. this lab covers the three reads that matter: what the names hold, whether the function actually returns, and which branch of the if-chain really fires. if you are starting here, this labels the reads. if you already ran a studio, you already wrote these.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":21,"xpTotal":78},{"number":2,"slug":"read-the-data-shapes","title":"read the data shapes — the json ai just dumped","blurb":"every api response, config file, and llm reply is lists and dicts nested in a tree. read the shape on sight, walk it with a loop, and catch the wrong-layer bug ai loves to ship.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":23,"xpTotal":80},{"number":3,"slug":"read-the-failure","title":"read the failure — cursor wrote this and crashed","blurb":"when python crashes, it tells you exactly what happened and where. most non-engineers panic at the wall of text — and ai 'fixes' the crash by hiding it. learn to read the failure, then catch only what's worth catching.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":24,"xpTotal":83},{"number":4,"slug":"read-the-pipeline","title":"read the pipeline — files in, api calls out","blurb":"every real script reads a file or calls an api — usually both. learn the with-block, the status-code families, and the retry rules: the seatbelts ai forgets to put on.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":26,"xpTotal":83},{"number":7,"slug":"mutation-and-state","title":"mutation — why your code mysteriously breaks","blurb":"when a list inside a function changes the list outside the function, that's mutation. ai does this constantly without flagging it, and it's the bug class that takes the longest to find.","status":"live","lessonCount":2,"stepCount":9,"liveLessonCount":2,"liveStepCount":9,"estMinutes":10,"xpTotal":32},{"number":13,"slug":"llm-apis","title":"llm apis — talking to claude and openai","blurb":"every ai feature you ship eventually calls a model api. learn the messages pattern, how to read the response, and the four lines ai writes every single time.","status":"live","lessonCount":4,"stepCount":27,"liveLessonCount":4,"liveStepCount":27,"estMinutes":45,"xpTotal":87},{"number":14,"slug":"structured-output","title":"structured output — making the model return json","blurb":"free-form text breaks every pipeline. learn the schema-first pattern ai uses to get reliable json back, validate it with pydantic, and catch the model's lies before they hit prod.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":38,"xpTotal":80},{"number":15,"slug":"mcp","title":"mcp — the model context protocol","blurb":"mcp is the new standard for plugging tools and data sources into ai agents. learn what an mcp server actually is, how claude code lists tools, and why this is replacing one-off integrations everywhere.","status":"live","lessonCount":3,"stepCount":26,"liveLessonCount":3,"liveStepCount":26,"estMinutes":40,"xpTotal":85},{"number":16,"slug":"agent-loops","title":"agent loops — tool_use and the request/tool/respond cycle","blurb":"an agent isn't magic. it's a while loop. learn the actual cycle claude code, cursor, and every other agent uses: model returns tool_use, you run the tool, you send the result back, repeat until end_turn.","status":"live","lessonCount":6,"stepCount":48,"liveLessonCount":6,"liveStepCount":48,"estMinutes":71,"xpTotal":154},{"number":17,"slug":"git-and-github","title":"git and github cli — the ai builder's actual workflow","blurb":"cursor and claude code commit on your behalf. reading those commits — and undoing the bad ones — is your job. learn the three-state model, the commands you'll run every day, and what `gh` does that `git` can't.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":36,"xpTotal":79},{"number":18,"slug":"secrets-and-env","title":"secrets — .env, api keys, and what not to commit","blurb":"ai ships keys to github all the time. learn the .env pattern, why os.getenv is non-negotiable, what to do when a key leaks, and the gitignore lines you need on day one.","status":"live","lessonCount":2,"stepCount":16,"liveLessonCount":2,"liveStepCount":16,"estMinutes":16,"xpTotal":58},{"number":19,"slug":"prompting","title":"prompting ai coding tools","blurb":"an AI coding tool writes and edits with you — Cursor is one brand, Claude Code is another. the difference between a one-shot session and a four-hour spiral is almost always the first prompt.","status":"live","lessonCount":5,"stepCount":36,"liveLessonCount":5,"liveStepCount":36,"estMinutes":59,"xpTotal":118},{"number":20,"slug":"agent-traces","title":"reading agent traces and telemetry","blurb":"when an agent fails, the trace tells you exactly where. learn to read tool calls, tool results, and stop reasons — the json breadcrumbs every agent leaves behind.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":30,"xpTotal":88},{"number":21,"slug":"evals","title":"test the agent before you trust it","blurb":"same input, check the output, pass or fail. don't ship on a demo that felt right.","status":"live","lessonCount":5,"stepCount":34,"liveLessonCount":5,"liveStepCount":34,"estMinutes":57,"xpTotal":112},{"number":22,"slug":"context-and-retrieval","title":"context and retrieval — feeding the model real data","blurb":"rag without the overengineering. chunking, embeddings, vector search, and the small set of patterns that make a model answer from your data instead of its training set.","status":"live","lessonCount":5,"stepCount":44,"liveLessonCount":5,"liveStepCount":44,"estMinutes":75,"xpTotal":144},{"number":23,"slug":"production-tradeoffs","title":"production tradeoffs — cost, latency, quality","blurb":"the three numbers every shipped llm feature lives or dies by. token math, caching, streaming, batching, and the small set of decisions that move the product more than a model swap ever will.","status":"live","lessonCount":4,"stepCount":33,"liveLessonCount":4,"liveStepCount":33,"estMinutes":57,"xpTotal":111},{"number":24,"slug":"debugging-output","title":"debugging broken ai output","blurb":"when the model lies to your customer. the methodology for narrowing down what went wrong, the four most-common breakage classes, and the discipline that separates 'we shipped a fix' from 'we blamed the model and shrugged'.","status":"live","lessonCount":4,"stepCount":34,"liveLessonCount":4,"liveStepCount":34,"estMinutes":61,"xpTotal":112},{"number":25,"slug":"capstone","title":"mid-path build — ship the CLI agent","blurb":"wire it all together. the prompt, the call, the validation, the trace, the eval, the MCP tool. a path checkpoint, not the end of the course: the smallest end-to-end llm feature you could ship to a real user. retrieval and prompt-cache cost work live in later chapters — extend this build with them when you scale past the demo input set.","status":"live","lessonCount":7,"stepCount":62,"liveLessonCount":7,"liveStepCount":62,"estMinutes":100,"xpTotal":219},{"number":26,"slug":"agent-harnesses","title":"agent harnesses — the layer between you and the raw api","blurb":"claude code, cursor, aider, codex cli — they're all the same four layers wrapped around the same model api. learn what those layers are, what each adds, and what you'd build yourself if you had to. when the harness changes the world, the rollback lives in the repo — a runbook a junior can execute at 3am.","status":"live","lessonCount":5,"stepCount":37,"liveLessonCount":5,"liveStepCount":37,"estMinutes":69,"xpTotal":119},{"number":27,"slug":"ai-image-generation","title":"ai image generation — from prompt to production asset","blurb":"the 2026 image model landscape, the prompts that work, and the pipeline that turns one good idea into a hundred ready-to-ship images. nano banana 2 for volume, nano banana pro for fidelity, flux, midjourney, ideogram, gpt-image-2. when each wins and what they cost.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":60,"xpTotal":67},{"number":28,"slug":"ai-video-generation","title":"ai video generation — omni, veo 3.1, sora, and the second wave","blurb":"video is the hardest content type to generate, the most expensive, and the most strategically interesting. learn the 2026 model lineup, the camera-control patterns that separate slop from craft, and the cost math that decides whether your idea is viable.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":50,"xpTotal":61},{"number":29,"slug":"programmatic-design","title":"programmatic design — code-driven video and the new design pipeline","blurb":"ai generates raw assets; code stitches them into something shippable. hyperframes, remotion, claude design — when each tool wins, how they combine, and the data-driven workflows that turn one template into a hundred videos.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":46,"xpTotal":66},{"number":30,"slug":"harness-engineering","title":"harness engineering — the discipline behind the model","blurb":"every coding agent is a model plus a harness. the model is bought; the harness is engineered. learn the craft: how to ratchet rules from failures, fight context rot, design long-horizon loops, wire hooks as enforcement, and read the haas shift that's reshaping what you build vs buy.","status":"live","lessonCount":6,"stepCount":52,"liveLessonCount":6,"liveStepCount":52,"estMinutes":126,"xpTotal":143},{"number":31,"slug":"intro-to-terminal","title":"intro to terminal — the text way to drive your computer","blurb":"the terminal is a text box: type a name, press enter, the computer does it. by the end you can move around your files, make folders, and read files without the mouse. every later tool chapter assumes this.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":26,"xpTotal":50},{"number":32,"slug":"intro-to-claude-cli","title":"intro to claude cli — the door to real building","blurb":"you've used claude in a chat window. the claude cli is the same model with its hands on your actual files. this chapter installs it, signs you in, and runs your first real command. by the end you've watched an ai read, plan, and change things on your machine, and you know when to reach for the cli instead of the chat box.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":48},{"number":33,"slug":"intro-to-codex-cli","title":"intro to openai codex cli — a second tool in the kit","blurb":"you know the claude cli. the openai codex cli does the same job, an ai working in your terminal on your real files, with a different company behind it. this chapter installs it, signs you in, and runs your first command. most of what you already know carries straight over, so this chapter is mostly about what is different and when to reach for which.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":45},{"number":34,"slug":"team-skills","title":"claude skills for teams — playbooks your whole team shares","blurb":"a claude skill is a packaged set of instructions — your team's playbook — that claude loads when it's relevant, so nobody has to re-explain it. this chapter is for people who manage teams. it covers what a skill is, how a team shares and provisions skills, real examples for hr, legal, and ops work, when a skill beats a one-off prompt, and the governance you need before any skill touches real work.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":41,"xpTotal":60},{"number":35,"slug":"dataframes-numpy-pandas","title":"dataframes with numpy and pandas","blurb":"tables are the working surface of applied ml. learn rows, columns, missing values, joins, aggregates, and the dataframe habits ai-generated notebooks assume.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":43,"xpTotal":156},{"number":36,"slug":"sql-for-ml-datasets","title":"sql for ml datasets","blurb":"most training data starts in a database. learn the select, join, filter, aggregate, and leakage traps that decide whether a model is learning signal or nonsense.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":28,"xpTotal":104},{"number":37,"slug":"dataset-pipelines","title":"dataset formats, ingestion, and validation pipelines","blurb":"a dataset is a product surface. build ingestion, validation, partitions, manifests, and checkpoints so the next run is not a mystery.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":38,"slug":"ml-math-and-stats","title":"ml math and statistics that actually show up","blurb":"vectors, probability, distributions, correlation, and uncertainty are not trivia. they are how you read model behavior without worshipping it.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":39,"slug":"supervised-learning-workflows","title":"supervised learning workflows","blurb":"labels, splits, baselines, training, prediction, and evaluation. the supervised workflow is the first complete model loop.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":40,"xpTotal":130},{"number":40,"slug":"unsupervised-learning-and-embeddings","title":"unsupervised learning, embeddings, and recommenders","blurb":"not every useful model has labels. cluster, compare, retrieve, and recommend by turning examples into useful neighborhoods.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":32,"xpTotal":104},{"number":41,"slug":"metrics-and-error-analysis","title":"metrics, slices, and error analysis","blurb":"accuracy is a blunt instrument. learn confusion matrices, precision, recall, thresholds, slices, and failure notes so model quality has evidence.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":42,"slug":"pytorch-tensors-and-autograd","title":"pytorch tensors and autograd","blurb":"read tensor code without flinching. tensors, shapes, broadcasting, gradients, and autograd are the grammar of modern deep learning scripts.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":30,"xpTotal":104},{"number":43,"slug":"training-loops-and-optimizers","title":"training loops, backprop, optimizers, and schedulers","blurb":"the training loop is where models change. learn loss, gradients, optimizer steps, schedules, checkpoints, and the bugs ai ships there.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":44,"slug":"deep-learning-architectures","title":"cnns, transformers, and useful llm internals","blurb":"architecture literacy for builders: convolution, attention, tokens, decoding, kv cache, quantization, and what those choices do to cost and behavior.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":45,"slug":"feature-experiments-registries","title":"feature pipelines, experiment tracking, and registries","blurb":"features, runs, configs, artifacts, and registries are how ml work becomes repeatable instead of a lucky notebook.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":46,"slug":"model-serving-and-mlops","title":"model serving, ci/cd, and mlops","blurb":"serving a model means handling inputs, versions, routes, batch jobs, ci gates, rollback, and production failures deliberately.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":47,"slug":"monitoring-cloud-portfolio","title":"monitoring, drift, cloud scale, and portfolio launch","blurb":"the last mile: logs, drift, alerts, retraining decisions, cloud cost, gpu constraints, architecture docs, demos, and role stories.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":48,"slug":"graphic-design-studio","title":"graphic design studio — prompt, kit, and fifty on-brand assets","blurb":"how a working designer runs image models: prompts written like creative briefs, a brand bible that travels as structured data — with the design tokens that carry it onto screens — and the seeds, reference locks, and QA filters that make fifty generated assets look like one studio shipped them.","status":"live","lessonCount":3,"stepCount":27,"liveLessonCount":3,"liveStepCount":27,"estMinutes":73,"xpTotal":82},{"number":49,"slug":"campaign-studio","title":"campaign studio — one campaign, end to end, with the gate on","blurb":"brief to voice-locked copy variants to batch plan to calendar to the numbers pass. the whole campaign as data your scripts can check — and an approval gate nothing skips. built for the person who owns the send button.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":59,"xpTotal":69},{"number":50,"slug":"support-studio","title":"support studio — the reply drafter you can trust","blurb":"how a working support agent runs AI: a policy pack the drafter can't talk around, a banned-promises linter that holds every draft, golden eval sets that make each macro earn autonomy one intent at a time, and the escalation line where money, law, and anger stay human.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":61,"xpTotal":71},{"number":51,"slug":"copy-studio","title":"copy studio — voice from evidence, variants at scale, claims that clear","blurb":"a client's voice measured from their own published copy, variant batches that pass the voice gate or die with a named reason, and a claim check that blocks the ship button while any fact is unsourced. corrections-log culture, as code.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":66,"xpTotal":72},{"number":52,"slug":"analytics-studio","title":"analytics studio — messy export to defensible memo","blurb":"how a working analyst runs AI: a cleaning log that accounts for every dropped row, sample-first checks that catch the plausible-but-wrong aggregation, a memo where every number carries the address of the cell that made it, SR 11-7-style challenge against the AI's recommendation, and the rerun test — same data, same code, same memo, or it isn't done.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":61,"xpTotal":74},{"number":53,"slug":"delivery-studio","title":"delivery studio — direct the build without becoming the bottleneck","blurb":"how a working delivery lead runs AI builders: specs with acceptance checks written before the build, a review that runs the checks instead of admiring the demo, status roll-ups computed from acceptance records instead of standup vibes, the not-ready call made on evidence you can read out loud to a steering committee, and an hours-and-dollars receipt — before/after on a named function, finance's loaded rate, a named owner, the accepted thing that shipped — small enough to defend in a staff meeting.","status":"live","lessonCount":4,"stepCount":32,"liveLessonCount":4,"liveStepCount":32,"estMinutes":74,"xpTotal":96},{"number":54,"slug":"people-studio","title":"people studio — draft fast, decide human, prove everything","blurb":"how a working hr specialist runs AI: a data firewall that keeps employee records out of prompts by code instead of willpower, recruiting briefs where every claim cites a resume line, policy drafts that cannot publish without a named legal reviewer, and the adverse-action paper trail that survives a discrimination claim.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":56,"xpTotal":72},{"number":55,"slug":"ops-studio","title":"ops studio — tribal knowledge to running automation","blurb":"how a working ops lead gets the process out of one person's head and into automation that can be trusted: SOPs captured as versioned data with the invented steps hunted down, recurring checklists with an explicit auto-run vs named-approval line, exception queues instead of silent failures, a receipt against the export for every number in the weekly report, and a PII proxy that masks member identifiers before the model and restores them only for the named human who sends.","status":"live","lessonCount":4,"stepCount":32,"liveLessonCount":4,"liveStepCount":32,"estMinutes":80,"xpTotal":96},{"number":56,"slug":"legal-studio","title":"legal studio — the cite-check gate and the log that survives","blurb":"how a working lawyer runs AI: a citation-verification drill where every authority gets opened at the source (the fabricated ones always look right), clause extraction that's worthless without a pinpoint, privilege rules written as a routing table instead of remembered under deadline, and the review log you'd want in the record when opposing counsel moves for sanctions.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":60,"xpTotal":69},{"number":57,"slug":"agent-studio","title":"agent studio — ship one agent feature before you stand up","blurb":"one sitting, one agent feature, end to end: a tool schema that makes the dangerous call a validation error, a loop that meters its own spend, golden cases wired before the router works, a trace you debug from instead of vibes, and the ratchet that turns tonight's incident into tomorrow's eval. practice at production tempo.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":66,"xpTotal":73}]},"accessToc":{"chapters":[{"slug":"before-you-build","lessons":[{"slug":"from-worker-to-builder","steps":[{"id":"before-you-build/from-worker-to-builder/s01-01-intro"}]},{"slug":"upgrade-dull-artifacts-into-interactive-sites","steps":[{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s01-01-intro"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s02-02-build-brief"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s03-03-your-artifact-brief"}]},{"slug":"why-building-matters-now","steps":[{"id":"before-you-build/why-building-matters-now/s01-01-intro"}]},{"slug":"the-builder-loop","steps":[{"id":"before-you-build/the-builder-loop/s01-01-intro"}]},{"slug":"your-first-builder-brief","steps":[{"id":"before-you-build/your-first-builder-brief/s01-01-intro"}]},{"slug":"what-the-ai-is-doing","steps":[{"id":"before-you-build/what-the-ai-is-doing/s01-01-intro"}]},{"slug":"build-your-first-reusable-tool","steps":[{"id":"before-you-build/build-your-first-reusable-tool/s01-01-intro"}]},{"slug":"from-prompt-to-project","steps":[{"id":"before-you-build/from-prompt-to-project/s01-01-intro"}]},{"slug":"why-code-appears-later","steps":[{"id":"before-you-build/why-code-appears-later/s01-01-intro"}]},{"slug":"use-it-again","steps":[{"id":"before-you-build/use-it-again/s01-01-intro"},{"id":"before-you-build/use-it-again/s02-02-which-fix-compounds"},{"id":"before-you-build/use-it-again/s03-03-save-version-two"}]}]},{"slug":"read-the-script","lessons":[{"slug":"names-and-values","steps":[{"id":"read-the-script/names-and-values/s01-01-intro"},{"id":"read-the-script/names-and-values/s02-02-spot-the-type"},{"id":"read-the-script/names-and-values/s03-03-assignment"},{"id":"read-the-script/names-and-values/s04-04-predict-the-score"},{"id":"read-the-script/names-and-values/s05-05-fill-the-name"},{"id":"read-the-script/names-and-values/s06-06-fix-the-typo"},{"id":"read-the-script/names-and-values/s07-07-write-greeting"},{"id":"read-the-script/names-and-values/s08-08-checkpoint"}]},{"slug":"functions-and-the-hallucinated-return","steps":[{"id":"read-the-script/functions-and-the-hallucinated-return/s01-01-intro"},{"id":"read-the-script/functions-and-the-hallucinated-return/s02-02-which-returns"},{"id":"read-the-script/functions-and-the-hallucinated-return/s03-03-return-vs-print"},{"id":"read-the-script/functions-and-the-hallucinated-return/s04-04-predict-the-result"},{"id":"read-the-script/functions-and-the-hallucinated-return/s05-05-fix-the-missing-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s06-06-fix-print-vs-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s07-07-write-square"},{"id":"read-the-script/functions-and-the-hallucinated-return/s08-08-checkpoint"}]},{"slug":"decisions-if-elif","steps":[{"id":"read-the-script/decisions-if-elif/s01-01-intro"},{"id":"read-the-script/decisions-if-elif/s02-02-truthiness-quiz"},{"id":"read-the-script/decisions-if-elif/s03-03-eq-vs-is"},{"id":"read-the-script/decisions-if-elif/s04-04-predict-the-empty-list"},{"id":"read-the-script/decisions-if-elif/s05-05-fix-the-zero-bug"},{"id":"read-the-script/decisions-if-elif/s06-06-fix-the-elif-order"},{"id":"read-the-script/decisions-if-elif/s07-07-checkpoint"},{"id":"read-the-script/decisions-if-elif/s08-08-going-deeper"}]}]},{"slug":"read-the-data-shapes","lessons":[{"slug":"the-shapes-ai-dumps-on-you","steps":[{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s01-01-intro"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s02-02-which-is-which"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s03-03-nested-paths"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s04-04-predict-the-deep-key"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s05-05-fill-the-index"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s06-06-fix-the-keyerror"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s07-07-fix-the-list-vs-dict"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s08-08-checkpoint"}]},{"slug":"walking-the-shape","steps":[{"id":"read-the-data-shapes/walking-the-shape/s01-01-intro"},{"id":"read-the-data-shapes/walking-the-shape/s02-02-which-loop-shape"},{"id":"read-the-data-shapes/walking-the-shape/s03-03-walking-the-shape"},{"id":"read-the-data-shapes/walking-the-shape/s04-04-predict-the-list-loop"},{"id":"read-the-data-shapes/walking-the-shape/s05-05-fill-the-loop-variable"},{"id":"read-the-data-shapes/walking-the-shape/s06-06-fix-the-dict-loop"},{"id":"read-the-data-shapes/walking-the-shape/s07-07-write-sum-loop"},{"id":"read-the-data-shapes/walking-the-shape/s08-08-checkpoint"}]},{"slug":"comprehensions-and-the-wrong-layer-bug","steps":[{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s01-01-intro"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s02-02-which-comp-shape"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s03-03-with-filter"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s04-04-predict-the-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s05-05-fix-the-wrong-layer"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s06-06-write-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s07-07-checkpoint"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s08-08-going-deeper"}]}]},{"slug":"read-the-failure","lessons":[{"slug":"read-bottom-up","steps":[{"id":"read-the-failure/read-bottom-up/s01-01-anatomy"},{"id":"read-the-failure/read-bottom-up/s02-02-which-error"},{"id":"read-the-failure/read-bottom-up/s03-03-the-five-fingerprints"},{"id":"read-the-failure/read-bottom-up/s04-04-predict-the-class"},{"id":"read-the-failure/read-bottom-up/s05-05-fix-nameerror"},{"id":"read-the-failure/read-bottom-up/s06-06-fix-attributeerror"},{"id":"read-the-failure/read-bottom-up/s07-07-write-defensive"},{"id":"read-the-failure/read-bottom-up/s08-08-checkpoint"}]},{"slug":"try-except-what-ai-misuses","steps":[{"id":"read-the-failure/try-except-what-ai-misuses/s01-01-intro"},{"id":"read-the-failure/try-except-what-ai-misuses/s02-02-which-block-runs"},{"id":"read-the-failure/try-except-what-ai-misuses/s03-03-naming-the-error"},{"id":"read-the-failure/try-except-what-ai-misuses/s04-04-predict-the-branch"},{"id":"read-the-failure/try-except-what-ai-misuses/s05-05-fill-the-keyword"},{"id":"read-the-failure/try-except-what-ai-misuses/s06-06-fix-the-bare-except"},{"id":"read-the-failure/try-except-what-ai-misuses/s07-07-write-safe-lookup"},{"id":"read-the-failure/try-except-what-ai-misuses/s08-08-checkpoint"}]},{"slug":"fail-loudly","steps":[{"id":"read-the-failure/fail-loudly/s01-01-intro"},{"id":"read-the-failure/fail-loudly/s02-02-which-raise-fires"},{"id":"read-the-failure/fail-loudly/s03-03-predict-the-raise"},{"id":"read-the-failure/fail-loudly/s04-04-fill-the-raise-keyword"},{"id":"read-the-failure/fail-loudly/s05-05-fix-the-silent-swallow"},{"id":"read-the-failure/fail-loudly/s06-06-write-validate-age"},{"id":"read-the-failure/fail-loudly/s07-07-checkpoint"},{"id":"read-the-failure/fail-loudly/s08-08-going-deeper"}]}]},{"slug":"read-the-pipeline","lessons":[{"slug":"files-without-fear","steps":[{"id":"read-the-pipeline/files-without-fear/s01-01-intro"},{"id":"read-the-pipeline/files-without-fear/s02-02-which-mode"},{"id":"read-the-pipeline/files-without-fear/s03-03-pathlib-and-manifests"},{"id":"read-the-pipeline/files-without-fear/s04-04-predict-the-suffix"},{"id":"read-the-pipeline/files-without-fear/s05-05-fill-the-mode"},{"id":"read-the-pipeline/files-without-fear/s06-06-fix-the-wrong-mode"},{"id":"read-the-pipeline/files-without-fear/s07-07-write-manifest-entry"},{"id":"read-the-pipeline/files-without-fear/s08-08-checkpoint"}]},{"slug":"talking-to-apis","steps":[{"id":"read-the-pipeline/talking-to-apis/s01-01-intro"},{"id":"read-the-pipeline/talking-to-apis/s02-02-which-status-family"},{"id":"read-the-pipeline/talking-to-apis/s03-03-parsing-the-json"},{"id":"read-the-pipeline/talking-to-apis/s04-04-predict-the-key"},{"id":"read-the-pipeline/talking-to-apis/s05-05-fill-the-method"},{"id":"read-the-pipeline/talking-to-apis/s06-06-fix-the-status-check"},{"id":"read-the-pipeline/talking-to-apis/s07-07-write-classify-status"},{"id":"read-the-pipeline/talking-to-apis/s08-08-checkpoint"}]},{"slug":"when-the-pipeline-breaks","steps":[{"id":"read-the-pipeline/when-the-pipeline-breaks/s01-01-intro"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s02-02-honor-the-retry-after"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s03-03-predict-the-status-handling"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s04-04-fill-the-raise-for-status"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s05-05-fix-the-missing-check"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s06-06-fix-the-broad-retry"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s07-07-checkpoint"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s08-08-going-deeper"}]}]},{"slug":"mutation-and-state","lessons":[{"slug":"why-it-breaks","steps":[{"id":"mutation-and-state/why-it-breaks/s01-01-intro"},{"id":"mutation-and-state/why-it-breaks/s02-02-which-mutates"},{"id":"mutation-and-state/why-it-breaks/s03-03-shared-reference"},{"id":"mutation-and-state/why-it-breaks/s04-04-fix-the-shared-list"},{"id":"mutation-and-state/why-it-breaks/s05-05-checkpoint"}]},{"slug":"copy-vs-reference","steps":[{"id":"mutation-and-state/copy-vs-reference/s01-01-deep-copy"},{"id":"mutation-and-state/copy-vs-reference/s02-02-predict-the-shallow"},{"id":"mutation-and-state/copy-vs-reference/s03-03-fix-the-shallow-bug"},{"id":"mutation-and-state/copy-vs-reference/s04-04-checkpoint"}]}]},{"slug":"llm-apis","lessons":[{"slug":"before-the-api-what-the-model-is-doing","steps":[{"id":"llm-apis/before-the-api-what-the-model-is-doing/s01-01-intro"}]},{"slug":"the-messages-pattern","steps":[{"id":"llm-apis/the-messages-pattern/s01-01-intro"},{"id":"llm-apis/the-messages-pattern/s02-02-which-role-is-yours"},{"id":"llm-apis/the-messages-pattern/s03-03-reading-the-response"},{"id":"llm-apis/the-messages-pattern/s04-04-predict-the-text"},{"id":"llm-apis/the-messages-pattern/s05-05-fill-the-role"},{"id":"llm-apis/the-messages-pattern/s06-06-fix-the-message-shape"},{"id":"llm-apis/the-messages-pattern/s07-07-fix-the-response-access"},{"id":"llm-apis/the-messages-pattern/s08-08-write-ask-claude"},{"id":"llm-apis/the-messages-pattern/s09-09-checkpoint"}]},{"slug":"reading-the-response","steps":[{"id":"llm-apis/reading-the-response/s01-01-intro"},{"id":"llm-apis/reading-the-response/s02-02-which-block-do-you-read"},{"id":"llm-apis/reading-the-response/s03-03-the-five-stop-reasons"},{"id":"llm-apis/reading-the-response/s04-04-predict-the-text"},{"id":"llm-apis/reading-the-response/s05-05-fill-the-text-extract"},{"id":"llm-apis/reading-the-response/s06-06-fix-the-content-zero"},{"id":"llm-apis/reading-the-response/s07-07-fix-the-stop-assumption"},{"id":"llm-apis/reading-the-response/s08-08-write-the-extractor"},{"id":"llm-apis/reading-the-response/s09-09-checkpoint"}]},{"slug":"the-model-picker","steps":[{"id":"llm-apis/the-model-picker/s01-01-why-model-choice-is-a-product-decision"},{"id":"llm-apis/the-model-picker/s02-02-haiku-territory"},{"id":"llm-apis/the-model-picker/s03-03-which-model-here"},{"id":"llm-apis/the-model-picker/s04-04-sonnet-territory"},{"id":"llm-apis/the-model-picker/s05-05-opus-and-the-thinking-budget"},{"id":"llm-apis/the-model-picker/s06-06-pick-the-model"},{"id":"llm-apis/the-model-picker/s07-07-write-the-model-routing-memo"},{"id":"llm-apis/the-model-picker/s08-08-checkpoint"}]}]},{"slug":"structured-output","lessons":[{"slug":"getting-json-back","steps":[{"id":"structured-output/getting-json-back/s01-01-intro"},{"id":"structured-output/getting-json-back/s02-02-why-validate"},{"id":"structured-output/getting-json-back/s03-03-pydantic-models"},{"id":"structured-output/getting-json-back/s04-04-predict-the-validated"},{"id":"structured-output/getting-json-back/s05-05-fill-the-field-type"},{"id":"structured-output/getting-json-back/s06-06-fix-the-required-field"},{"id":"structured-output/getting-json-back/s07-07-fix-the-type-mismatch"},{"id":"structured-output/getting-json-back/s08-08-write-parse-extraction"},{"id":"structured-output/getting-json-back/s09-09-checkpoint"}]},{"slug":"schemas-eat-prompts","steps":[{"id":"structured-output/schemas-eat-prompts/s01-01-the-trust-boundary"},{"id":"structured-output/schemas-eat-prompts/s02-02-three-shipped-breakages"},{"id":"structured-output/schemas-eat-prompts/s03-03-which-boundary-broke"},{"id":"structured-output/schemas-eat-prompts/s04-04-schema-as-prompt-replacement"},{"id":"structured-output/schemas-eat-prompts/s05-05-where-validation-belongs"},{"id":"structured-output/schemas-eat-prompts/s06-06-the-pydantic-migration-as-war-story"},{"id":"structured-output/schemas-eat-prompts/s07-07-write-the-boundary-audit"},{"id":"structured-output/schemas-eat-prompts/s08-08-checkpoint"}]},{"slug":"dataset-schema-validation","steps":[{"id":"structured-output/dataset-schema-validation/s01-01-validate-at-the-boundary"},{"id":"structured-output/dataset-schema-validation/s02-02-the-silent-drop"},{"id":"structured-output/dataset-schema-validation/s03-03-predict-the-missing-fields"},{"id":"structured-output/dataset-schema-validation/s04-04-fill-the-membership-test"},{"id":"structured-output/dataset-schema-validation/s05-05-fix-the-range-check"},{"id":"structured-output/dataset-schema-validation/s06-06-write-validate-batch"},{"id":"structured-output/dataset-schema-validation/s07-07-claims-with-receipts"},{"id":"structured-output/dataset-schema-validation/s08-08-checkpoint"}]}]},{"slug":"mcp","lessons":[{"slug":"what-mcp-is","steps":[{"id":"mcp/what-mcp-is/s01-01-intro"},{"id":"mcp/what-mcp-is/s02-02-which-side-is-the-server"},{"id":"mcp/what-mcp-is/s03-03-the-tool-list"},{"id":"mcp/what-mcp-is/s04-04-predict-the-tool-result"},{"id":"mcp/what-mcp-is/s05-05-fill-the-tool-name"},{"id":"mcp/what-mcp-is/s06-06-fix-the-missing-argument"},{"id":"mcp/what-mcp-is/s07-07-fix-the-result-shape"},{"id":"mcp/what-mcp-is/s08-08-write-call-tool"},{"id":"mcp/what-mcp-is/s09-09-checkpoint"}]},{"slug":"writing-a-tiny-mcp-server","steps":[{"id":"mcp/writing-a-tiny-mcp-server/s01-01-intro"},{"id":"mcp/writing-a-tiny-mcp-server/s02-02-which-message-is-call-tool"},{"id":"mcp/writing-a-tiny-mcp-server/s03-03-the-tool-registry"},{"id":"mcp/writing-a-tiny-mcp-server/s04-04-predict-the-list-tools-output"},{"id":"mcp/writing-a-tiny-mcp-server/s05-05-fill-the-input-schema"},{"id":"mcp/writing-a-tiny-mcp-server/s06-06-fix-the-missing-error-response"},{"id":"mcp/writing-a-tiny-mcp-server/s07-07-fix-the-validation-gap"},{"id":"mcp/writing-a-tiny-mcp-server/s08-08-write-the-call-tool-dispatcher"},{"id":"mcp/writing-a-tiny-mcp-server/s09-09-checkpoint"}]},{"slug":"why-mcp-won","steps":[{"id":"mcp/why-mcp-won/s01-01-the-graveyard-of-tool-protocols"},{"id":"mcp/why-mcp-won/s02-02-what-openai-plugins-got-wrong"},{"id":"mcp/why-mcp-won/s03-03-which-failure-mode"},{"id":"mcp/why-mcp-won/s04-04-the-mcp-bet"},{"id":"mcp/why-mcp-won/s05-05-the-network-effect"},{"id":"mcp/why-mcp-won/s06-06-spot-the-mcp-wedge"},{"id":"mcp/why-mcp-won/s07-07-write-the-protocol-postmortem"},{"id":"mcp/why-mcp-won/s08-08-checkpoint"}]}]},{"slug":"agent-loops","lessons":[{"slug":"the-loop","steps":[{"id":"agent-loops/the-loop/s01-01-intro"},{"id":"agent-loops/the-loop/s02-02-which-stop-reason"},{"id":"agent-loops/the-loop/s03-03-tool-result-shape"},{"id":"agent-loops/the-loop/s04-04-predict-the-next-call"},{"id":"agent-loops/the-loop/s05-05-fill-the-stop-condition"},{"id":"agent-loops/the-loop/s06-06-fix-the-tool-id"},{"id":"agent-loops/the-loop/s07-07-fix-the-loop-exit"},{"id":"agent-loops/the-loop/s08-08-write-the-loop"},{"id":"agent-loops/the-loop/s09-09-checkpoint"}]},{"slug":"multi-step-tools","steps":[{"id":"agent-loops/multi-step-tools/s01-01-intro"},{"id":"agent-loops/multi-step-tools/s02-02-which-tool-runs"},{"id":"agent-loops/multi-step-tools/s03-03-tool-registry"},{"id":"agent-loops/multi-step-tools/s04-04-predict-the-chain"},{"id":"agent-loops/multi-step-tools/s05-05-fill-the-dispatch"},{"id":"agent-loops/multi-step-tools/s06-06-fix-the-missing-tool"},{"id":"agent-loops/multi-step-tools/s07-07-fix-the-parallel-ids"},{"id":"agent-loops/multi-step-tools/s08-08-write-multi-tool-agent"},{"id":"agent-loops/multi-step-tools/s09-09-checkpoint"}]},{"slug":"routing","steps":[{"id":"agent-loops/routing/s01-01-intro"},{"id":"agent-loops/routing/s02-02-when-to-route"},{"id":"agent-loops/routing/s03-03-classifier-output"},{"id":"agent-loops/routing/s04-04-predict-the-route"},{"id":"agent-loops/routing/s05-05-fill-the-dispatch"},{"id":"agent-loops/routing/s06-06-fix-the-fallback"},{"id":"agent-loops/routing/s07-07-fix-the-loose-classifier"},{"id":"agent-loops/routing/s08-08-write-the-router"},{"id":"agent-loops/routing/s09-09-checkpoint"}]},{"slug":"evaluator-optimizer","steps":[{"id":"agent-loops/evaluator-optimizer/s01-01-intro"},{"id":"agent-loops/evaluator-optimizer/s02-02-when-it-earns-its-keep"},{"id":"agent-loops/evaluator-optimizer/s03-03-judge-output-shape"},{"id":"agent-loops/evaluator-optimizer/s04-04-predict-the-iterations"},{"id":"agent-loops/evaluator-optimizer/s05-05-fill-the-exit"},{"id":"agent-loops/evaluator-optimizer/s06-06-fix-the-judge-parsing"},{"id":"agent-loops/evaluator-optimizer/s07-07-fix-the-missing-cap"},{"id":"agent-loops/evaluator-optimizer/s08-08-write-the-evaluator-optimizer"},{"id":"agent-loops/evaluator-optimizer/s09-09-checkpoint"}]},{"slug":"why-every-framework-is-thirty-lines","steps":[{"id":"agent-loops/why-every-framework-is-thirty-lines/s01-01-the-frameworks-converged"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s02-02-langgraph-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s03-03-the-work-is-a-graph"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s04-04-which-framework-fits"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s05-05-vercel-ai-sdk-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s06-06-the-five-canonical-patterns"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s07-07-when-to-stay-framework-free"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s08-08-write-the-tradeoff-memo"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s09-09-checkpoint"}]},{"slug":"the-industry-map","steps":[{"id":"agent-loops/the-industry-map/s01-01-the-five-layers"},{"id":"agent-loops/the-industry-map/s02-02-which-layer-failed"},{"id":"agent-loops/the-industry-map/s03-03-the-shape-of-the-stack"}]}]},{"slug":"git-and-github","lessons":[{"slug":"the-three-states","steps":[{"id":"git-and-github/the-three-states/s01-01-intro"},{"id":"git-and-github/the-three-states/s02-02-which-command-stages"},{"id":"git-and-github/the-three-states/s03-03-status-and-diff"},{"id":"git-and-github/the-three-states/s04-04-predict-the-status"},{"id":"git-and-github/the-three-states/s05-05-fill-the-command"},{"id":"git-and-github/the-three-states/s06-06-fix-the-staged-files"},{"id":"git-and-github/the-three-states/s07-07-fix-the-pr-command"},{"id":"git-and-github/the-three-states/s08-08-write-status-summary"},{"id":"git-and-github/the-three-states/s09-09-checkpoint"}]},{"slug":"three-git-disasters-ai-shipped","steps":[{"id":"git-and-github/three-git-disasters-ai-shipped/s01-01-why-postmortems-teach-better"},{"id":"git-and-github/three-git-disasters-ai-shipped/s02-02-uber-2016-credentials-in-repo"},{"id":"git-and-github/three-git-disasters-ai-shipped/s03-03-which-control-would-have-caught-it"},{"id":"git-and-github/three-git-disasters-ai-shipped/s04-04-the-samsung-paste"},{"id":"git-and-github/three-git-disasters-ai-shipped/s05-05-the-cursor-git-add-dot-default"},{"id":"git-and-github/three-git-disasters-ai-shipped/s06-06-spot-the-failure-mode"},{"id":"git-and-github/three-git-disasters-ai-shipped/s07-07-write-your-pre-commit-checklist"},{"id":"git-and-github/three-git-disasters-ai-shipped/s08-08-checkpoint"}]},{"slug":"github-actions-for-model-checks","steps":[{"id":"git-and-github/github-actions-for-model-checks/s01-01-intro"},{"id":"git-and-github/github-actions-for-model-checks/s02-02-green-but-wrong"},{"id":"git-and-github/github-actions-for-model-checks/s03-03-predict-the-run"},{"id":"git-and-github/github-actions-for-model-checks/s04-04-fill-the-trigger"},{"id":"git-and-github/github-actions-for-model-checks/s05-05-fix-the-eval-gate"},{"id":"git-and-github/github-actions-for-model-checks/s06-06-write-run-job"},{"id":"git-and-github/github-actions-for-model-checks/s07-07-checkpoint"}]}]},{"slug":"secrets-and-env","lessons":[{"slug":"keeping-keys-safe","steps":[{"id":"secrets-and-env/keeping-keys-safe/s01-01-intro"},{"id":"secrets-and-env/keeping-keys-safe/s02-02-which-line-leaks"},{"id":"secrets-and-env/keeping-keys-safe/s03-03-loading-env-vars"},{"id":"secrets-and-env/keeping-keys-safe/s04-04-predict-the-key"},{"id":"secrets-and-env/keeping-keys-safe/s05-05-fill-the-getenv"},{"id":"secrets-and-env/keeping-keys-safe/s06-06-fix-the-hardcoded-key"},{"id":"secrets-and-env/keeping-keys-safe/s07-07-fix-the-missing-default"},{"id":"secrets-and-env/keeping-keys-safe/s08-08-write-load-config"},{"id":"secrets-and-env/keeping-keys-safe/s09-09-checkpoint"}]},{"slug":"ml-service-secrets","steps":[{"id":"secrets-and-env/ml-service-secrets/s01-01-intro"},{"id":"secrets-and-env/ml-service-secrets/s02-02-which-log-line-leaks"},{"id":"secrets-and-env/ml-service-secrets/s03-03-predict-the-scan"},{"id":"secrets-and-env/ml-service-secrets/s04-04-fill-the-lookup"},{"id":"secrets-and-env/ml-service-secrets/s05-05-fix-the-logged-key"},{"id":"secrets-and-env/ml-service-secrets/s06-06-write-scan-logs"},{"id":"secrets-and-env/ml-service-secrets/s07-07-checkpoint"}]}]},{"slug":"prompting","lessons":[{"slug":"builder-briefs-before-prompts","steps":[{"id":"prompting/builder-briefs-before-prompts/s01-01-intro"}]},{"slug":"the-prompt-craft","steps":[{"id":"prompting/the-prompt-craft/s01-01-intro"},{"id":"prompting/the-prompt-craft/s02-02-which-prompt-is-better"},{"id":"prompting/the-prompt-craft/s03-03-context-rot"},{"id":"prompting/the-prompt-craft/s04-04-predict-the-output"},{"id":"prompting/the-prompt-craft/s05-05-fill-the-constraint"},{"id":"prompting/the-prompt-craft/s06-06-fix-the-vague-prompt"},{"id":"prompting/the-prompt-craft/s07-07-fix-the-bloated-context"},{"id":"prompting/the-prompt-craft/s08-08-write-the-prompt"},{"id":"prompting/the-prompt-craft/s09-09-checkpoint"}]},{"slug":"few-shot-and-reasoning","steps":[{"id":"prompting/few-shot-and-reasoning/s01-01-intro"},{"id":"prompting/few-shot-and-reasoning/s02-02-when-few-shot-helps"},{"id":"prompting/few-shot-and-reasoning/s03-03-the-cot-trap"},{"id":"prompting/few-shot-and-reasoning/s04-04-predict-the-format-lock"},{"id":"prompting/few-shot-and-reasoning/s05-05-fill-the-example-format"},{"id":"prompting/few-shot-and-reasoning/s06-06-fix-the-cot-on-reasoning"},{"id":"prompting/few-shot-and-reasoning/s07-07-fix-the-format-mismatch"},{"id":"prompting/few-shot-and-reasoning/s08-08-write-the-prompt-builder"},{"id":"prompting/few-shot-and-reasoning/s09-09-checkpoint"}]},{"slug":"agent-config-files","steps":[{"id":"prompting/agent-config-files/s01-01-intro"},{"id":"prompting/agent-config-files/s02-02-which-belongs-in-the-file"},{"id":"prompting/agent-config-files/s03-03-anatomy-of-a-good-file"},{"id":"prompting/agent-config-files/s04-04-predict-which-rule-fires"},{"id":"prompting/agent-config-files/s05-05-fill-the-rule-block"},{"id":"prompting/agent-config-files/s06-06-fix-the-too-vague"},{"id":"prompting/agent-config-files/s07-07-fix-the-stale-pointer"},{"id":"prompting/agent-config-files/s08-08-write-a-claude-md"},{"id":"prompting/agent-config-files/s09-09-checkpoint"}]},{"slug":"what-aged-in-prompting","steps":[{"id":"prompting/what-aged-in-prompting/s01-01-prompting-as-a-moving-target"},{"id":"prompting/what-aged-in-prompting/s02-02-cot-then-vs-now"},{"id":"prompting/what-aged-in-prompting/s03-03-which-still-works"},{"id":"prompting/what-aged-in-prompting/s04-04-the-system-prompt-arms-race"},{"id":"prompting/what-aged-in-prompting/s05-05-spot-the-stale-technique"},{"id":"prompting/what-aged-in-prompting/s06-06-the-prompt-engineer-debate"},{"id":"prompting/what-aged-in-prompting/s07-07-write-the-cheatsheet-update"},{"id":"prompting/what-aged-in-prompting/s08-08-checkpoint"}]}]},{"slug":"agent-traces","lessons":[{"slug":"reading-the-trace","steps":[{"id":"agent-traces/reading-the-trace/s01-01-intro"},{"id":"agent-traces/reading-the-trace/s02-02-which-turn-failed"},{"id":"agent-traces/reading-the-trace/s03-03-the-stop-reasons"},{"id":"agent-traces/reading-the-trace/s04-04-predict-the-stop"},{"id":"agent-traces/reading-the-trace/s05-05-fill-the-tool-name"},{"id":"agent-traces/reading-the-trace/s06-06-fix-the-loop"},{"id":"agent-traces/reading-the-trace/s07-07-fix-the-arg-shape"},{"id":"agent-traces/reading-the-trace/s08-08-write-the-summarizer"},{"id":"agent-traces/reading-the-trace/s09-09-checkpoint"}]},{"slug":"trace-driven-debugging","steps":[{"id":"agent-traces/trace-driven-debugging/s01-01-intro"},{"id":"agent-traces/trace-driven-debugging/s02-02-which-pattern-is-this"},{"id":"agent-traces/trace-driven-debugging/s03-03-the-loop-signature"},{"id":"agent-traces/trace-driven-debugging/s04-04-predict-the-call-count"},{"id":"agent-traces/trace-driven-debugging/s05-05-fill-the-detector"},{"id":"agent-traces/trace-driven-debugging/s06-06-fix-the-tool-routing"},{"id":"agent-traces/trace-driven-debugging/s07-07-fix-the-prompt-bloat"},{"id":"agent-traces/trace-driven-debugging/s08-08-write-the-trace-summary"},{"id":"agent-traces/trace-driven-debugging/s09-09-checkpoint"}]},{"slug":"prediction-ids-and-inference-logs","steps":[{"id":"agent-traces/prediction-ids-and-inference-logs/s01-01-intro"},{"id":"agent-traces/prediction-ids-and-inference-logs/s02-02-pick-the-log-line"},{"id":"agent-traces/prediction-ids-and-inference-logs/s03-03-predict-the-ids"},{"id":"agent-traces/prediction-ids-and-inference-logs/s04-04-fill-the-join-key"},{"id":"agent-traces/prediction-ids-and-inference-logs/s05-05-fix-the-join"},{"id":"agent-traces/prediction-ids-and-inference-logs/s06-06-write-the-logger"},{"id":"agent-traces/prediction-ids-and-inference-logs/s07-07-checkpoint"}]}]},{"slug":"evals","lessons":[{"slug":"check-before-you-trust","steps":[{"id":"evals/check-before-you-trust/s01-01-intro"}]},{"slug":"writing-evals","steps":[{"id":"evals/writing-evals/s01-01-intro"},{"id":"evals/writing-evals/s02-02-which-eval-passes"},{"id":"evals/writing-evals/s03-03-eval-patterns"},{"id":"evals/writing-evals/s04-04-predict-the-pass-rate"},{"id":"evals/writing-evals/s05-05-fill-the-assertion"},{"id":"evals/writing-evals/s06-06-fix-the-flaky-eval"},{"id":"evals/writing-evals/s07-07-fix-the-overstrict-eval"},{"id":"evals/writing-evals/s08-08-write-the-suite"},{"id":"evals/writing-evals/s09-09-checkpoint"}]},{"slug":"llm-as-judge","steps":[{"id":"evals/llm-as-judge/s01-01-intro"},{"id":"evals/llm-as-judge/s02-02-pairwise-or-rubric"},{"id":"evals/llm-as-judge/s03-03-the-four-biases"},{"id":"evals/llm-as-judge/s04-04-predict-the-bias"},{"id":"evals/llm-as-judge/s05-05-fill-the-rubric-call"},{"id":"evals/llm-as-judge/s06-06-fix-the-likert-trap"},{"id":"evals/llm-as-judge/s07-07-fix-the-position-bias"},{"id":"evals/llm-as-judge/s08-08-write-the-rubric-judge"},{"id":"evals/llm-as-judge/s09-09-checkpoint"}]},{"slug":"the-rise-of-evals-as-a-discipline","steps":[{"id":"evals/the-rise-of-evals-as-a-discipline/s01-01-the-prompt-engineering-era"},{"id":"evals/the-rise-of-evals-as-a-discipline/s02-02-the-eval-turn"},{"id":"evals/the-rise-of-evals-as-a-discipline/s03-03-which-company-survives-a-model-swap"},{"id":"evals/the-rise-of-evals-as-a-discipline/s04-04-case-studies"},{"id":"evals/the-rise-of-evals-as-a-discipline/s05-05-eval-first-vs-prompt-first"},{"id":"evals/the-rise-of-evals-as-a-discipline/s06-06-why-the-eval-engineer-eats-the-prompt-engineer"},{"id":"evals/the-rise-of-evals-as-a-discipline/s07-07-write-the-eval-readiness-audit"},{"id":"evals/the-rise-of-evals-as-a-discipline/s08-08-checkpoint"}]},{"slug":"model-validation-gates","steps":[{"id":"evals/model-validation-gates/s01-01-intro"},{"id":"evals/model-validation-gates/s02-02-spot-the-theater"},{"id":"evals/model-validation-gates/s03-03-predict-the-decision"},{"id":"evals/model-validation-gates/s04-04-fill-the-regression-check"},{"id":"evals/model-validation-gates/s05-05-fix-the-theater-gate"},{"id":"evals/model-validation-gates/s06-06-write-the-gate-runner"},{"id":"evals/model-validation-gates/s07-07-checkpoint"}]}]},{"slug":"context-and-retrieval","lessons":[{"slug":"chunking-that-respects-structure","steps":[{"id":"context-and-retrieval/chunking-that-respects-structure/s01-01-intro"},{"id":"context-and-retrieval/chunking-that-respects-structure/s02-02-which-content-shape-breaks"},{"id":"context-and-retrieval/chunking-that-respects-structure/s03-03-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s04-04-predict-the-shred"},{"id":"context-and-retrieval/chunking-that-respects-structure/s05-05-fill-the-separator-list"},{"id":"context-and-retrieval/chunking-that-respects-structure/s06-06-fix-the-mid-sentence-cut"},{"id":"context-and-retrieval/chunking-that-respects-structure/s07-07-fix-the-missing-overlap"},{"id":"context-and-retrieval/chunking-that-respects-structure/s08-08-write-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s09-09-checkpoint"}]},{"slug":"embedding-that-fits-the-budget","steps":[{"id":"context-and-retrieval/embedding-that-fits-the-budget/s01-01-intro"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s02-02-which-model-fits"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s03-03-the-vector-space"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s04-04-predict-the-closest"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s05-05-fill-the-cosine-formula"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s06-06-fix-the-wrong-embedding-model"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s07-07-fix-the-missing-normalization"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s08-08-write-the-similarity-ranker"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s09-09-checkpoint"}]},{"slug":"retrieval-that-finds-the-right-thing","steps":[{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s01-01-intro"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s02-02-which-result-is-good"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s03-03-the-distance-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s04-04-predict-the-top-k"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s05-05-fill-the-cutoff"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s06-06-fix-the-no-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s07-07-fix-the-dup-results"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s08-08-write-the-ranker"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s09-09-checkpoint"}]},{"slug":"rag-vs-long-context-vs-fine-tune","steps":[{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s01-01-the-three-way-fork"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s02-02-rag-when-you-need-freshness"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s03-03-which-fork-for-this-scenario"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s04-04-long-context-when-the-corpus-fits"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s05-05-fine-tune-when-style-or-format-is-the-product"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s06-06-which-fork-dies-first"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s07-07-graph-vector-or-both"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s08-08-the-rubric"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s09-09-write-the-fork-decision"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s10-10-checkpoint"}]},{"slug":"retrieval-metrics-and-vector-db-shape","steps":[{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s01-01-the-eval-set"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s02-02-filter-or-search-first"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s03-03-predict-recall-precision"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s04-04-fill-the-reciprocal"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s05-05-fix-the-late-filter"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s06-06-write-recall-at-k"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s07-07-checkpoint"}]}]},{"slug":"production-tradeoffs","lessons":[{"slug":"prompt-caching-correctly","steps":[{"id":"production-tradeoffs/prompt-caching-correctly/s01-01-intro"},{"id":"production-tradeoffs/prompt-caching-correctly/s02-02-which-piece-caches"},{"id":"production-tradeoffs/prompt-caching-correctly/s03-03-the-cache-breakpoints"},{"id":"production-tradeoffs/prompt-caching-correctly/s04-04-predict-the-savings"},{"id":"production-tradeoffs/prompt-caching-correctly/s05-05-fill-the-cache-control"},{"id":"production-tradeoffs/prompt-caching-correctly/s06-06-fix-the-cache-buster"},{"id":"production-tradeoffs/prompt-caching-correctly/s07-07-fix-the-ttl-default"},{"id":"production-tradeoffs/prompt-caching-correctly/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/prompt-caching-correctly/s09-09-checkpoint"}]},{"slug":"read-the-token-bill","steps":[{"id":"production-tradeoffs/read-the-token-bill/s01-01-intro"},{"id":"production-tradeoffs/read-the-token-bill/s02-02-which-prompt-costs-more"},{"id":"production-tradeoffs/read-the-token-bill/s03-03-the-shape-of-a-bill"},{"id":"production-tradeoffs/read-the-token-bill/s04-04-predict-the-monthly-spend"},{"id":"production-tradeoffs/read-the-token-bill/s05-05-fill-the-cost-formula"},{"id":"production-tradeoffs/read-the-token-bill/s06-06-fix-the-pricing-ratio-error"},{"id":"production-tradeoffs/read-the-token-bill/s07-07-fix-the-units-error"},{"id":"production-tradeoffs/read-the-token-bill/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/read-the-token-bill/s09-09-checkpoint"}]},{"slug":"the-model-price-war","steps":[{"id":"production-tradeoffs/the-model-price-war/s01-01-the-price-curve"},{"id":"production-tradeoffs/the-model-price-war/s02-02-three-bets-and-how-they-aged"},{"id":"production-tradeoffs/the-model-price-war/s03-03-which-startup-survived"},{"id":"production-tradeoffs/the-model-price-war/s04-04-the-incumbent-trap"},{"id":"production-tradeoffs/the-model-price-war/s05-05-features-now-viable"},{"id":"production-tradeoffs/the-model-price-war/s06-06-pricing-for-a-war-you-cant-predict"},{"id":"production-tradeoffs/the-model-price-war/s07-07-write-the-cost-model"},{"id":"production-tradeoffs/the-model-price-war/s08-08-checkpoint"}]},{"slug":"batch-realtime-costs","steps":[{"id":"production-tradeoffs/batch-realtime-costs/s01-01-who-is-waiting"},{"id":"production-tradeoffs/batch-realtime-costs/s02-02-which-lane"},{"id":"production-tradeoffs/batch-realtime-costs/s03-03-predict-the-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s04-04-fill-the-miss-rate"},{"id":"production-tradeoffs/batch-realtime-costs/s05-05-fix-the-free-discount"},{"id":"production-tradeoffs/batch-realtime-costs/s06-06-write-monthly-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s07-07-checkpoint"}]}]},{"slug":"debugging-output","lessons":[{"slug":"read-the-trace-not-the-chat","steps":[{"id":"debugging-output/read-the-trace-not-the-chat/s01-01-intro"},{"id":"debugging-output/read-the-trace-not-the-chat/s02-02-where-to-look-first"},{"id":"debugging-output/read-the-trace-not-the-chat/s03-03-the-four-failure-classes"},{"id":"debugging-output/read-the-trace-not-the-chat/s04-04-predict-the-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s05-05-fill-the-find-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s06-06-fix-the-fix-the-symptom"},{"id":"debugging-output/read-the-trace-not-the-chat/s07-07-fix-the-blame-the-model"},{"id":"debugging-output/read-the-trace-not-the-chat/s08-08-write-the-trace-summary"},{"id":"debugging-output/read-the-trace-not-the-chat/s09-09-checkpoint"}]},{"slug":"the-four-breakage-classes","steps":[{"id":"debugging-output/the-four-breakage-classes/s01-01-intro"},{"id":"debugging-output/the-four-breakage-classes/s02-02-classify-the-refund-bug"},{"id":"debugging-output/the-four-breakage-classes/s03-03-the-trace-anatomy"},{"id":"debugging-output/the-four-breakage-classes/s04-04-predict-the-class"},{"id":"debugging-output/the-four-breakage-classes/s05-05-fill-the-class-name"},{"id":"debugging-output/the-four-breakage-classes/s06-06-fix-the-misclassification"},{"id":"debugging-output/the-four-breakage-classes/s07-07-fix-the-parse-mangle"},{"id":"debugging-output/the-four-breakage-classes/s08-08-write-the-classifier"},{"id":"debugging-output/the-four-breakage-classes/s09-09-checkpoint"}]},{"slug":"five-real-postmortems","steps":[{"id":"debugging-output/five-real-postmortems/s01-01-the-postmortem-template"},{"id":"debugging-output/five-real-postmortems/s02-02-air-canada-refund-lawsuit"},{"id":"debugging-output/five-real-postmortems/s03-03-dpd-and-nyc-mycity"},{"id":"debugging-output/five-real-postmortems/s04-04-classify-the-five"},{"id":"debugging-output/five-real-postmortems/s05-05-recruiter-json-mangle"},{"id":"debugging-output/five-real-postmortems/s06-06-glean-style-retrieval-miss"},{"id":"debugging-output/five-real-postmortems/s07-07-which-fix-catches-the-most"},{"id":"debugging-output/five-real-postmortems/s08-08-write-the-postmortem"},{"id":"debugging-output/five-real-postmortems/s09-09-checkpoint"}]},{"slug":"prediction-failure-triage","steps":[{"id":"debugging-output/prediction-failure-triage/s01-01-reproduce-classify-route"},{"id":"debugging-output/prediction-failure-triage/s02-02-first-move"},{"id":"debugging-output/prediction-failure-triage/s03-03-predict-the-route"},{"id":"debugging-output/prediction-failure-triage/s04-04-fill-the-owners"},{"id":"debugging-output/prediction-failure-triage/s05-05-fix-the-paraphrase-repro"},{"id":"debugging-output/prediction-failure-triage/s06-06-write-the-router"},{"id":"debugging-output/prediction-failure-triage/s07-07-checkpoint"}]}]},{"slug":"capstone","lessons":[{"slug":"pick-a-project-that-ships","steps":[{"id":"capstone/pick-a-project-that-ships/s01-01-the-builders-graveyard"},{"id":"capstone/pick-a-project-that-ships/s02-02-which-project-survives-contact-with-reality"},{"id":"capstone/pick-a-project-that-ships/s03-03-the-three-shapes-that-ship"},{"id":"capstone/pick-a-project-that-ships/s04-04-the-wedge-checklist"},{"id":"capstone/pick-a-project-that-ships/s05-05-spot-the-missing-wedge"},{"id":"capstone/pick-a-project-that-ships/s06-06-scope-down-the-capstone"},{"id":"capstone/pick-a-project-that-ships/s07-07-checkpoint"}]},{"slug":"build-a-cli-agent","steps":[{"id":"capstone/build-a-cli-agent/s01-01-intro"},{"id":"capstone/build-a-cli-agent/s02-02-which-piece-is-missing"},{"id":"capstone/build-a-cli-agent/s03-03-the-tool-loop"},{"id":"capstone/build-a-cli-agent/s04-04-predict-the-final-answer"},{"id":"capstone/build-a-cli-agent/s05-05-fill-the-stop-check"},{"id":"capstone/build-a-cli-agent/s06-06-fix-the-tool-dispatch"},{"id":"capstone/build-a-cli-agent/s07-07-fix-the-message-append"},{"id":"capstone/build-a-cli-agent/s08-08-the-logging-pattern"},{"id":"capstone/build-a-cli-agent/s09-09-write-the-tool-runner"},{"id":"capstone/build-a-cli-agent/s10-10-fix-the-loop-exit"},{"id":"capstone/build-a-cli-agent/s11-11-write-the-agent"},{"id":"capstone/build-a-cli-agent/s12-12-checkpoint"}]},{"slug":"wire-the-real-model","steps":[{"id":"capstone/wire-the-real-model/s01-01-intro"},{"id":"capstone/wire-the-real-model/s02-02-where-the-key-lives"},{"id":"capstone/wire-the-real-model/s03-03-the-sdk-response-shape"},{"id":"capstone/wire-the-real-model/s04-04-predict-the-block-type"},{"id":"capstone/wire-the-real-model/s05-05-fill-the-key-load"},{"id":"capstone/wire-the-real-model/s06-06-fix-the-hardcoded-key"},{"id":"capstone/wire-the-real-model/s07-07-fix-the-block-access"},{"id":"capstone/wire-the-real-model/s08-08-write-the-shape-adapter"},{"id":"capstone/wire-the-real-model/s09-09-checkpoint"}]},{"slug":"validate-tool-inputs","steps":[{"id":"capstone/validate-tool-inputs/s01-01-intro"},{"id":"capstone/validate-tool-inputs/s02-02-which-input-needs-validation"},{"id":"capstone/validate-tool-inputs/s03-03-the-validator-surface"},{"id":"capstone/validate-tool-inputs/s04-04-predict-the-error"},{"id":"capstone/validate-tool-inputs/s05-05-fill-the-validate-call"},{"id":"capstone/validate-tool-inputs/s06-06-fix-the-untyped-tool"},{"id":"capstone/validate-tool-inputs/s07-07-fix-the-error-leak"},{"id":"capstone/validate-tool-inputs/s08-08-write-the-validator"},{"id":"capstone/validate-tool-inputs/s09-09-checkpoint"}]},{"slug":"add-evals-and-traces","steps":[{"id":"capstone/add-evals-and-traces/s01-01-intro"},{"id":"capstone/add-evals-and-traces/s02-02-which-eval-fits"},{"id":"capstone/add-evals-and-traces/s03-03-the-trace-shape"},{"id":"capstone/add-evals-and-traces/s04-04-predict-the-pass-rate"},{"id":"capstone/add-evals-and-traces/s05-05-fill-the-trace-append"},{"id":"capstone/add-evals-and-traces/s06-06-fix-the-trace-mutation"},{"id":"capstone/add-evals-and-traces/s07-07-fix-the-tautological-eval"},{"id":"capstone/add-evals-and-traces/s08-08-write-the-eval-runner"},{"id":"capstone/add-evals-and-traces/s09-09-checkpoint"}]},{"slug":"wire-an-mcp-tool","steps":[{"id":"capstone/wire-an-mcp-tool/s01-01-intro"},{"id":"capstone/wire-an-mcp-tool/s02-02-when-to-use-mcp"},{"id":"capstone/wire-an-mcp-tool/s03-03-the-mcp-protocol"},{"id":"capstone/wire-an-mcp-tool/s04-04-predict-the-tool-list"},{"id":"capstone/wire-an-mcp-tool/s05-05-fill-the-tools-call"},{"id":"capstone/wire-an-mcp-tool/s06-06-fix-the-result-shape"},{"id":"capstone/wire-an-mcp-tool/s07-07-fix-the-iserror-flag"},{"id":"capstone/wire-an-mcp-tool/s08-08-write-the-mcp-bridge"},{"id":"capstone/wire-an-mcp-tool/s09-09-checkpoint"}]},{"slug":"portfolio-ml-system-option","steps":[{"id":"capstone/portfolio-ml-system-option/s01-01-the-other-capstone"},{"id":"capstone/portfolio-ml-system-option/s02-02-what-reviewers-open"},{"id":"capstone/portfolio-ml-system-option/s03-03-predict-the-gate"},{"id":"capstone/portfolio-ml-system-option/s04-04-fill-the-gate"},{"id":"capstone/portfolio-ml-system-option/s05-05-fix-the-train-accuracy-gate"},{"id":"capstone/portfolio-ml-system-option/s06-06-write-the-scope-check"},{"id":"capstone/portfolio-ml-system-option/s07-07-checkpoint"}]}]},{"slug":"agent-harnesses","lessons":[{"slug":"project-workspaces-not-product-names","steps":[{"id":"agent-harnesses/project-workspaces-not-product-names/s01-01-intro"}]},{"slug":"what-a-harness-is","steps":[{"id":"agent-harnesses/what-a-harness-is/s01-01-intro"},{"id":"agent-harnesses/what-a-harness-is/s02-02-which-is-a-harness"},{"id":"agent-harnesses/what-a-harness-is/s03-03-the-four-layers"},{"id":"agent-harnesses/what-a-harness-is/s04-04-predict-the-layer"},{"id":"agent-harnesses/what-a-harness-is/s05-05-fill-the-dispatch"},{"id":"agent-harnesses/what-a-harness-is/s06-06-fix-the-no-retry"},{"id":"agent-harnesses/what-a-harness-is/s07-07-fix-the-hardcoded-registry"},{"id":"agent-harnesses/what-a-harness-is/s08-08-write-mini-harness"},{"id":"agent-harnesses/what-a-harness-is/s09-09-checkpoint"}]},{"slug":"architecting-an-ai-native-workflow","steps":[{"id":"agent-harnesses/architecting-an-ai-native-workflow/s01-01-intro"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s02-02-which-workflow-fits"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s03-03-mapping-the-workflow"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s04-04-predict-the-readiness-score"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s05-05-fill-the-required-fields"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s06-06-fix-the-implicit-policy"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s07-07-fix-the-wrong-metric"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s08-08-write-the-workflow-validator"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s09-09-checkpoint"}]},{"slug":"five-industries-walked-through","steps":[{"id":"agent-harnesses/five-industries-walked-through/s01-01-the-case-study-method"},{"id":"agent-harnesses/five-industries-walked-through/s02-02-home-services"},{"id":"agent-harnesses/five-industries-walked-through/s03-03-which-task-belongs-to-the-agent"},{"id":"agent-harnesses/five-industries-walked-through/s04-04-insurance-brokerage"},{"id":"agent-harnesses/five-industries-walked-through/s05-05-recruiting-firm"},{"id":"agent-harnesses/five-industries-walked-through/s06-06-which-industry-wedges-fastest"},{"id":"agent-harnesses/five-industries-walked-through/s07-07-the-incumbent-trap"},{"id":"agent-harnesses/five-industries-walked-through/s08-08-write-the-process-debt-audit"},{"id":"agent-harnesses/five-industries-walked-through/s09-09-checkpoint"}]},{"slug":"incident-runbook-from-the-repo","steps":[{"id":"agent-harnesses/incident-runbook-from-the-repo/s01-01-intro"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s02-02-which-is-a-runbook"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s03-03-detect-decide-writeback"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s04-04-predict-the-decision"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s05-05-fill-the-decision-rule"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s06-06-fix-the-wiki-page"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s07-07-fix-the-unnamed-human"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s08-08-write-the-runbook"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s09-09-checkpoint"}]}]},{"slug":"ai-image-generation","lessons":[{"slug":"the-image-model-landscape","steps":[{"id":"ai-image-generation/the-image-model-landscape/s01-01-the-six-families"},{"id":"ai-image-generation/the-image-model-landscape/s02-02-flux-vs-midjourney"},{"id":"ai-image-generation/the-image-model-landscape/s03-03-pick-the-model"},{"id":"ai-image-generation/the-image-model-landscape/s04-04-nano-banana-and-the-batch-economics"},{"id":"ai-image-generation/the-image-model-landscape/s05-05-text-in-image"},{"id":"ai-image-generation/the-image-model-landscape/s06-06-which-model-for-text"},{"id":"ai-image-generation/the-image-model-landscape/s07-07-the-decision-tree"},{"id":"ai-image-generation/the-image-model-landscape/s08-08-write-pick-image-model"},{"id":"ai-image-generation/the-image-model-landscape/s09-09-checkpoint"}]},{"slug":"prompting-for-real-output","steps":[{"id":"ai-image-generation/prompting-for-real-output/s01-01-why-the-girl-with-red-hair-fails"},{"id":"ai-image-generation/prompting-for-real-output/s02-02-the-eight-knobs"},{"id":"ai-image-generation/prompting-for-real-output/s03-03-fix-the-vague-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s04-04-controlnet-and-reference"},{"id":"ai-image-generation/prompting-for-real-output/s05-05-which-knob-is-missing"},{"id":"ai-image-generation/prompting-for-real-output/s06-06-failure-modes"},{"id":"ai-image-generation/prompting-for-real-output/s07-07-write-score-image-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s08-08-checkpoint"}]},{"slug":"the-image-pipeline","steps":[{"id":"ai-image-generation/the-image-pipeline/s01-01-the-six-stages"},{"id":"ai-image-generation/the-image-pipeline/s02-02-batch-and-filter"},{"id":"ai-image-generation/the-image-pipeline/s03-03-which-stage-is-broken"},{"id":"ai-image-generation/the-image-pipeline/s04-04-format-and-platform"},{"id":"ai-image-generation/the-image-pipeline/s05-05-fill-the-pipeline"},{"id":"ai-image-generation/the-image-pipeline/s06-06-write-cost-for-batch"},{"id":"ai-image-generation/the-image-pipeline/s07-07-checkpoint"}]}]},{"slug":"ai-video-generation","lessons":[{"slug":"the-video-model-lineup","steps":[{"id":"ai-video-generation/the-video-model-lineup/s01-01-the-landscape"},{"id":"ai-video-generation/the-video-model-lineup/s02-02-sora-and-veo"},{"id":"ai-video-generation/the-video-model-lineup/s03-03-higgsfield-and-camera-control"},{"id":"ai-video-generation/the-video-model-lineup/s04-04-the-second-wave"},{"id":"ai-video-generation/the-video-model-lineup/s05-05-text-vs-image-vs-video"},{"id":"ai-video-generation/the-video-model-lineup/s06-06-which-lane"},{"id":"ai-video-generation/the-video-model-lineup/s07-07-pick-the-model"},{"id":"ai-video-generation/the-video-model-lineup/s08-08-write-pick-video-model"},{"id":"ai-video-generation/the-video-model-lineup/s09-09-checkpoint"}]},{"slug":"camera-control-and-motion","steps":[{"id":"ai-video-generation/camera-control-and-motion/s01-01-the-static-camera-problem"},{"id":"ai-video-generation/camera-control-and-motion/s02-02-shot-vocabulary"},{"id":"ai-video-generation/camera-control-and-motion/s03-03-which-shot-is-which"},{"id":"ai-video-generation/camera-control-and-motion/s04-04-camera-moves"},{"id":"ai-video-generation/camera-control-and-motion/s05-05-the-i2v-keyframe-pattern"},{"id":"ai-video-generation/camera-control-and-motion/s06-06-spot-the-generic-prompt"},{"id":"ai-video-generation/camera-control-and-motion/s07-07-write-audit-shot-list"},{"id":"ai-video-generation/camera-control-and-motion/s08-08-checkpoint"}]},{"slug":"the-cost-math","steps":[{"id":"ai-video-generation/the-cost-math/s01-01-the-unit-economics"},{"id":"ai-video-generation/the-cost-math/s02-02-the-retake-rate-reality"},{"id":"ai-video-generation/the-cost-math/s03-03-which-tier-for-this-budget"},{"id":"ai-video-generation/the-cost-math/s04-04-the-60-second-promo"},{"id":"ai-video-generation/the-cost-math/s05-05-write-cost-per-minute"},{"id":"ai-video-generation/the-cost-math/s06-06-checkpoint"}]}]},{"slug":"programmatic-design","lessons":[{"slug":"why-programmatic","steps":[{"id":"programmatic-design/why-programmatic/s01-01-intro"},{"id":"programmatic-design/why-programmatic/s02-02-the-three-jobs"},{"id":"programmatic-design/why-programmatic/s03-03-which-task-is-code"},{"id":"programmatic-design/why-programmatic/s04-04-parametric-vs-one-shot"},{"id":"programmatic-design/why-programmatic/s05-05-the-classifier"},{"id":"programmatic-design/why-programmatic/s06-06-write-classify-video-task"},{"id":"programmatic-design/why-programmatic/s07-07-checkpoint"}]},{"slug":"hyperframes-and-remotion","steps":[{"id":"programmatic-design/hyperframes-and-remotion/s01-01-the-two-tools"},{"id":"programmatic-design/hyperframes-and-remotion/s02-02-hyperframes-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s03-03-hyperframes-cli"},{"id":"programmatic-design/hyperframes-and-remotion/s04-04-remotion-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s05-05-the-decision-tree"},{"id":"programmatic-design/hyperframes-and-remotion/s06-06-pick-the-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s07-07-claude-design-html"},{"id":"programmatic-design/hyperframes-and-remotion/s08-08-write-pick-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s09-09-checkpoint"}]},{"slug":"the-ai-native-design-pipeline","steps":[{"id":"programmatic-design/the-ai-native-design-pipeline/s01-01-the-seven-steps"},{"id":"programmatic-design/the-ai-native-design-pipeline/s02-02-where-ai-fits-each-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s03-03-which-model-which-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s04-04-website-to-hyperframes"},{"id":"programmatic-design/the-ai-native-design-pipeline/s05-05-the-cost-model"},{"id":"programmatic-design/the-ai-native-design-pipeline/s06-06-score-the-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s07-07-write-score-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s08-08-checkpoint"}]}]},{"slug":"harness-engineering","lessons":[{"slug":"the-harness-engineering-mindset","steps":[{"id":"harness-engineering/the-harness-engineering-mindset/s01-01-the-model-is-only-one-input"},{"id":"harness-engineering/the-harness-engineering-mindset/s02-02-the-six-pieces-of-a-harness"},{"id":"harness-engineering/the-harness-engineering-mindset/s03-03-which-piece-is-missing"},{"id":"harness-engineering/the-harness-engineering-mindset/s04-04-the-skill-issue-reframe"},{"id":"harness-engineering/the-harness-engineering-mindset/s05-05-pick-the-fix"},{"id":"harness-engineering/the-harness-engineering-mindset/s06-06-the-harness-gap"},{"id":"harness-engineering/the-harness-engineering-mindset/s07-07-write-the-harness-inventory"},{"id":"harness-engineering/the-harness-engineering-mindset/s08-08-checkpoint"}]},{"slug":"the-ratchet","steps":[{"id":"harness-engineering/the-ratchet/s01-01-failures-are-signals-not-flukes"},{"id":"harness-engineering/the-ratchet/s02-02-every-line-traces-to-a-failure"},{"id":"harness-engineering/the-ratchet/s03-03-which-rule-earned-its-line"},{"id":"harness-engineering/the-ratchet/s04-04-working-backwards-from-behavior"},{"id":"harness-engineering/the-ratchet/s05-05-the-ratchet-log"},{"id":"harness-engineering/the-ratchet/s06-06-spot-the-cruft"},{"id":"harness-engineering/the-ratchet/s07-07-write-the-ratchet-update"},{"id":"harness-engineering/the-ratchet/s08-08-audit-the-ratchet"},{"id":"harness-engineering/the-ratchet/s09-09-checkpoint"}]},{"slug":"context-engineering","steps":[{"id":"harness-engineering/context-engineering/s01-01-the-context-rot"},{"id":"harness-engineering/context-engineering/s02-02-compaction"},{"id":"harness-engineering/context-engineering/s03-03-tool-call-offloading"},{"id":"harness-engineering/context-engineering/s04-04-progressive-disclosure"},{"id":"harness-engineering/context-engineering/s05-05-which-technique-fits"},{"id":"harness-engineering/context-engineering/s06-06-the-memory-hierarchy"},{"id":"harness-engineering/context-engineering/s07-07-classify-the-data"},{"id":"harness-engineering/context-engineering/s08-08-write-the-context-budget"},{"id":"harness-engineering/context-engineering/s09-09-checkpoint"}]},{"slug":"long-horizon-execution","steps":[{"id":"harness-engineering/long-horizon-execution/s01-01-the-40-step-problem"},{"id":"harness-engineering/long-horizon-execution/s02-02-loops-intercepting-early-stops"},{"id":"harness-engineering/long-horizon-execution/s03-03-planning-as-a-separate-step"},{"id":"harness-engineering/long-horizon-execution/s04-04-splits-generation-vs-evaluation"},{"id":"harness-engineering/long-horizon-execution/s05-05-which-pattern-fixes-the-failure"},{"id":"harness-engineering/long-horizon-execution/s06-06-hooks-as-the-enforcement-layer"},{"id":"harness-engineering/long-horizon-execution/s07-07-the-type-check-backpressure-pattern"},{"id":"harness-engineering/long-horizon-execution/s08-08-write-the-hook-router"},{"id":"harness-engineering/long-horizon-execution/s09-09-write-the-completion-guard"},{"id":"harness-engineering/long-horizon-execution/s10-10-checkpoint"}]},{"slug":"the-haas-shift","steps":[{"id":"harness-engineering/the-haas-shift/s01-01-llm-apis-to-harness-apis"},{"id":"harness-engineering/the-haas-shift/s02-02-the-convergence"},{"id":"harness-engineering/the-haas-shift/s03-03-which-haas-fits"},{"id":"harness-engineering/the-haas-shift/s04-04-harnesses-dont-shrink-they-move"},{"id":"harness-engineering/the-haas-shift/s05-05-the-overfitting-feedback-loop"},{"id":"harness-engineering/the-haas-shift/s06-06-mcp-security-as-prompt-injection"},{"id":"harness-engineering/the-haas-shift/s07-07-which-component-is-debt"},{"id":"harness-engineering/the-haas-shift/s08-08-write-the-build-vs-buy"},{"id":"harness-engineering/the-haas-shift/s09-09-checkpoint"}]},{"slug":"pipeline-boundaries","steps":[{"id":"harness-engineering/pipeline-boundaries/s01-01-the-boundary-is-an-artifact"},{"id":"harness-engineering/pipeline-boundaries/s02-02-spot-the-risky-handoff"},{"id":"harness-engineering/pipeline-boundaries/s03-03-predict-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s04-04-fill-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s05-05-fix-the-overtrust"},{"id":"harness-engineering/pipeline-boundaries/s06-06-write-the-advance-check"},{"id":"harness-engineering/pipeline-boundaries/s07-07-checkpoint"}]}]},{"slug":"intro-to-terminal","lessons":[{"slug":"what-the-terminal-is","steps":[{"id":"intro-to-terminal/what-the-terminal-is/s01-01-not-a-cockpit"},{"id":"intro-to-terminal/what-the-terminal-is/s02-02-what-it-actually-is"},{"id":"intro-to-terminal/what-the-terminal-is/s03-03-open-it"},{"id":"intro-to-terminal/what-the-terminal-is/s04-04-which-app"},{"id":"intro-to-terminal/what-the-terminal-is/s05-05-the-prompt"},{"id":"intro-to-terminal/what-the-terminal-is/s06-06-what-a-command-is"},{"id":"intro-to-terminal/what-the-terminal-is/s07-07-checkpoint"}]},{"slug":"moving-around","steps":[{"id":"intro-to-terminal/moving-around/s01-01-the-filesystem-is-a-tree"},{"id":"intro-to-terminal/moving-around/s02-02-pwd-and-ls"},{"id":"intro-to-terminal/moving-around/s03-03-where-am-i"},{"id":"intro-to-terminal/moving-around/s04-04-read-this-listing"},{"id":"intro-to-terminal/moving-around/s05-05-cd"},{"id":"intro-to-terminal/moving-around/s06-06-follow-the-moves"},{"id":"intro-to-terminal/moving-around/s07-07-pick-the-command"},{"id":"intro-to-terminal/moving-around/s08-08-checkpoint"}]},{"slug":"making-things","steps":[{"id":"intro-to-terminal/making-things/s01-01-mkdir-and-touch"},{"id":"intro-to-terminal/making-things/s02-02-folder-or-file"},{"id":"intro-to-terminal/making-things/s03-03-cat-and-echo"},{"id":"intro-to-terminal/making-things/s04-04-reading-a-file"},{"id":"intro-to-terminal/making-things/s05-05-rm-carefully"},{"id":"intro-to-terminal/making-things/s06-06-the-no-undo-rule"},{"id":"intro-to-terminal/making-things/s07-07-string-it-together"},{"id":"intro-to-terminal/making-things/s08-08-checkpoint"}]}]},{"slug":"intro-to-claude-cli","lessons":[{"slug":"what-the-claude-cli-is","steps":[{"id":"intro-to-claude-cli/what-the-claude-cli-is/s01-01-the-glass-is-gone"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s02-02-chat-vs-cli"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s03-03-which-tool"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s04-04-the-agent-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s05-05-read-the-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s06-06-permission-and-limits"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-claude-cli/install-and-sign-in/s01-01-two-ways-to-install"},{"id":"intro-to-claude-cli/install-and-sign-in/s02-02-run-the-installer"},{"id":"intro-to-claude-cli/install-and-sign-in/s03-03-install-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s04-04-sign-in"},{"id":"intro-to-claude-cli/install-and-sign-in/s05-05-what-sign-in-did"},{"id":"intro-to-claude-cli/install-and-sign-in/s06-06-auth-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s07-07-account-not-api-key"},{"id":"intro-to-claude-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-claude-cli/your-first-session/s01-01-start-in-a-folder"},{"id":"intro-to-claude-cli/your-first-session/s02-02-your-first-instruction"},{"id":"intro-to-claude-cli/your-first-session/s03-03-watch-the-loop"},{"id":"intro-to-claude-cli/your-first-session/s04-04-read-what-it-did"},{"id":"intro-to-claude-cli/your-first-session/s05-05-one-shot-or-interactive"},{"id":"intro-to-claude-cli/your-first-session/s06-06-chat-or-cli-decision"},{"id":"intro-to-claude-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-claude-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"intro-to-codex-cli","lessons":[{"slug":"another-tool-same-idea","steps":[{"id":"intro-to-codex-cli/another-tool-same-idea/s01-01-why-a-second-tool"},{"id":"intro-to-codex-cli/another-tool-same-idea/s02-02-what-carries-over"},{"id":"intro-to-codex-cli/another-tool-same-idea/s03-03-what-carries-over-check"},{"id":"intro-to-codex-cli/another-tool-same-idea/s04-04-what-is-different"},{"id":"intro-to-codex-cli/another-tool-same-idea/s05-05-spot-the-difference"},{"id":"intro-to-codex-cli/another-tool-same-idea/s06-06-when-to-reach-for-which"},{"id":"intro-to-codex-cli/another-tool-same-idea/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-codex-cli/install-and-sign-in/s01-01-the-install-paths"},{"id":"intro-to-codex-cli/install-and-sign-in/s02-02-installing-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s03-03-verify-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s04-04-install-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s05-05-sign-in"},{"id":"intro-to-codex-cli/install-and-sign-in/s06-06-account-vs-api-key"},{"id":"intro-to-codex-cli/install-and-sign-in/s07-07-auth-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-codex-cli/your-first-session/s01-01-start-and-instruct"},{"id":"intro-to-codex-cli/your-first-session/s02-02-the-loop-again"},{"id":"intro-to-codex-cli/your-first-session/s03-03-read-what-it-did"},{"id":"intro-to-codex-cli/your-first-session/s04-04-usage-patterns"},{"id":"intro-to-codex-cli/your-first-session/s05-05-usage-scenario"},{"id":"intro-to-codex-cli/your-first-session/s06-06-running-both"},{"id":"intro-to-codex-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-codex-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"team-skills","lessons":[{"slug":"what-a-skill-is","steps":[{"id":"team-skills/what-a-skill-is/s01-01-the-re-explaining-problem"},{"id":"team-skills/what-a-skill-is/s02-02-a-skill-defined"},{"id":"team-skills/what-a-skill-is/s03-03-a-real-skill"},{"id":"team-skills/what-a-skill-is/s04-04-what-a-skill-is-check"},{"id":"team-skills/what-a-skill-is/s05-05-skill-vs-prompt-vs-agent"},{"id":"team-skills/what-a-skill-is/s06-06-skill-or-agent"},{"id":"team-skills/what-a-skill-is/s07-07-checkpoint"}]},{"slug":"skills-as-a-team-thing","steps":[{"id":"team-skills/skills-as-a-team-thing/s01-01-the-team-problem"},{"id":"team-skills/skills-as-a-team-thing/s02-02-org-wide-skills"},{"id":"team-skills/skills-as-a-team-thing/s03-03-sharing-and-the-directory"},{"id":"team-skills/skills-as-a-team-thing/s04-04-deployment-check"},{"id":"team-skills/skills-as-a-team-thing/s05-05-collaborating-through-skills"},{"id":"team-skills/skills-as-a-team-thing/s06-06-sharing-scenario"},{"id":"team-skills/skills-as-a-team-thing/s07-07-checkpoint"}]},{"slug":"role-playbooks","steps":[{"id":"team-skills/role-playbooks/s01-01-hr-skills"},{"id":"team-skills/role-playbooks/s02-02-legal-skills"},{"id":"team-skills/role-playbooks/s03-03-operations-skills"},{"id":"team-skills/role-playbooks/s04-04-match-the-skill"},{"id":"team-skills/role-playbooks/s05-05-skill-or-one-off"},{"id":"team-skills/role-playbooks/s06-06-skill-or-one-off-check"},{"id":"team-skills/role-playbooks/s07-07-checkpoint"}]},{"slug":"governance-and-trust","steps":[{"id":"team-skills/governance-and-trust/s01-01-a-skill-can-do-real-things"},{"id":"team-skills/governance-and-trust/s02-02-treat-it-like-software"},{"id":"team-skills/governance-and-trust/s03-03-evaluate-before-you-deploy"},{"id":"team-skills/governance-and-trust/s04-04-access-control"},{"id":"team-skills/governance-and-trust/s05-05-governance-scenario"},{"id":"team-skills/governance-and-trust/s06-06-a-rollout-checklist"},{"id":"team-skills/governance-and-trust/s07-07-checkpoint"}]}]},{"slug":"dataframes-numpy-pandas","lessons":[{"slug":"ml-package-map","steps":[{"id":"dataframes-numpy-pandas/ml-package-map/s01-01-intro"},{"id":"dataframes-numpy-pandas/ml-package-map/s02-02-match-the-tool"},{"id":"dataframes-numpy-pandas/ml-package-map/s03-03-predict-sys-modules"},{"id":"dataframes-numpy-pandas/ml-package-map/s04-04-fill-the-registry-key"},{"id":"dataframes-numpy-pandas/ml-package-map/s05-05-fix-the-swapped-map"},{"id":"dataframes-numpy-pandas/ml-package-map/s06-06-write-find-unused"},{"id":"dataframes-numpy-pandas/ml-package-map/s07-07-checkpoint"}]},{"slug":"array-and-series-shapes","steps":[{"id":"dataframes-numpy-pandas/array-and-series-shapes/s01-01-intro"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s07-07-checkpoint"}]},{"slug":"dataframe-selection-and-cleaning","steps":[{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s01-01-intro"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s07-07-checkpoint"}]},{"slug":"missing-values-and-types","steps":[{"id":"dataframes-numpy-pandas/missing-values-and-types/s01-01-intro"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s07-07-checkpoint"}]},{"slug":"groupby-joins-and-features","steps":[{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s01-01-intro"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataframe","steps":[{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s01-01-intro"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s07-07-checkpoint"}]}]},{"slug":"sql-for-ml-datasets","lessons":[{"slug":"select-filter-aggregate","steps":[{"id":"sql-for-ml-datasets/select-filter-aggregate/s01-01-intro"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s07-07-checkpoint"}]},{"slug":"joins-and-label-windows","steps":[{"id":"sql-for-ml-datasets/joins-and-label-windows/s01-01-intro"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s07-07-checkpoint"}]},{"slug":"sql-quality-checks","steps":[{"id":"sql-for-ml-datasets/sql-quality-checks/s01-01-intro"},{"id":"sql-for-ml-datasets/sql-quality-checks/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/sql-quality-checks/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/sql-quality-checks/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/sql-quality-checks/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/sql-quality-checks/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/sql-quality-checks/s07-07-checkpoint"}]},{"slug":"mission-sql-feature-query-lab","steps":[{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s01-01-intro"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s07-07-checkpoint"}]}]},{"slug":"dataset-pipelines","lessons":[{"slug":"csv-jsonl-parquet-tradeoffs","steps":[{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s01-01-intro"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s02-02-choose-the-risk"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s03-03-predict-the-checks"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s04-04-fill-the-return"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s06-06-write-count-ready"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s07-07-checkpoint"}]},{"slug":"pagination-retries-checkpoints","steps":[{"id":"dataset-pipelines/pagination-retries-checkpoints/s01-01-intro"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s04-04-fill-the-return"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s06-06-write-count-ready"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s07-07-checkpoint"}]},{"slug":"schema-drift-and-contracts","steps":[{"id":"dataset-pipelines/schema-drift-and-contracts/s01-01-intro"},{"id":"dataset-pipelines/schema-drift-and-contracts/s02-02-choose-the-risk"},{"id":"dataset-pipelines/schema-drift-and-contracts/s03-03-predict-the-checks"},{"id":"dataset-pipelines/schema-drift-and-contracts/s04-04-fill-the-return"},{"id":"dataset-pipelines/schema-drift-and-contracts/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/schema-drift-and-contracts/s06-06-write-count-ready"},{"id":"dataset-pipelines/schema-drift-and-contracts/s07-07-checkpoint"}]},{"slug":"pipeline-orchestration-shape","steps":[{"id":"dataset-pipelines/pipeline-orchestration-shape/s01-01-intro"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s04-04-fill-the-return"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s06-06-write-count-ready"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataset-pipeline","steps":[{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s01-01-intro"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s02-02-choose-the-risk"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s03-03-predict-the-checks"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s04-04-fill-the-return"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s06-06-write-count-ready"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s07-07-checkpoint"}]}]},{"slug":"ml-math-and-stats","lessons":[{"slug":"vectors-matrices-dot-products","steps":[{"id":"ml-math-and-stats/vectors-matrices-dot-products/s01-01-intro"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s04-04-fill-the-return"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s06-06-write-count-ready"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s07-07-checkpoint"}]},{"slug":"probability-and-baselines","steps":[{"id":"ml-math-and-stats/probability-and-baselines/s01-01-intro"},{"id":"ml-math-and-stats/probability-and-baselines/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/probability-and-baselines/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/probability-and-baselines/s04-04-fill-the-return"},{"id":"ml-math-and-stats/probability-and-baselines/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/probability-and-baselines/s06-06-write-count-ready"},{"id":"ml-math-and-stats/probability-and-baselines/s07-07-checkpoint"}]},{"slug":"distributions-sampling-variance","steps":[{"id":"ml-math-and-stats/distributions-sampling-variance/s01-01-intro"},{"id":"ml-math-and-stats/distributions-sampling-variance/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/distributions-sampling-variance/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/distributions-sampling-variance/s04-04-fill-the-return"},{"id":"ml-math-and-stats/distributions-sampling-variance/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/distributions-sampling-variance/s06-06-write-variance"},{"id":"ml-math-and-stats/distributions-sampling-variance/s07-07-checkpoint"}]},{"slug":"bias-variance-and-leakage","steps":[{"id":"ml-math-and-stats/bias-variance-and-leakage/s01-01-intro"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s04-04-fill-the-return"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s06-06-write-diagnose"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s07-07-checkpoint"}]},{"slug":"mission-statistical-sanity-check","steps":[{"id":"ml-math-and-stats/mission-statistical-sanity-check/s01-01-intro"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s04-04-fill-the-return"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s06-06-write-count-ready"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s07-07-checkpoint"}]}]},{"slug":"supervised-learning-workflows","lessons":[{"slug":"labels-features-splits","steps":[{"id":"supervised-learning-workflows/labels-features-splits/s01-01-intro"},{"id":"supervised-learning-workflows/labels-features-splits/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/labels-features-splits/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/labels-features-splits/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/labels-features-splits/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/labels-features-splits/s06-06-write-split-audit"},{"id":"supervised-learning-workflows/labels-features-splits/s07-07-checkpoint"}]},{"slug":"baselines-before-fancy-models","steps":[{"id":"supervised-learning-workflows/baselines-before-fancy-models/s01-01-intro"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s06-06-write-baseline-comparison"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s07-07-checkpoint"}]},{"slug":"training-and-prediction-loop","steps":[{"id":"supervised-learning-workflows/training-and-prediction-loop/s01-01-intro"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s06-06-write-train"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s07-07-checkpoint"}]},{"slug":"overfitting-and-regularization","steps":[{"id":"supervised-learning-workflows/overfitting-and-regularization/s01-01-intro"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s06-06-write-overfit-audit"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s07-07-checkpoint"}]},{"slug":"mission-baseline-model-showdown","steps":[{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s01-01-intro"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s06-06-write-classifier-brief"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s07-07-checkpoint"}]}]},{"slug":"unsupervised-learning-and-embeddings","lessons":[{"slug":"clustering-without-labels","steps":[{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s06-06-write-cluster-summary"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s07-07-checkpoint"}]},{"slug":"dimensionality-and-distance","steps":[{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s06-06-write-euclidean"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s07-07-checkpoint"}]},{"slug":"embeddings-as-features","steps":[{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s06-06-write-retrieval-receipt"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s07-07-checkpoint"}]},{"slug":"mission-tiny-recommender","steps":[{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s06-06-write-recommendation-card"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s07-07-checkpoint"}]}]},{"slug":"metrics-and-error-analysis","lessons":[{"slug":"confusion-matrix","steps":[{"id":"metrics-and-error-analysis/confusion-matrix/s01-01-intro"},{"id":"metrics-and-error-analysis/confusion-matrix/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/confusion-matrix/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/confusion-matrix/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/confusion-matrix/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/confusion-matrix/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/confusion-matrix/s07-07-checkpoint"}]},{"slug":"precision-recall-thresholds","steps":[{"id":"metrics-and-error-analysis/precision-recall-thresholds/s01-01-intro"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s07-07-checkpoint"}]},{"slug":"slice-based-error-analysis","steps":[{"id":"metrics-and-error-analysis/slice-based-error-analysis/s01-01-intro"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s07-07-checkpoint"}]},{"slug":"regression-metrics-and-residuals","steps":[{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s01-01-intro"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s07-07-checkpoint"}]},{"slug":"mission-confusion-matrix-triage","steps":[{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s01-01-intro"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s07-07-checkpoint"}]}]},{"slug":"pytorch-tensors-and-autograd","lessons":[{"slug":"tensors-and-shapes","steps":[{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s07-07-checkpoint"}]},{"slug":"broadcasting-and-vectorization","steps":[{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s07-07-checkpoint"}]},{"slug":"autograd-mental-model","steps":[{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s07-07-checkpoint"}]},{"slug":"mission-tensor-playground","steps":[{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s07-07-checkpoint"}]}]},{"slug":"training-loops-and-optimizers","lessons":[{"slug":"loss-forward-backward-step","steps":[{"id":"training-loops-and-optimizers/loss-forward-backward-step/s01-01-intro"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s07-07-checkpoint"}]},{"slug":"gradient-descent-by-hand","steps":[{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s01-01-intro"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s07-07-checkpoint"}]},{"slug":"optimizers-and-schedulers","steps":[{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s01-01-intro"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s07-07-checkpoint"}]},{"slug":"checkpoints-and-reproducibility","steps":[{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s01-01-intro"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s07-07-checkpoint"}]},{"slug":"mission-overfit-then-recover","steps":[{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s01-01-intro"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s07-07-checkpoint"}]}]},{"slug":"deep-learning-architectures","lessons":[{"slug":"cnns-and-local-patterns","steps":[{"id":"deep-learning-architectures/cnns-and-local-patterns/s01-01-intro"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s04-04-fill-the-return"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s06-06-write-count-ready"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s07-07-checkpoint"}]},{"slug":"tokenizers-and-context-budget","steps":[{"id":"deep-learning-architectures/tokenizers-and-context-budget/s01-01-intro"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s04-04-fill-the-return"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s06-06-write-count-ready"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s07-07-checkpoint"}]},{"slug":"attention-and-transformer-blocks","steps":[{"id":"deep-learning-architectures/attention-and-transformer-blocks/s01-01-intro"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s04-04-fill-the-return"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s06-06-write-count-ready"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s07-07-checkpoint"}]},{"slug":"decoding-kv-cache-quantization","steps":[{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s01-01-intro"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s04-04-fill-the-return"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s06-06-write-count-ready"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s07-07-checkpoint"}]},{"slug":"mission-architecture-tradeoff-note","steps":[{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s01-01-intro"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s04-04-fill-the-return"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s06-06-write-count-ready"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s07-07-checkpoint"}]}]},{"slug":"feature-experiments-registries","lessons":[{"slug":"feature-pipeline-contracts","steps":[{"id":"feature-experiments-registries/feature-pipeline-contracts/s01-01-intro"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s04-04-fill-the-return"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s06-06-write-count-ready"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s07-07-checkpoint"}]},{"slug":"train-inference-skew","steps":[{"id":"feature-experiments-registries/train-inference-skew/s01-01-intro"},{"id":"feature-experiments-registries/train-inference-skew/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/train-inference-skew/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/train-inference-skew/s04-04-fill-the-return"},{"id":"feature-experiments-registries/train-inference-skew/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/train-inference-skew/s06-06-write-count-ready"},{"id":"feature-experiments-registries/train-inference-skew/s07-07-checkpoint"}]},{"slug":"experiment-tracker-lite","steps":[{"id":"feature-experiments-registries/experiment-tracker-lite/s01-01-intro"},{"id":"feature-experiments-registries/experiment-tracker-lite/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/experiment-tracker-lite/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/experiment-tracker-lite/s04-04-fill-the-return"},{"id":"feature-experiments-registries/experiment-tracker-lite/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/experiment-tracker-lite/s06-06-write-count-ready"},{"id":"feature-experiments-registries/experiment-tracker-lite/s07-07-checkpoint"}]},{"slug":"model-registry-shape","steps":[{"id":"feature-experiments-registries/model-registry-shape/s01-01-intro"},{"id":"feature-experiments-registries/model-registry-shape/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/model-registry-shape/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/model-registry-shape/s04-04-fill-the-return"},{"id":"feature-experiments-registries/model-registry-shape/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/model-registry-shape/s06-06-write-count-ready"},{"id":"feature-experiments-registries/model-registry-shape/s07-07-checkpoint"}]},{"slug":"mission-feature-pipeline-and-tracker","steps":[{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s01-01-intro"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s04-04-fill-the-return"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s06-06-write-count-ready"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s07-07-checkpoint"}]}]},{"slug":"model-serving-and-mlops","lessons":[{"slug":"fastapi-inference-shape","steps":[{"id":"model-serving-and-mlops/fastapi-inference-shape/s01-01-intro"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s07-07-checkpoint"}]},{"slug":"batch-vs-realtime-serving","steps":[{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s01-01-intro"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s07-07-checkpoint"}]},{"slug":"docker-runtime-and-config","steps":[{"id":"model-serving-and-mlops/docker-runtime-and-config/s01-01-intro"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s07-07-checkpoint"}]},{"slug":"github-actions-validation-gates","steps":[{"id":"model-serving-and-mlops/github-actions-validation-gates/s01-01-intro"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s06-06-write-failing-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s07-07-checkpoint"}]},{"slug":"continuous-training-and-rollout","steps":[{"id":"model-serving-and-mlops/continuous-training-and-rollout/s01-01-intro"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s07-07-checkpoint"}]},{"slug":"mission-fastapi-model-server","steps":[{"id":"model-serving-and-mlops/mission-fastapi-model-server/s01-01-intro"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s07-07-checkpoint"}]}]},{"slug":"monitoring-cloud-portfolio","lessons":[{"slug":"structured-logs-and-alerts","steps":[{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s01-01-intro"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s07-07-checkpoint"}]},{"slug":"data-and-concept-drift","steps":[{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s01-01-intro"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s07-07-checkpoint"}]},{"slug":"retraining-triggers","steps":[{"id":"monitoring-cloud-portfolio/retraining-triggers/s01-01-intro"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s07-07-checkpoint"}]},{"slug":"cloud-gpu-kubernetes-cost","steps":[{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s01-01-intro"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s07-07-checkpoint"}]},{"slug":"portfolio-architecture-docs","steps":[{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s01-01-intro"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s07-07-checkpoint"}]},{"slug":"mission-final-portfolio-ml-system","steps":[{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s01-01-intro"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s06-06-write-decide"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s07-07-checkpoint"}]}]},{"slug":"graphic-design-studio","lessons":[{"slug":"prompting-like-an-art-director","steps":[{"id":"graphic-design-studio/prompting-like-an-art-director/s01-01-the-prompt-is-a-creative-brief"},{"id":"graphic-design-studio/prompting-like-an-art-director/s02-02-style-language"},{"id":"graphic-design-studio/prompting-like-an-art-director/s03-03-which-tool-for-the-headline"},{"id":"graphic-design-studio/prompting-like-an-art-director/s04-04-predict-the-scaffold"},{"id":"graphic-design-studio/prompting-like-an-art-director/s05-05-negative-space-and-the-iteration-loop"},{"id":"graphic-design-studio/prompting-like-an-art-director/s06-06-fix-the-mood-word-prompt"},{"id":"graphic-design-studio/prompting-like-an-art-director/s07-07-write-prompt-linter"},{"id":"graphic-design-studio/prompting-like-an-art-director/s08-08-checkpoint"}]},{"slug":"the-design-kit","steps":[{"id":"graphic-design-studio/the-design-kit/s01-01-the-brand-bible-as-data"},{"id":"graphic-design-studio/the-design-kit/s02-02-fill-the-palette-line"},{"id":"graphic-design-studio/the-design-kit/s03-03-banned-looks"},{"id":"graphic-design-studio/the-design-kit/s04-04-predict-the-negatives"},{"id":"graphic-design-studio/the-design-kit/s05-05-which-field-failed"},{"id":"graphic-design-studio/the-design-kit/s06-06-fix-the-crossed-kit"},{"id":"graphic-design-studio/the-design-kit/s07-07-write-contrast-check"},{"id":"graphic-design-studio/the-design-kit/s08-08-tokens-for-the-interface"},{"id":"graphic-design-studio/the-design-kit/s09-09-write-token-audit"},{"id":"graphic-design-studio/the-design-kit/s10-10-checkpoint"}]},{"slug":"fifty-assets-one-brand","steps":[{"id":"graphic-design-studio/fifty-assets-one-brand/s01-01-the-consistency-problem"},{"id":"graphic-design-studio/fifty-assets-one-brand/s02-02-seeds-the-same-dice-twice"},{"id":"graphic-design-studio/fifty-assets-one-brand/s03-03-why-does-the-mascot-drift"},{"id":"graphic-design-studio/fifty-assets-one-brand/s04-04-predict-the-batch-plan"},{"id":"graphic-design-studio/fifty-assets-one-brand/s05-05-the-qa-filter"},{"id":"graphic-design-studio/fifty-assets-one-brand/s06-06-defend-the-pick"},{"id":"graphic-design-studio/fifty-assets-one-brand/s07-07-fix-the-qa-threshold"},{"id":"graphic-design-studio/fifty-assets-one-brand/s08-08-write-palette-distance"},{"id":"graphic-design-studio/fifty-assets-one-brand/s09-09-checkpoint"}]}]},{"slug":"campaign-studio","lessons":[{"slug":"voice-pack-as-data","steps":[{"id":"campaign-studio/voice-pack-as-data/s01-01-the-campaign-and-the-gate"},{"id":"campaign-studio/voice-pack-as-data/s02-02-lint-the-draft"},{"id":"campaign-studio/voice-pack-as-data/s03-03-predict-the-violations"},{"id":"campaign-studio/voice-pack-as-data/s04-04-testable-or-mood"},{"id":"campaign-studio/voice-pack-as-data/s05-05-fill-the-linter"},{"id":"campaign-studio/voice-pack-as-data/s06-06-variants-as-records"},{"id":"campaign-studio/voice-pack-as-data/s07-07-write-lint-copy"},{"id":"campaign-studio/voice-pack-as-data/s08-08-checkpoint"}]},{"slug":"batch-plan-and-calendar","steps":[{"id":"campaign-studio/batch-plan-and-calendar/s01-01-slots-before-prompts"},{"id":"campaign-studio/batch-plan-and-calendar/s02-02-which-field-is-missing"},{"id":"campaign-studio/batch-plan-and-calendar/s03-03-write-batch-plan"},{"id":"campaign-studio/batch-plan-and-calendar/s04-04-the-calendar-is-data"},{"id":"campaign-studio/batch-plan-and-calendar/s05-05-predict-the-queue"},{"id":"campaign-studio/batch-plan-and-calendar/s06-06-fix-the-gate"},{"id":"campaign-studio/batch-plan-and-calendar/s07-07-checkpoint"}]},{"slug":"the-numbers-pass","steps":[{"id":"campaign-studio/the-numbers-pass/s01-01-a-report-is-a-draft-over-data"},{"id":"campaign-studio/the-numbers-pass/s02-02-predict-the-verdict"},{"id":"campaign-studio/the-numbers-pass/s03-03-write-verify-numbers"},{"id":"campaign-studio/the-numbers-pass/s04-04-denominators-and-the-ftc"},{"id":"campaign-studio/the-numbers-pass/s05-05-what-may-not-ship"},{"id":"campaign-studio/the-numbers-pass/s06-06-fill-the-policy-router"},{"id":"campaign-studio/the-numbers-pass/s07-07-checkpoint"}]}]},{"slug":"support-studio","lessons":[{"slug":"the-policy-pack","steps":[{"id":"support-studio/the-policy-pack/s01-01-the-reply-that-binds-the-company"},{"id":"support-studio/the-policy-pack/s02-02-the-pack-as-data"},{"id":"support-studio/the-policy-pack/s03-03-spot-the-banned-promise"},{"id":"support-studio/the-policy-pack/s04-04-predict-the-lint"},{"id":"support-studio/the-policy-pack/s05-05-fix-the-stale-citation"},{"id":"support-studio/the-policy-pack/s06-06-the-review-loop-evidence"},{"id":"support-studio/the-policy-pack/s07-07-write-lint-draft"},{"id":"support-studio/the-policy-pack/s08-08-checkpoint"}]},{"slug":"golden-evals-earn-autonomy","steps":[{"id":"support-studio/golden-evals-earn-autonomy/s01-01-the-klarna-arc"},{"id":"support-studio/golden-evals-earn-autonomy/s02-02-a-macro-is-three-things"},{"id":"support-studio/golden-evals-earn-autonomy/s03-03-inside-the-lane"},{"id":"support-studio/golden-evals-earn-autonomy/s04-04-predict-the-eval-run"},{"id":"support-studio/golden-evals-earn-autonomy/s05-05-fill-the-promotion-rule"},{"id":"support-studio/golden-evals-earn-autonomy/s06-06-write-run-golden-evals"},{"id":"support-studio/golden-evals-earn-autonomy/s07-07-checkpoint"}]},{"slug":"the-escalation-boundary","steps":[{"id":"support-studio/the-escalation-boundary/s01-01-drafting-is-not-spending"},{"id":"support-studio/the-escalation-boundary/s02-02-triggers-as-conditions"},{"id":"support-studio/the-escalation-boundary/s03-03-route-this-ticket"},{"id":"support-studio/the-escalation-boundary/s04-04-predict-the-routing"},{"id":"support-studio/the-escalation-boundary/s05-05-fix-the-refund-hole"},{"id":"support-studio/the-escalation-boundary/s06-06-disclosure-and-the-kill-switch"},{"id":"support-studio/the-escalation-boundary/s07-07-write-route-ticket"},{"id":"support-studio/the-escalation-boundary/s08-08-checkpoint"}]}]},{"slug":"copy-studio","lessons":[{"slug":"the-voice-fingerprint","steps":[{"id":"copy-studio/the-voice-fingerprint/s01-01-the-client-you-inherit"},{"id":"copy-studio/the-voice-fingerprint/s02-02-the-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s03-03-predict-the-ceiling"},{"id":"copy-studio/the-voice-fingerprint/s04-04-the-samples-say-otherwise"},{"id":"copy-studio/the-voice-fingerprint/s05-05-fill-derive-the-rules"},{"id":"copy-studio/the-voice-fingerprint/s06-06-the-pack-with-provenance"},{"id":"copy-studio/the-voice-fingerprint/s07-07-write-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s08-08-checkpoint"}]},{"slug":"variants-that-hold-the-line","steps":[{"id":"copy-studio/variants-that-hold-the-line/s01-01-twelve-by-thursday"},{"id":"copy-studio/variants-that-hold-the-line/s02-02-the-drift-problem"},{"id":"copy-studio/variants-that-hold-the-line/s03-03-predict-the-drift-report"},{"id":"copy-studio/variants-that-hold-the-line/s04-04-which-subject-line-ships"},{"id":"copy-studio/variants-that-hold-the-line/s05-05-fix-the-channel-check"},{"id":"copy-studio/variants-that-hold-the-line/s06-06-the-kill-log"},{"id":"copy-studio/variants-that-hold-the-line/s07-07-write-voice-gate"},{"id":"copy-studio/variants-that-hold-the-line/s08-08-checkpoint"}]},{"slug":"the-claim-check","steps":[{"id":"copy-studio/the-claim-check/s01-01-forty-one-of-seventy-seven"},{"id":"copy-studio/the-claim-check/s02-02-what-counts-as-checkable"},{"id":"copy-studio/the-claim-check/s03-03-predict-the-claim-list"},{"id":"copy-studio/the-claim-check/s04-04-what-closes-a-row"},{"id":"copy-studio/the-claim-check/s05-05-fill-the-ship-decision"},{"id":"copy-studio/the-claim-check/s06-06-the-corrections-log"},{"id":"copy-studio/the-claim-check/s07-07-write-ship-gate"},{"id":"copy-studio/the-claim-check/s08-08-checkpoint"}]}]},{"slug":"analytics-studio","lessons":[{"slug":"the-cleaning-log","steps":[{"id":"analytics-studio/the-cleaning-log/s01-01-where-did-those-rows-go"},{"id":"analytics-studio/the-cleaning-log/s02-02-the-log-as-data"},{"id":"analytics-studio/the-cleaning-log/s03-03-refuse-the-silent-step"},{"id":"analytics-studio/the-cleaning-log/s04-04-predict-the-receipts"},{"id":"analytics-studio/the-cleaning-log/s05-05-fill-the-null-decision"},{"id":"analytics-studio/the-cleaning-log/s06-06-fix-the-vanishing-rows"},{"id":"analytics-studio/the-cleaning-log/s07-07-write-the-null-ledger"},{"id":"analytics-studio/the-cleaning-log/s08-08-checkpoint"}]},{"slug":"every-number-has-an-address","steps":[{"id":"analytics-studio/every-number-has-an-address/s01-01-the-plausible-wrong-number"},{"id":"analytics-studio/every-number-has-an-address/s02-02-sample-first-then-full"},{"id":"analytics-studio/every-number-has-an-address/s03-03-which-check-catches-it"},{"id":"analytics-studio/every-number-has-an-address/s04-04-predict-the-fan-out"},{"id":"analytics-studio/every-number-has-an-address/s05-05-fill-the-trace"},{"id":"analytics-studio/every-number-has-an-address/s06-06-write-audit-the-memo"},{"id":"analytics-studio/every-number-has-an-address/s07-07-checkpoint"}]},{"slug":"challenge-then-rerun","steps":[{"id":"analytics-studio/challenge-then-rerun/s01-01-the-confident-recommendation"},{"id":"analytics-studio/challenge-then-rerun/s02-02-the-triad-as-a-checklist"},{"id":"analytics-studio/challenge-then-rerun/s03-03-name-the-leg"},{"id":"analytics-studio/challenge-then-rerun/s04-04-predict-the-rerun"},{"id":"analytics-studio/challenge-then-rerun/s05-05-fix-the-drifting-memo"},{"id":"analytics-studio/challenge-then-rerun/s06-06-write-score-the-challenge"},{"id":"analytics-studio/challenge-then-rerun/s07-07-checkpoint"}]}]},{"slug":"delivery-studio","lessons":[{"slug":"the-spec-is-the-build","steps":[{"id":"delivery-studio/the-spec-is-the-build/s01-01-the-clarifying-question-is-gone"},{"id":"delivery-studio/the-spec-is-the-build/s02-02-the-ticket-as-data"},{"id":"delivery-studio/the-spec-is-the-build/s03-03-spot-the-untestable-check"},{"id":"delivery-studio/the-spec-is-the-build/s04-04-predict-the-spec-lint"},{"id":"delivery-studio/the-spec-is-the-build/s05-05-fill-the-scope-fence"},{"id":"delivery-studio/the-spec-is-the-build/s06-06-inside-or-outside-the-frontier"},{"id":"delivery-studio/the-spec-is-the-build/s07-07-write-lint-spec"},{"id":"delivery-studio/the-spec-is-the-build/s08-08-checkpoint"}]},{"slug":"acceptance-before-the-build","steps":[{"id":"delivery-studio/acceptance-before-the-build/s01-01-the-ninety-seven-thousand-dollar-skip"},{"id":"delivery-studio/acceptance-before-the-build/s02-02-evidence-against-the-checks"},{"id":"delivery-studio/acceptance-before-the-build/s03-03-which-check-catches-it"},{"id":"delivery-studio/acceptance-before-the-build/s04-04-fix-the-vibes-review"},{"id":"delivery-studio/acceptance-before-the-build/s05-05-predict-the-review-run"},{"id":"delivery-studio/acceptance-before-the-build/s06-06-rework-is-a-list-not-a-feeling"},{"id":"delivery-studio/acceptance-before-the-build/s07-07-write-review-deliverable"},{"id":"delivery-studio/acceptance-before-the-build/s08-08-checkpoint"}]},{"slug":"status-from-artifacts","steps":[{"id":"delivery-studio/status-from-artifacts/s01-01-watermelon-status"},{"id":"delivery-studio/status-from-artifacts/s02-02-derive-the-rag"},{"id":"delivery-studio/status-from-artifacts/s03-03-spot-the-artifact"},{"id":"delivery-studio/status-from-artifacts/s04-04-fill-the-worst-of"},{"id":"delivery-studio/status-from-artifacts/s05-05-predict-the-rollup"},{"id":"delivery-studio/status-from-artifacts/s06-06-the-not-ready-call"},{"id":"delivery-studio/status-from-artifacts/s07-07-write-readiness-gate"},{"id":"delivery-studio/status-from-artifacts/s08-08-checkpoint"}]},{"slug":"the-hours-saved-receipt","steps":[{"id":"delivery-studio/the-hours-saved-receipt/s01-01-the-token-chart-is-not-a-receipt"},{"id":"delivery-studio/the-hours-saved-receipt/s02-02-the-receipt-as-data"},{"id":"delivery-studio/the-hours-saved-receipt/s03-03-which-line-survives-staff-meeting"},{"id":"delivery-studio/the-hours-saved-receipt/s04-04-fix-the-vanity-chart"},{"id":"delivery-studio/the-hours-saved-receipt/s05-05-predict-the-receipt-run"},{"id":"delivery-studio/the-hours-saved-receipt/s06-06-only-accepted-ships-count"},{"id":"delivery-studio/the-hours-saved-receipt/s07-07-write-hours-receipt"},{"id":"delivery-studio/the-hours-saved-receipt/s08-08-checkpoint"}]}]},{"slug":"people-studio","lessons":[{"slug":"the-data-firewall","steps":[{"id":"people-studio/the-data-firewall/s01-01-the-rule-that-cannot-live-in-memory"},{"id":"people-studio/the-data-firewall/s02-02-the-firewall-as-data"},{"id":"people-studio/the-data-firewall/s03-03-which-payload-passes"},{"id":"people-studio/the-data-firewall/s04-04-predict-the-scan"},{"id":"people-studio/the-data-firewall/s05-05-fix-the-leaky-gate"},{"id":"people-studio/the-data-firewall/s06-06-employee-relations-takes-nothing"},{"id":"people-studio/the-data-firewall/s07-07-write-prompt-gate"},{"id":"people-studio/the-data-firewall/s08-08-checkpoint"}]},{"slug":"briefs-with-receipts","steps":[{"id":"people-studio/briefs-with-receipts/s01-01-the-most-automated-desk"},{"id":"people-studio/briefs-with-receipts/s02-02-a-claim-with-a-receipt"},{"id":"people-studio/briefs-with-receipts/s03-03-sourced-or-inference"},{"id":"people-studio/briefs-with-receipts/s04-04-predict-the-receipt-audit"},{"id":"people-studio/briefs-with-receipts/s05-05-fill-the-lookup"},{"id":"people-studio/briefs-with-receipts/s06-06-the-screening-line"},{"id":"people-studio/briefs-with-receipts/s07-07-write-audit-brief"},{"id":"people-studio/briefs-with-receipts/s08-08-checkpoint"}]},{"slug":"the-paper-trail","steps":[{"id":"people-studio/the-paper-trail/s01-01-policy-is-legal-surface"},{"id":"people-studio/the-paper-trail/s02-02-the-boundary-as-a-state-machine"},{"id":"people-studio/the-paper-trail/s03-03-who-signs-off"},{"id":"people-studio/the-paper-trail/s04-04-predict-the-version-log"},{"id":"people-studio/the-paper-trail/s05-05-fix-the-shortcut"},{"id":"people-studio/the-paper-trail/s06-06-the-evidence-trail"},{"id":"people-studio/the-paper-trail/s07-07-write-decision-record"},{"id":"people-studio/the-paper-trail/s08-08-checkpoint"}]}]},{"slug":"ops-studio","lessons":[{"slug":"the-sop-is-data","steps":[{"id":"ops-studio/the-sop-is-data/s01-01-the-two-weeks-notice-problem"},{"id":"ops-studio/the-sop-is-data/s02-02-the-sop-as-data"},{"id":"ops-studio/the-sop-is-data/s03-03-spot-the-invented-step"},{"id":"ops-studio/the-sop-is-data/s04-04-fill-the-sop-lint"},{"id":"ops-studio/the-sop-is-data/s05-05-predict-the-diff"},{"id":"ops-studio/the-sop-is-data/s06-06-the-diff-is-the-change-control"},{"id":"ops-studio/the-sop-is-data/s07-07-write-lint-sop"},{"id":"ops-studio/the-sop-is-data/s08-08-checkpoint"}]},{"slug":"auto-run-or-named-approval","steps":[{"id":"ops-studio/auto-run-or-named-approval/s01-01-the-checklist-that-lied"},{"id":"ops-studio/auto-run-or-named-approval/s02-02-the-schedule-is-data"},{"id":"ops-studio/auto-run-or-named-approval/s03-03-which-item-may-auto-run"},{"id":"ops-studio/auto-run-or-named-approval/s04-04-fix-the-silent-auto-complete"},{"id":"ops-studio/auto-run-or-named-approval/s05-05-predict-the-exception-queue"},{"id":"ops-studio/auto-run-or-named-approval/s06-06-exceptions-are-a-queue-not-a-shrug"},{"id":"ops-studio/auto-run-or-named-approval/s07-07-write-run-checklist"},{"id":"ops-studio/auto-run-or-named-approval/s08-08-checkpoint"}]},{"slug":"numbers-with-receipts","steps":[{"id":"ops-studio/numbers-with-receipts/s01-01-the-number-that-moves-money"},{"id":"ops-studio/numbers-with-receipts/s02-02-recompute-dont-trust"},{"id":"ops-studio/numbers-with-receipts/s03-03-which-claim-has-no-receipt"},{"id":"ops-studio/numbers-with-receipts/s04-04-predict-the-verification"},{"id":"ops-studio/numbers-with-receipts/s05-05-fill-the-receipt-check"},{"id":"ops-studio/numbers-with-receipts/s06-06-the-anomaly-is-a-question"},{"id":"ops-studio/numbers-with-receipts/s07-07-write-verify-report"},{"id":"ops-studio/numbers-with-receipts/s08-08-checkpoint"}]},{"slug":"the-pii-proxy","steps":[{"id":"ops-studio/the-pii-proxy/s01-01-the-ticket-that-left-the-building"},{"id":"ops-studio/the-pii-proxy/s02-02-mask-then-unmask"},{"id":"ops-studio/the-pii-proxy/s03-03-which-payload-may-leave"},{"id":"ops-studio/the-pii-proxy/s04-04-fill-the-leftover-check"},{"id":"ops-studio/the-pii-proxy/s05-05-predict-the-two-sides"},{"id":"ops-studio/the-pii-proxy/s06-06-unmask-is-a-named-human-act"},{"id":"ops-studio/the-pii-proxy/s07-07-write-the-proxy"},{"id":"ops-studio/the-pii-proxy/s08-08-checkpoint"}]}]},{"slug":"legal-studio","lessons":[{"slug":"the-cite-check-gate","steps":[{"id":"legal-studio/the-cite-check-gate/s01-01-the-brief-that-reads-perfectly"},{"id":"legal-studio/the-cite-check-gate/s02-02-three-verdicts-as-data"},{"id":"legal-studio/the-cite-check-gate/s03-03-which-cite-needs-no-check"},{"id":"legal-studio/the-cite-check-gate/s04-04-predict-the-verification-run"},{"id":"legal-studio/the-cite-check-gate/s05-05-fix-the-verifier-that-asks-again"},{"id":"legal-studio/the-cite-check-gate/s06-06-write-verify-authorities"},{"id":"legal-studio/the-cite-check-gate/s07-07-checkpoint"}]},{"slug":"clause-extraction-with-pinpoints","steps":[{"id":"legal-studio/clause-extraction-with-pinpoints/s01-01-a-claim-without-a-pinpoint-is-a-rumor"},{"id":"legal-studio/clause-extraction-with-pinpoints/s02-02-the-playbook-as-data"},{"id":"legal-studio/clause-extraction-with-pinpoints/s03-03-which-row-is-unusable"},{"id":"legal-studio/clause-extraction-with-pinpoints/s04-04-predict-the-triage"},{"id":"legal-studio/clause-extraction-with-pinpoints/s05-05-fill-the-walk-away-route"},{"id":"legal-studio/clause-extraction-with-pinpoints/s06-06-write-triage-extraction"},{"id":"legal-studio/clause-extraction-with-pinpoints/s07-07-checkpoint"}]},{"slug":"privilege-rules-and-the-review-log","steps":[{"id":"legal-studio/privilege-rules-and-the-review-log/s01-01-privilege-is-a-routing-question"},{"id":"legal-studio/privilege-rules-and-the-review-log/s02-02-the-routing-table"},{"id":"legal-studio/privilege-rules-and-the-review-log/s03-03-which-request-is-the-waiver-risk"},{"id":"legal-studio/privilege-rules-and-the-review-log/s04-04-predict-the-router"},{"id":"legal-studio/privilege-rules-and-the-review-log/s05-05-fix-the-privilege-hole"},{"id":"legal-studio/privilege-rules-and-the-review-log/s06-06-the-log-that-survives-a-motion"},{"id":"legal-studio/privilege-rules-and-the-review-log/s07-07-write-log-ready"},{"id":"legal-studio/privilege-rules-and-the-review-log/s08-08-checkpoint"}]}]},{"slug":"agent-studio","lessons":[{"slug":"the-tool-contract","steps":[{"id":"agent-studio/the-tool-contract/s01-01-the-feature-you-ship-today"},{"id":"agent-studio/the-tool-contract/s02-02-the-schema-is-the-blast-radius"},{"id":"agent-studio/the-tool-contract/s03-03-spot-the-loose-tool"},{"id":"agent-studio/the-tool-contract/s04-04-predict-the-validation"},{"id":"agent-studio/the-tool-contract/s05-05-fix-the-yolo-rerun"},{"id":"agent-studio/the-tool-contract/s06-06-write-validate-call"},{"id":"agent-studio/the-tool-contract/s07-07-checkpoint"}]},{"slug":"the-loop-with-a-meter","steps":[{"id":"agent-studio/the-loop-with-a-meter/s01-01-a-loop-that-cannot-stop-is-an-invoice"},{"id":"agent-studio/the-loop-with-a-meter/s02-02-caps-are-part-of-the-loop"},{"id":"agent-studio/the-loop-with-a-meter/s03-03-why-did-it-stop"},{"id":"agent-studio/the-loop-with-a-meter/s04-04-predict-the-stop-reason"},{"id":"agent-studio/the-loop-with-a-meter/s05-05-fill-the-budget-check"},{"id":"agent-studio/the-loop-with-a-meter/s06-06-the-trace-is-the-receipt"},{"id":"agent-studio/the-loop-with-a-meter/s07-07-write-run-loop"},{"id":"agent-studio/the-loop-with-a-meter/s08-08-checkpoint"}]},{"slug":"evals-before-it-works","steps":[{"id":"agent-studio/evals-before-it-works/s01-01-green-before-it-runs"},{"id":"agent-studio/evals-before-it-works/s02-02-a-golden-case-is-a-past-incident"},{"id":"agent-studio/evals-before-it-works/s03-03-which-case-catches-it"},{"id":"agent-studio/evals-before-it-works/s04-04-predict-the-eval-run"},{"id":"agent-studio/evals-before-it-works/s05-05-fix-the-router-to-green"},{"id":"agent-studio/evals-before-it-works/s06-06-the-ratchet-entry"},{"id":"agent-studio/evals-before-it-works/s07-07-write-lint-the-ratchet"},{"id":"agent-studio/evals-before-it-works/s08-08-checkpoint"}]}]}]},"detail":{"number":55,"slug":"ops-studio","title":"ops studio — tribal knowledge to running automation","blurb":"how a working ops lead gets the process out of one person's head and into automation that can be trusted: SOPs captured as versioned data with the invented steps hunted down, recurring checklists with an explicit auto-run vs named-approval line, exception queues instead of silent failures, a receipt against the export for every number in the weekly report, and a PII proxy that masks member identifiers before the model and restores them only for the named human who sends.","overview":"$27","status":"live","lessons":[{"slug":"the-sop-is-data","title":"The SOP is data — get the process out of Vera's head without letting the model invent it","estMinutes":20,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"ops-studio/the-sop-is-data/s01-01-the-two-weeks-notice-problem","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":180,"concept":"transcript-to-sop-pattern","body":"$28","cta":"Got it","runnable":true},{"type":"read","id":"ops-studio/the-sop-is-data/s02-02-the-sop-as-data","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":200,"concept":"sop-as-versioned-data","body":"$29","cta":"Got it","code":"SOP = {\n \"id\": \"sop-stuck-orders\",\n \"owner\": \"vera\",\n \"version\": \"1.0\",\n \"date\": \"2026-07-20\",\n \"steps\": [\n {\"n\": 1, \"action\": \"Pull the open-orders export from the WMS\",\n \"source\": \"transcript\", \"verified\": True},\n {\"n\": 2, \"action\": \"Filter to orders stuck more than 5 days\",\n \"source\": \"transcript\", \"verified\": True},\n {\"n\": 3, \"action\": \"Open Admin > Batch Tools > Requeue All\",\n \"source\": \"model-filled\", \"verified\": False},\n {\"n\": 4, \"action\": \"Post the carried list in #fulfillment\",\n \"source\": \"transcript\", \"verified\": True},\n ],\n}\n\nprint(f\"{SOP['id']} v{SOP['version']} — owner: {SOP['owner']}, dated {SOP['date']}\")\nfor s in SOP[\"steps\"]:\n risky = s[\"source\"] == \"model-filled\" and not s[\"verified\"]\n tag = \"NEEDS PERFORMER CHECK\" if risky else \"ok\"\n print(f\" step {s['n']}: {tag} ({s['action']})\")\n","runnable":true},{"type":"mc","id":"ops-studio/the-sop-is-data/s03-03-spot-the-invented-step","xp":2,"hint":[{"level":1,"body":"Find the transcript line where Vera explicitly rules something OUT. Then find the draft step that does the ruled-out thing anyway — dressed in a menu path she never mentioned.","cost":0},{"level":2,"body":"Step 3. Vera said 'one at a time — there is no bulk requeue, I checked.' The model hit that beat, invented an 'Admin > Batch Tools > Requeue All' path, and wrote the opposite of her method in her voice. This is the exact failure practitioners report: hallucinated menu paths, catchable only by the performer.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"catch-the-model-filled-step","prompt":"Vera's Monday, written down. An SOP is the written steps for that\nrecurring job; the WMS is the warehouse system she pulls the\nexport from. You're checking the draft against what she actually\nsaid. Which step did the model invent?\n","options":[{"id":"step_3","label":"3 — 'Open Admin > Batch Tools > Requeue All to requeue in bulk'"},{"id":"step_1","label":"1 — 'Pull the open-orders export from the WMS'","explain":"Vera's first transcript line says exactly this. Transcript-sourced, word for word — this one starts life presumed real."},{"id":"step_2","label":"2 — 'Filter to orders stuck more than 5 days'","explain":"'More than five days' is in the transcript; the model only converted the number to a digit. Reformatting isn't invention."},{"id":"step_4","label":"4 — 'Post the carried list in #fulfillment'","explain":"Vera said 'the fulfillment channel'; the model wrote it as '#fulfillment'. A rename worth confirming, but the action itself is straight from the recording."}],"answerIds":["step_3"],"shuffle":false,"code":"TRANSCRIPT = [\n \"first I pull the open-orders export from the WMS\",\n \"then I filter to anything stuck more than five days\",\n \"each stuck order gets requeued ONE AT A TIME — there is no bulk requeue, I checked\",\n \"and I post the carried list in the fulfillment channel\",\n]\nDRAFT = [\n \"1. Pull the open-orders export from the WMS\",\n \"2. Filter to orders stuck more than 5 days\",\n \"3. Open Admin > Batch Tools > Requeue All to requeue in bulk\",\n \"4. Post the carried list in #fulfillment\",\n]\nprint(\"— what Vera said —\")\nfor line in TRANSCRIPT:\n print(f\" {line}\")\nprint(\"— what the model drafted —\")\nfor line in DRAFT:\n print(f\" {line}\")\n","runnable":true},{"type":"fill","id":"ops-studio/the-sop-is-data/s04-04-fill-the-sop-lint","xp":3,"hint":[{"level":1,"body":"Two conditions joined by `and`: the step's `source` equals `\"model-filled\"`, and its `verified` flag is falsy (`not s[\"verified\"]`). Step 4 proves you need both — model-filled but verified prints ok.","cost":0},{"level":2,"body":"`s[\"source\"] == \"model-filled\" and not s[\"verified\"]` — flag only the steps the model wrote that the performer has not yet confirmed.","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"lint-unverified-model-fills","prompt":"The lint walks every step and flags the dangerous combination:\nthe model wrote it AND nobody who does the task has confirmed it.\nFill in the condition.\n\nNote step 4: model-filled but Vera already verified it — that one\nis fine. Provenance alone isn't the problem; provenance without\nverification is.\n\nExpected output:\n```\nstep 1: ok\nstep 2: ok\nstep 3: UNVERIFIED MODEL FILL\nstep 4: ok\n```\n","code":"SOP_STEPS = [\n {\"n\": 1, \"action\": \"Pull the open-orders export from the WMS\",\n \"source\": \"transcript\", \"verified\": True},\n {\"n\": 2, \"action\": \"Filter to orders stuck more than 5 days\",\n \"source\": \"transcript\", \"verified\": True},\n {\"n\": 3, \"action\": \"Open Admin > Batch Tools > Requeue All\",\n \"source\": \"model-filled\", \"verified\": False},\n {\"n\": 4, \"action\": \"Email the carried list to fulfillment\",\n \"source\": \"model-filled\", \"verified\": True},\n]\n\nfor s in SOP_STEPS:\n risky = ___\n print(f\"step {s['n']}: {'UNVERIFIED MODEL FILL' if risky else 'ok'}\")\n","blanks":[{"id":"risky-condition","accept":["s[\"source\"] == \"model-filled\" and not s[\"verified\"]","s['source'] == 'model-filled' and not s['verified']","s[\"source\"]==\"model-filled\" and not s[\"verified\"]","s['source']=='model-filled' and not s['verified']","not s[\"verified\"] and s[\"source\"] == \"model-filled\"","not s['verified'] and s['source'] == 'model-filled'","not s[\"verified\"] and s[\"source\"]==\"model-filled\"","not s['verified'] and s['source']=='model-filled'","s[\"source\"] == \"model-filled\" and s[\"verified\"] == False","s['source'] == 'model-filled' and s['verified'] == False","s[\"source\"]==\"model-filled\" and s[\"verified\"]==False","s['source']=='model-filled' and s['verified']==False","s[\"verified\"] == False and s[\"source\"] == \"model-filled\"","s['verified'] == False and s['source'] == 'model-filled'","(s[\"source\"] == \"model-filled\") and (not s[\"verified\"])","(s['source'] == 'model-filled') and (not s['verified'])","(not s[\"verified\"]) and (s[\"source\"] == \"model-filled\")","(not s['verified']) and (s['source'] == 'model-filled')"],"caseSensitive":true,"normalize":"collapse-ws"}]},{"type":"predict","id":"ops-studio/the-sop-is-data/s05-05-predict-the-diff","xp":3,"hint":[{"level":1,"body":"The merged keys are 1,2,3,4,5. Steps 1 and 4 are identical → skipped. Step 2 exists in both with different text → `~` with the old and new joined by `->`. Step 3 exists only in V1 → `-`. Step 5 exists only in V2 → `+`. Then the version line always prints.","cost":0},{"level":2,"body":"```\n~ step 2: Filter to orders stuck more than 5 days -> Filter to orders stuck more than 3 days\n- step 3: Requeue stuck orders one at a time\n+ step 5: Flag any order touching hazmat for the safety owner\nreview required: v1.0 -> v1.1\n```\n\nNotice what the diff surfaces that a fresh read-through wouldn't: someone is deleting the one-at-a-time requeue rule — the step Vera added for a reason. That's exactly the kind of change a reviewer needs to see as a `-` line, not absorb silently from a rewritten doc.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"sop-version-diff","code":"V1 = {\"version\": \"1.0\", \"steps\": {\n 1: \"Pull the open-orders export from the WMS\",\n 2: \"Filter to orders stuck more than 5 days\",\n 3: \"Requeue stuck orders one at a time\",\n 4: \"Post the carried list in #fulfillment\",\n}}\nV2 = {\"version\": \"1.1\", \"steps\": {\n 1: \"Pull the open-orders export from the WMS\",\n 2: \"Filter to orders stuck more than 3 days\",\n 4: \"Post the carried list in #fulfillment\",\n 5: \"Flag any order touching hazmat for the safety owner\",\n}}\n\nfor n in sorted(V1[\"steps\"] | V2[\"steps\"]):\n a, b = V1[\"steps\"].get(n), V2[\"steps\"].get(n)\n if a == b:\n continue\n if a is None:\n print(f\"+ step {n}: {b}\")\n elif b is None:\n print(f\"- step {n}: {a}\")\n else:\n print(f\"~ step {n}: {a} -> {b}\")\nprint(f\"review required: v{V1['version']} -> v{V2['version']}\")\n","prompt":"A proposed SOP change, rendered as the diff the reviewer will read.\n`V1[\"steps\"] | V2[\"steps\"]` merges both step dicts, so the loop\nvisits every step number that exists in either version, in sorted\norder. Unchanged steps are skipped; a step only in V2 prints as `+`,\nonly in V1 as `-`, present-but-different as `~`. What prints?\n","grader":{"kind":"stdout-equality","expected":"~ step 2: Filter to orders stuck more than 5 days -> Filter to orders stuck more than 3 days\n- step 3: Requeue stuck orders one at a time\n+ step 5: Flag any order touching hazmat for the safety owner\nreview required: v1.0 -> v1.1\n","normalize":"collapse-trailing-newline"}},{"type":"read","id":"ops-studio/the-sop-is-data/s06-06-the-diff-is-the-change-control","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":170,"concept":"change-control-as-reviewed-diff","body":"$2a","cta":"Got it","runnable":true},{"type":"write","id":"ops-studio/the-sop-is-data/s07-07-write-lint-sop","xp":5,"hint":[{"level":1,"body":"Owner check first, then ONE loop over the steps applying both step rules to each step in turn. The hazmat SOP is the trap: step 2 is model-filled but verified (no fill flag) while its safety_reviewed is False (safety flag fires). And step 1 has safety_reviewed True — `.get` matters because the receiving SOP's steps don't carry the key at all.","cost":0},{"level":2,"body":"```python\ndef lint_sop(sop):\n reasons = []\n if not sop[\"owner\"]:\n reasons.append(\"no owner\")\n for s in sop[\"steps\"]:\n if s[\"source\"] == \"model-filled\" and not s[\"verified\"]:\n reasons.append(f\"step {s['n']} unverified model fill\")\n if s[\"safety\"] and not s.get(\"safety_reviewed\"):\n reasons.append(f\"step {s['n']} safety step unreviewed\")\n return reasons\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":240,"concept":"write-sop-lint","needs":["functions-and-returns","lists-and-dicts","loops"],"prompt":"Finish `lint_sop(sop)` — the check you already read and filled.\nAn SOP is the written steps for a recurring job. This function\nis the gate every SOP passes before it ships. The shape is on\nthe page; fill the two checks. It returns a list of reason\nstrings, in this order:\n\n1. If `sop[\"owner\"]` is falsy → `\"no owner\"`\n2. Then for each step, in order, check both rules on that step:\n - `source == \"model-filled\"` and not `verified`\n → `f\"step {n} unverified model fill\"`\n - `safety` is truthy and `safety_reviewed` is not\n (use `.get(\"safety_reviewed\")` — most steps don't have the key)\n → `f\"step {n} safety step unreviewed\"`\n\nThe call site prints `READY` for an empty list, else\n`BLOCKED — ` with reasons joined by `\"; \"`.\n\nExpected output:\n```\nsop-receiving: READY\nsop-stuck-orders: BLOCKED — step 3 unverified model fill\nsop-hazmat-intake: BLOCKED — no owner; step 2 safety step unreviewed\n```\n","starter":"$2b","grader":{"kind":"stdout-equality","expected":"sop-receiving: READY\nsop-stuck-orders: BLOCKED — step 3 unverified model fill\nsop-hazmat-intake: BLOCKED — no owner; step 2 safety step unreviewed\n","normalize":"collapse-trailing-newline"},"solution":"$2c","hiddenTests":[]},{"type":"checkpoint","id":"ops-studio/the-sop-is-data/s08-08-checkpoint","xp":8,"hint":[{"level":1,"body":"Three rule groups, appended in order: version bump, reviewer (an if/elif — 'no reviewer' and 'reviewer is the drafter' can't both fire), then the two step rules per proposed step. Traps: sop-stuck-orders bumped its version and lints clean — it blocks ONLY on self-review. sop-hazmat-intake collects two reasons from two different groups, in gate order.","cost":0},{"level":2,"body":"```python\ndef release_sop(change):\n reasons = []\n if change[\"proposed\"][\"version\"] == change[\"current\"][\"version\"]:\n reasons.append(\"version not bumped\")\n if not change.get(\"reviewer\"):\n reasons.append(\"no reviewer\")\n elif change[\"reviewer\"] == change[\"drafter\"]:\n reasons.append(\"reviewer is the drafter\")\n for s in change[\"proposed\"][\"steps\"]:\n if s[\"source\"] == \"model-filled\" and not s[\"verified\"]:\n reasons.append(f\"step {s['n']} unverified model fill\")\n if s.get(\"safety\") and not s.get(\"safety_reviewed\"):\n reasons.append(f\"step {s['n']} safety step unreviewed\")\n return reasons\n```\n\nThis gate is the lesson in one function: provenance and verification from the capture rules, version and reviewer independence from change control, safety review because the duty behind it never delegates. An SOP that clears all three groups is one a stranger can trust — and change.\n","cost":0}],"personalize":false,"phase":"check","estSeconds":300,"concept":"sop-release-gate","prompt":"Checkpoint: the release gate. Every proposed SOP change goes through\n`release_sop(change)` before it becomes the new version.\n\nA `change` has `id`, `drafter`, `reviewer`, `current` (with\n`version`), and `proposed` (with `version` and `steps`). Collect\nreason strings in this order:\n\n1. Proposed version equals current version → `\"version not bumped\"`\n2. Reviewer falsy → `\"no reviewer\"`; otherwise reviewer equals\n drafter → `\"reviewer is the drafter\"`\n3. For each proposed step, both step rules from the lint:\n - model-filled and unverified → `f\"step {n} unverified model fill\"`\n - safety-touching without safety review (use `.get` for both\n keys) → `f\"step {n} safety step unreviewed\"`\n\nReturn the list. The call site prints `RELEASED v{version}` when\nempty, else `BLOCKED — ` with reasons `\"; \"`-joined.\n\nExpected output:\n```\nsop-receiving: RELEASED v1.1\nsop-stuck-orders: BLOCKED — reviewer is the drafter\nsop-hazmat-intake: BLOCKED — version not bumped; step 2 unverified model fill\n```\n","starter":"$2d","grader":{"kind":"stdout-equality","expected":"sop-receiving: RELEASED v1.1\nsop-stuck-orders: BLOCKED — reviewer is the drafter\nsop-hazmat-intake: BLOCKED — version not bumped; step 2 unverified model fill\n","normalize":"collapse-trailing-newline"},"solution":"$2e","passThreshold":0.66}],"xpTotal":24},{"slug":"auto-run-or-named-approval","title":"Auto-run or named approval — the checklist automation that never forges a signature","estMinutes":20,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"ops-studio/auto-run-or-named-approval/s01-01-the-checklist-that-lied","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":180,"concept":"sign-offs-are-duties","body":"$2f","cta":"Got it","runnable":true},{"type":"read","id":"ops-studio/auto-run-or-named-approval/s02-02-the-schedule-is-data","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":200,"concept":"checklist-as-schedule-data","body":"$30","cta":"Got it","code":"TODAY = 200 # day-of-year; deterministic on purpose — no datetime.now() in a grader\n\nCHECKLIST = [\n {\"task\": \"regenerate-low-stock-report\", \"every\": 7, \"last_done\": 192,\n \"owner\": \"sam\", \"approval\": \"auto\"},\n {\"task\": \"forklift-inspection-log\", \"every\": 1, \"last_done\": 199,\n \"owner\": \"miguel\", \"approval\": \"miguel\", \"anchor\": \"daily equipment inspection\"},\n {\"task\": \"post-300a-summary\", \"every\": 365, \"last_done\": -200,\n \"owner\": \"vera\", \"approval\": \"vera\", \"anchor\": \"29 CFR 1904\"},\n {\"task\": \"vendor-scorecards\", \"every\": 30, \"last_done\": 185,\n \"owner\": \"sam\", \"approval\": \"auto\"},\n]\n\nfor item in CHECKLIST:\n due = TODAY - item[\"last_done\"] >= item[\"every\"]\n route = \"auto-run\" if item[\"approval\"] == \"auto\" else f\"needs {item['approval']}\"\n print(f\"{item['task']}: {'DUE — ' + route if due else 'not due'}\")\n","runnable":true},{"type":"mc","id":"ops-studio/auto-run-or-named-approval/s03-03-which-item-may-auto-run","xp":2,"hint":[{"level":1,"body":"Apply the two-question test from the reading: is it reversible, and is there a duty or a dollar behind it? Four of these carry a legal duty, a safety act, or spend. One is a draft over an export that you can regenerate at zero cost if it's wrong.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"draw-the-approval-line","prompt":"Five actions your checklist automation could take tonight,\nunattended. Exactly one belongs on the auto-run side of the line.\nWhich?\n","options":[{"id":"cand_a","label":"A — regenerate the low-stock report draft"},{"id":"cand_b","label":"B — mark the forklift daily inspection as done","explain":"The inspection is a physical act with a safety duty behind it. A script marking it done inspects nothing — it manufactures a record that says nobody needed to. This is the checklist-that-lied failure, verbatim."},{"id":"cand_c","label":"C — submit the OSHA 300A summary electronically","explain":"The 300A is a recordkeeping duty under 29 CFR 1904, held by the employer's named people — someone has to answer for what the submission says. The machine may prepare it and flag it as due; a named human submits."},{"id":"cand_d","label":"D — approve vendor invoice #4471 for $18,400","explain":"Approving spend binds the company — contract authority is a designated-human act, and $18,400 is real money leaving a real account. Route to the approver with the invoice attached; never approve unattended."},{"id":"cand_e","label":"E — close out Tuesday's entry in the injury log","explain":"The Form 300 injury log is an OSHA record employees have the right to review. Entries open and close under a named owner. An automation editing the injury log unattended is an audit finding writing itself."}],"answerIds":["cand_a"],"shuffle":false,"code":"CANDIDATES = [\n (\"A\", \"Regenerate the low-stock report draft from last night's export\"),\n (\"B\", \"Mark the forklift daily inspection as done\"),\n (\"C\", \"Submit the OSHA 300A summary electronically\"),\n (\"D\", \"Approve vendor invoice #4471 for $18,400\"),\n (\"E\", \"Close out Tuesday's entry in the injury log\"),\n]\nfor k, action in CANDIDATES:\n print(f\"{k}. {action}\")\n","runnable":true},{"type":"fix","id":"ops-studio/auto-run-or-named-approval/s04-04-fix-the-silent-auto-complete","xp":3,"hint":[{"level":1,"body":"Branch inside the loop on `it[\"approval\"]`. Equal to `\"auto\"` → print the `ran (auto)` line. Anything else is a NAME — print `queued for` plus that name. The approval value does double duty as the routing target; that's why the field holds a person, not a boolean.","cost":0},{"level":2,"body":"```python\ndef run_due(due):\n for it in due:\n if it[\"approval\"] == \"auto\":\n print(f\"{it['task']}: ran (auto)\")\n else:\n print(f\"{it['task']}: queued for {it['approval']}\")\n```\n\nThe forklift inspection and the 300A now end the night in a named human's queue instead of a fake done-pile. The bot lost the power to lie; that was the fix.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":110,"concept":"never-auto-complete-a-sign-off","brokenCode":"DUE = [\n {\"task\": \"regenerate-low-stock-report\", \"approval\": \"auto\"},\n {\"task\": \"post-300a-summary\", \"approval\": \"vera\", \"anchor\": \"29 CFR 1904\"},\n {\"task\": \"forklift-inspection-log\", \"approval\": \"miguel\", \"anchor\": \"daily equipment inspection\"},\n]\n\ndef run_due(due):\n for it in due:\n # bug: everything \"completes\" — including the legal sign-offs\n print(f\"{it['task']}: done\")\n\nrun_due(DUE)\n","prompt":"This run function is the checklist that lied, in three lines: every\ndue item \"completes,\" including the two with legal duties behind\nthem. Fix `run_due` so it respects the `approval` field — items\nwith `\"auto\"` print `{task}: ran (auto)`; items with a named\napprover print `{task}: queued for {name}` and are NOT run.\n\nExpected output:\n```\nregenerate-low-stock-report: ran (auto)\npost-300a-summary: queued for vera\nforklift-inspection-log: queued for miguel\n```\n","grader":{"kind":"stdout-equality","expected":"regenerate-low-stock-report: ran (auto)\npost-300a-summary: queued for vera\nforklift-inspection-log: queued for miguel\n","normalize":"collapse-trailing-newline"},"bugLines":[9,10],"revealAfter":6},{"type":"predict","id":"ops-studio/auto-run-or-named-approval/s05-05-predict-the-exception-queue","xp":3,"hint":[{"level":1,"body":"Two passes through the output: first the ok items print as they're met (tasks 1 and 3), failures are silently collected. Then the count line, then the queue — each exception indented, task and reason joined by an em dash, in the order they were appended.","cost":0},{"level":2,"body":"```\npull-wms-export: done\ncompile-reorder-suggestions: done\nopen exceptions: 2\n regenerate-low-stock-report — export missing\n vendor-scorecards — source stale: 41 days old\n```\n\nThe shape to notice: the failures don't interleave with the successes — they land in one block at the bottom, countable and assignable. That's a queue you can walk into standup with.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":85,"concept":"exceptions-route-to-a-queue","code":"RESULTS = {\n \"pull-wms-export\": \"ok\",\n \"regenerate-low-stock-report\": \"export missing\",\n \"compile-reorder-suggestions\": \"ok\",\n \"vendor-scorecards\": \"source stale: 41 days old\",\n}\n\nexceptions = []\nfor task, result in RESULTS.items():\n if result == \"ok\":\n print(f\"{task}: done\")\n else:\n exceptions.append((task, result))\n\nprint(f\"open exceptions: {len(exceptions)}\")\nfor task, reason in exceptions:\n print(f\" {task} — {reason}\")\n","prompt":"Last night's auto-run results. Anything not \"ok\" is deferred into\nthe exception queue and reported at the end, with its reason —\nit does NOT print in place. Dict order is insertion order.\nWhat prints?\n","grader":{"kind":"stdout-equality","expected":"pull-wms-export: done\ncompile-reorder-suggestions: done\nopen exceptions: 2\n regenerate-low-stock-report — export missing\n vendor-scorecards — source stale: 41 days old\n","normalize":"collapse-trailing-newline"}},{"type":"read","id":"ops-studio/auto-run-or-named-approval/s06-06-exceptions-are-a-queue-not-a-shrug","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":160,"concept":"exception-queue-over-silent-failure","body":"$31","cta":"Got it","runnable":true},{"type":"write","id":"ops-studio/auto-run-or-named-approval/s07-07-write-run-checklist","xp":5,"hint":[{"level":1,"body":"Collect three lists in one loop (due check first — an item that isn't due never reaches the approval question), then print the three sections. Due math from the schedule step: `today - last_done >= every` means due, so `< every` means not. Check: low-stock 200-192=8 ≥ 7 → due, auto. Forklift 1 ≥ 1 → due, named. 300A 400 ≥ 365 → due, named. Scorecards 15 < 30 → not due.","cost":0},{"level":2,"body":"```python\ndef run_checklist(items, today):\n auto, approval, not_due = [], [], []\n for it in items:\n if today - it[\"last_done\"] < it[\"every\"]:\n not_due.append(it[\"task\"])\n elif it[\"approval\"] == \"auto\":\n auto.append(it[\"task\"])\n else:\n approval.append(f\"{it['task']} -> {it['approval']}\")\n print(\"auto-run:\")\n for t in auto:\n print(f\" {t}\")\n print(\"approval needed:\")\n for t in approval:\n print(f\" {t}\")\n print(\"not due:\")\n for t in not_due:\n print(f\" {t}\")\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":240,"concept":"write-checklist-router","prompt":"Write `run_checklist(items, today)` — the router. One pass over the\nitems, three buckets, printed as sections:\n\n1. Not yet due (`today - last_done < every`) → bucket `not due`\n2. Due with `approval == \"auto\"` → bucket `auto-run`\n3. Due with a named approver → bucket `approval needed`, rendered\n as `{task} -> {approver}`\n\nPrint the sections in this order — `auto-run:`, `approval needed:`,\n`not due:` — each item on its own line indented two spaces, items\nin input order within each section.\n\nExpected output:\n```\nauto-run:\n regenerate-low-stock-report\napproval needed:\n forklift-inspection-log -> miguel\n post-300a-summary -> vera\nnot due:\n vendor-scorecards\n```\n","starter":"TODAY = 200\n\nITEMS = [\n {\"task\": \"regenerate-low-stock-report\", \"every\": 7, \"last_done\": 192, \"approval\": \"auto\"},\n {\"task\": \"forklift-inspection-log\", \"every\": 1, \"last_done\": 199, \"approval\": \"miguel\"},\n {\"task\": \"post-300a-summary\", \"every\": 365, \"last_done\": -200, \"approval\": \"vera\"},\n {\"task\": \"vendor-scorecards\", \"every\": 30, \"last_done\": 185, \"approval\": \"auto\"},\n]\n\n# define run_checklist(items, today) below\n\nrun_checklist(ITEMS, TODAY)\n","grader":{"kind":"stdout-equality","expected":"auto-run:\n regenerate-low-stock-report\napproval needed:\n forklift-inspection-log -> miguel\n post-300a-summary -> vera\nnot due:\n vendor-scorecards\n","normalize":"collapse-trailing-newline"},"solution":"TODAY = 200\n\nITEMS = [\n {\"task\": \"regenerate-low-stock-report\", \"every\": 7, \"last_done\": 192, \"approval\": \"auto\"},\n {\"task\": \"forklift-inspection-log\", \"every\": 1, \"last_done\": 199, \"approval\": \"miguel\"},\n {\"task\": \"post-300a-summary\", \"every\": 365, \"last_done\": -200, \"approval\": \"vera\"},\n {\"task\": \"vendor-scorecards\", \"every\": 30, \"last_done\": 185, \"approval\": \"auto\"},\n]\n\ndef run_checklist(items, today):\n auto, approval, not_due = [], [], []\n for it in items:\n if today - it[\"last_done\"] < it[\"every\"]:\n not_due.append(it[\"task\"])\n elif it[\"approval\"] == \"auto\":\n auto.append(it[\"task\"])\n else:\n approval.append(f\"{it['task']} -> {it['approval']}\")\n print(\"auto-run:\")\n for t in auto:\n print(f\" {t}\")\n print(\"approval needed:\")\n for t in approval:\n print(f\" {t}\")\n print(\"not due:\")\n for t in not_due:\n print(f\" {t}\")\n\nrun_checklist(ITEMS, TODAY)\n","hiddenTests":[]},{"type":"checkpoint","id":"ops-studio/auto-run-or-named-approval/s08-08-checkpoint","xp":8,"hint":[{"level":1,"body":"Order the branches by authority: due check first, then the named-approval route (BEFORE looking at result — named items don't have a result, because the machine never ran them; that's the point), then ok/exception for auto items. Count as you print. The forklift and 300A rows must never touch `it[\"result\"]` — indexing it would KeyError, and that crash is the data structure telling you the machine has no business knowing.","cost":0},{"level":2,"body":"```python\ndef nightly_run(items, today):\n ran = exceptions = waiting = 0\n for it in items:\n if today - it[\"last_done\"] < it[\"every\"]:\n print(f\"{it['task']}: not due\")\n elif it[\"approval\"] != \"auto\":\n waiting += 1\n print(f\"{it['task']}: awaiting sign-off ({it['approval']})\")\n elif it[\"result\"] == \"ok\":\n ran += 1\n print(f\"{it['task']}: ran\")\n else:\n exceptions += 1\n print(f\"{it['task']}: EXCEPTION — {it['result']}\")\n print(f\"ran: {ran} exceptions: {exceptions} awaiting sign-off: {waiting}\")\n if exceptions:\n print(f\"verdict: NEEDS ATTENTION — {exceptions} exception(s)\")\n else:\n print(\"verdict: CLEAN\")\n```\n\nRead the report like the 8am human will: one failure, loud, with a reason; two duties sitting with the right named people; nothing faked. This run had problems and the report is still trustworthy — which is the entire job description of an automated checklist.\n","cost":0}],"personalize":false,"phase":"check","estSeconds":300,"concept":"full-nightly-run-report","prompt":"Checkpoint: the complete nightly run. Due computation, approval\nrouting, exception queue — one function, one honest report.\n\nWrite `nightly_run(items, today)`. For each item in order, print\none line:\n\n- not due (`today - last_done < every`) → `{task}: not due`\n- due, named approver → `{task}: awaiting sign-off ({name})`\n- due, auto, `result == \"ok\"` → `{task}: ran`\n- due, auto, any other result → `{task}: EXCEPTION — {result}`\n\nThen the summary line\n`ran: {n} exceptions: {n} awaiting sign-off: {n}`\n(two spaces between fields), and the verdict:\n`verdict: NEEDS ATTENTION — {n} exception(s)` if any exceptions,\nelse `verdict: CLEAN`.\n\nExpected output:\n```\npull-wms-export: EXCEPTION — export missing\nregenerate-low-stock-report: ran\nforklift-inspection-log: awaiting sign-off (miguel)\npost-300a-summary: awaiting sign-off (vera)\nvendor-scorecards: not due\nran: 1 exceptions: 1 awaiting sign-off: 2\nverdict: NEEDS ATTENTION — 1 exception(s)\n```\n","starter":"TODAY = 200\n\nITEMS = [\n {\"task\": \"pull-wms-export\", \"every\": 1, \"last_done\": 199,\n \"approval\": \"auto\", \"result\": \"export missing\"},\n {\"task\": \"regenerate-low-stock-report\", \"every\": 7, \"last_done\": 192,\n \"approval\": \"auto\", \"result\": \"ok\"},\n {\"task\": \"forklift-inspection-log\", \"every\": 1, \"last_done\": 199,\n \"approval\": \"miguel\"},\n {\"task\": \"post-300a-summary\", \"every\": 365, \"last_done\": -200,\n \"approval\": \"vera\"},\n {\"task\": \"vendor-scorecards\", \"every\": 30, \"last_done\": 185,\n \"approval\": \"auto\", \"result\": \"ok\"},\n]\n\n# define nightly_run(items, today) below\n\nnightly_run(ITEMS, TODAY)\n","grader":{"kind":"stdout-equality","expected":"pull-wms-export: EXCEPTION — export missing\nregenerate-low-stock-report: ran\nforklift-inspection-log: awaiting sign-off (miguel)\npost-300a-summary: awaiting sign-off (vera)\nvendor-scorecards: not due\nran: 1 exceptions: 1 awaiting sign-off: 2\nverdict: NEEDS ATTENTION — 1 exception(s)\n","normalize":"collapse-trailing-newline"},"solution":"$32","passThreshold":0.66}],"xpTotal":24},{"slug":"numbers-with-receipts","title":"Numbers with receipts — every figure in the ops report traces to the export or it doesn't ship","estMinutes":18,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"ops-studio/numbers-with-receipts/s01-01-the-number-that-moves-money","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":180,"concept":"report-numbers-move-money","body":"$33","cta":"Got it","runnable":true},{"type":"read","id":"ops-studio/numbers-with-receipts/s02-02-recompute-dont-trust","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":190,"concept":"recompute-against-the-export","body":"$34","cta":"Got it","code":"EXPORT = [\n {\"order\": \"A-101\", \"days_to_ship\": 2},\n {\"order\": \"A-102\", \"days_to_ship\": 9},\n {\"order\": \"A-103\", \"days_to_ship\": 3},\n {\"order\": \"A-104\", \"days_to_ship\": 4},\n {\"order\": \"A-105\", \"days_to_ship\": 12},\n]\n\nclaimed_avg = 4.2 # the number sitting in the AI-drafted weekly report\n\nactual_avg = sum(r[\"days_to_ship\"] for r in EXPORT) / len(EXPORT)\nprint(f\"claimed: {claimed_avg}\")\nprint(f\"actual: {actual_avg}\")\nif abs(claimed_avg - actual_avg) < 0.05:\n print(\"VERIFIED\")\nelse:\n print(f\"MISMATCH — report says {claimed_avg}, export says {actual_avg}\")\n","runnable":true},{"type":"mc","id":"ops-studio/numbers-with-receipts/s03-03-which-claim-has-no-receipt","xp":2,"hint":[{"level":1,"body":"Play the auditor's game: for each claim, name the column(s) and the formula that would reproduce it. Three claims survive. One references a quantity — satisfaction — that appears in no column, this week or any week. The model didn't miscalculate it; it conjured it, comparison and all.","cost":0},{"level":2,"body":"C. There is no satisfaction data anywhere in this export, so no computation can verify OR refute the claim — and 'improved 12%' additionally implies a prior-week baseline the model also never saw. Unfalsifiable-from-source is worse than wrong: a wrong number gets corrected, an untraceable one gets repeated. It comes out of the report entirely.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"claims-must-be-derivable","prompt":"Four claims from the AI-drafted weekly report, and the columns of\nthe export it was drafted from. Three claims can be recomputed from\nthose columns. One cannot be traced to this export at all — no\nformula over these fields produces it. Which claim has no receipt?\n","options":[{"id":"claim_c","label":"C — customer satisfaction improved 12% this week"},{"id":"claim_a","label":"A — average days to ship was 6.0","explain":"Mean of days_to_ship — one line of arithmetic over a column that exists. Verifiable, and (from the last step) actually true."},{"id":"claim_b","label":"B — 2 of 5 orders shipped late (over 5 days)","explain":"Count rows where days_to_ship > 5. Derivable straight from the export — the definition of 'late' is even stated in the claim."},{"id":"claim_d","label":"D — 1 order carries a damage flag","explain":"Count of damage_flag — a column the export has. Trivially checkable."}],"answerIds":["claim_c"],"shuffle":false,"code":"# the export's columns — the ONLY data the report may draw on\nCOLUMNS = [\"order\", \"days_to_ship\", \"carrier\", \"damage_flag\"]\n\nCLAIMS = [\n (\"A\", \"average days to ship was 6.0\"),\n (\"B\", \"2 of 5 orders shipped late (over 5 days)\"),\n (\"C\", \"customer satisfaction improved 12% this week\"),\n (\"D\", \"1 order carries a damage flag\"),\n]\n\nprint(f\"export columns: {', '.join(COLUMNS)}\")\nfor k, claim in CLAIMS:\n print(f\"{k}. {claim}\")\n","runnable":true},{"type":"predict","id":"ops-studio/numbers-with-receipts/s04-04-predict-the-verification","xp":3,"hint":[{"level":1,"body":"len(DAYS) is 6, not 7 — the report over-counted its own row count, which is the first MISMATCH with both numbers shown. sum(DAYS)=33, 33/6=5.5 exactly → VERIFIED. max is 9 → VERIFIED. `d <= 5` admits 4, 5, and 2 — three of six — and 100*3/6 is 50.0 → VERIFIED.","cost":0},{"level":2,"body":"```\norders: MISMATCH (report 7, export 6)\navg_days: VERIFIED\nmax_days: VERIFIED\npct_within_5: VERIFIED\n```\n\nThe catch is the most mundane claim on the list — the row count. That's typical: fabrications cluster in the numbers nobody bothers to check because they seem too boring to be wrong.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"predict-verification-pass","code":"DAYS = [4, 6, 5, 9, 2, 7]\n\nCLAIMS = [\n (\"orders\", 7),\n (\"avg_days\", 5.5),\n (\"max_days\", 9),\n (\"pct_within_5\", 50.0),\n]\n\nACTUAL = {\n \"orders\": len(DAYS),\n \"avg_days\": sum(DAYS) / len(DAYS),\n \"max_days\": max(DAYS),\n \"pct_within_5\": 100 * sum(1 for d in DAYS if d <= 5) / len(DAYS),\n}\n\nfor name, claimed in CLAIMS:\n actual = ACTUAL[name]\n if claimed == actual:\n print(f\"{name}: VERIFIED\")\n else:\n print(f\"{name}: MISMATCH (report {claimed}, export {actual})\")\n","prompt":"A four-claim verification pass over six shipping durations. Do the\narithmetic before you answer: the sum of DAYS, the count, and —\ncarefully — which values pass `d <= 5` (it's `<=`, not `<`).\n(These fixture values divide exactly, so `==` is safe here —\nthe tolerance version comes next step.) What prints?\n","grader":{"kind":"stdout-equality","expected":"orders: MISMATCH (report 7, export 6)\navg_days: VERIFIED\nmax_days: VERIFIED\npct_within_5: VERIFIED\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"ops-studio/numbers-with-receipts/s05-05-fill-the-receipt-check","xp":3,"hint":[{"level":1,"body":"Distance between the two numbers, compared to the tolerance: `abs(claimed - actual)` on the left, `<= tolerance` on the right. Order inside `abs` doesn't matter — that's why it's there. Check it against the data: 0.0 passes, 1.8 fails, 0.02 passes.","cost":0},{"level":2,"body":"`abs(claimed - actual) <= tolerance` — the one-line policy that separates rounding from fabrication. Everything the rest of this lesson builds calls back to this expression.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"tolerance-comparison","prompt":"The receipt check in one expression: a claim is verified when its\nabsolute difference from the recomputed actual is within tolerance.\nFill in the return expression using `abs`, so that honest rounding\n(94.98 vs 95.0) passes and fabrication (4.2 vs 6.0) fails.\n\nExpected output:\n```\nVERIFIED\nMISMATCH\nVERIFIED\n```\n","code":"def receipt_check(claimed, actual, tolerance=0.05):\n return ___\n\nchecks = [(6.0, 6.0), (4.2, 6.0), (94.98, 95.0)]\nfor claimed, actual in checks:\n print(\"VERIFIED\" if receipt_check(claimed, actual) else \"MISMATCH\")\n","blanks":[{"id":"tolerance-expr","accept":["abs(claimed - actual) <= tolerance","abs(actual - claimed) <= tolerance","abs(claimed-actual) <= tolerance","abs(actual-claimed) <= tolerance","abs(claimed - actual) < tolerance","abs(actual - claimed) < tolerance","abs(claimed-actual) < tolerance","abs(actual-claimed) < tolerance","abs(claimed - actual)<=tolerance","abs(actual - claimed)<=tolerance","abs(claimed-actual)<=tolerance","abs(actual-claimed)<=tolerance","abs(claimed - actual) 5),\n \"damage_rate_pct\": 100 * sum(1 for r in export if r[\"damaged\"]) / len(export),\n }\n for name, claimed in claims:\n actual = actuals[name]\n if abs(claimed - actual) <= 0.05:\n print(f\"{name}: VERIFIED ({claimed})\")\n else:\n print(f\"{name}: MISMATCH — report {claimed}, export {actual}\")\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":240,"concept":"write-report-verifier","prompt":"Write `verify_report(claims, export)` — the incoming inspection for\nan AI-drafted report. Recompute the four metrics from the export:\n\n- `orders_shipped` — row count\n- `avg_days` — mean of `days`\n- `late_orders` — count of rows with `days > 5`\n- `damage_rate_pct` — percent of rows with `damaged` truthy\n (`100 * count / rows`)\n\nThen for each `(name, claimed)` pair in order, compare with\ntolerance 0.05:\n\n- within tolerance → `{name}: VERIFIED ({claimed})`\n- otherwise → `{name}: MISMATCH — report {claimed}, export {actual}`\n\nExpected output:\n```\norders_shipped: VERIFIED (5)\navg_days: MISMATCH — report 4.2, export 6.0\nlate_orders: VERIFIED (2)\ndamage_rate_pct: VERIFIED (20.0)\n```\n","starter":"EXPORT = [\n {\"order\": \"A-101\", \"days\": 2, \"damaged\": False},\n {\"order\": \"A-102\", \"days\": 9, \"damaged\": False},\n {\"order\": \"A-103\", \"days\": 3, \"damaged\": True},\n {\"order\": \"A-104\", \"days\": 4, \"damaged\": False},\n {\"order\": \"A-105\", \"days\": 12, \"damaged\": False},\n]\n\nCLAIMS = [\n (\"orders_shipped\", 5),\n (\"avg_days\", 4.2),\n (\"late_orders\", 2),\n (\"damage_rate_pct\", 20.0),\n]\n\n# define verify_report(claims, export) below\n\nverify_report(CLAIMS, EXPORT)\n","grader":{"kind":"stdout-equality","expected":"orders_shipped: VERIFIED (5)\navg_days: MISMATCH — report 4.2, export 6.0\nlate_orders: VERIFIED (2)\ndamage_rate_pct: VERIFIED (20.0)\n","normalize":"collapse-trailing-newline"},"solution":"EXPORT = [\n {\"order\": \"A-101\", \"days\": 2, \"damaged\": False},\n {\"order\": \"A-102\", \"days\": 9, \"damaged\": False},\n {\"order\": \"A-103\", \"days\": 3, \"damaged\": True},\n {\"order\": \"A-104\", \"days\": 4, \"damaged\": False},\n {\"order\": \"A-105\", \"days\": 12, \"damaged\": False},\n]\n\nCLAIMS = [\n (\"orders_shipped\", 5),\n (\"avg_days\", 4.2),\n (\"late_orders\", 2),\n (\"damage_rate_pct\", 20.0),\n]\n\ndef verify_report(claims, export):\n actuals = {\n \"orders_shipped\": len(export),\n \"avg_days\": sum(r[\"days\"] for r in export) / len(export),\n \"late_orders\": sum(1 for r in export if r[\"days\"] > 5),\n \"damage_rate_pct\": 100 * sum(1 for r in export if r[\"damaged\"]) / len(export),\n }\n for name, claimed in claims:\n actual = actuals[name]\n if abs(claimed - actual) <= 0.05:\n print(f\"{name}: VERIFIED ({claimed})\")\n else:\n print(f\"{name}: MISMATCH — report {claimed}, export {actual}\")\n\nverify_report(CLAIMS, EXPORT)\n","hiddenTests":[]},{"type":"checkpoint","id":"ops-studio/numbers-with-receipts/s08-08-checkpoint","xp":8,"hint":[{"level":1,"body":"This is the previous step's verifier plus three things: a name header, a `failing` list appended to inside the MISMATCH branch, and the verdict line after the loop. Everything indents two spaces under the report name. The first report holds on exactly one metric; the corrected one — same export, avg fixed to 6.0 — sends clean.","cost":0},{"level":2,"body":"$36","cost":0}],"personalize":false,"phase":"check","estSeconds":300,"concept":"send-hold-gate","prompt":"$37","starter":"EXPORT = [\n {\"order\": \"A-101\", \"days\": 2, \"damaged\": False},\n {\"order\": \"A-102\", \"days\": 9, \"damaged\": False},\n {\"order\": \"A-103\", \"days\": 3, \"damaged\": True},\n {\"order\": \"A-104\", \"days\": 4, \"damaged\": False},\n {\"order\": \"A-105\", \"days\": 12, \"damaged\": False},\n]\n\nREPORTS = [\n (\"weekly-ops-w30\", [\n (\"orders_shipped\", 5),\n (\"avg_days\", 4.2),\n (\"late_orders\", 2),\n (\"damage_rate_pct\", 20.0),\n ]),\n (\"weekly-ops-w30-corrected\", [\n (\"orders_shipped\", 5),\n (\"avg_days\", 6.0),\n (\"late_orders\", 2),\n (\"damage_rate_pct\", 20.0),\n ]),\n]\n\n# define report_gate(name, claims, export) below\n\nfor name, claims in REPORTS:\n report_gate(name, claims, EXPORT)\n","grader":{"kind":"stdout-equality","expected":"weekly-ops-w30:\n orders_shipped: VERIFIED (5)\n avg_days: MISMATCH — report 4.2, export 6.0\n late_orders: VERIFIED (2)\n damage_rate_pct: VERIFIED (20.0)\n verdict: HOLD — avg_days\nweekly-ops-w30-corrected:\n orders_shipped: VERIFIED (5)\n avg_days: VERIFIED (6.0)\n late_orders: VERIFIED (2)\n damage_rate_pct: VERIFIED (20.0)\n verdict: SEND\n","normalize":"collapse-trailing-newline"},"solution":"$38","passThreshold":0.66}],"xpTotal":24},{"slug":"the-pii-proxy","title":"The PII proxy — mask before the model, unmask for the named human","estMinutes":22,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"ops-studio/the-pii-proxy/s01-01-the-ticket-that-left-the-building","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":180,"concept":"ticket-must-not-leave-raw","body":"$39","cta":"Got it","runnable":true},{"type":"read","id":"ops-studio/the-pii-proxy/s02-02-mask-then-unmask","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":200,"concept":"mask-before-unmask-after","body":"$3a","cta":"Got it","code":"KNOWN = [\n \"ada@harbor-clinic.org\",\n \"415-555-0142\",\n \"MBR-104221\",\n]\n\nTICKET = (\n \"Member MBR-104221 at ada@harbor-clinic.org — late refill. \"\n \"Call 415-555-0142.\"\n)\nPII = [\n {\"kind\": \"member\", \"value\": \"MBR-104221\"},\n {\"kind\": \"email\", \"value\": \"ada@harbor-clinic.org\"},\n {\"kind\": \"phone\", \"value\": \"415-555-0142\"},\n]\n\nmasked = TICKET\ntoken_map = {}\nfor i, item in enumerate(PII, 1):\n token = f\"[{item['kind'].upper()}_{i}]\"\n token_map[token] = item[\"value\"]\n masked = masked.replace(item[\"value\"], token)\n\nleftover = [v for v in KNOWN if v in masked]\nprint(f\"outbound: {masked}\")\nprint(f\"leftover: {leftover if leftover else 'none'}\")\nprint(\"token map:\")\nfor token, value in token_map.items():\n print(f\" {token} -> {value}\")\n","runnable":true},{"type":"mc","id":"ops-studio/the-pii-proxy/s03-03-which-payload-may-leave","xp":2,"hint":[{"level":1,"body":"The leftover line already did the work. Find the payload whose leftover list is `none`. The other three still contain at least one value from KNOWN — raw ticket, partial mask, or a draft that was already unmasked.","cost":0},{"level":2,"body":"C. A is the paste. B tokenized the email and left the member ID and phone sitting in the clear. D is the *output* of unmask — it belongs in front of Vera, not in the vendor prompt. Only C has tokens where every known value used to be.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"catch-the-raw-payload","prompt":"Four payloads sitting on the send path. Exactly one has no known\nidentifier in it — that is the only one that may leave for the\nmodel. Which?\n","options":[{"id":"payload_c","label":"C — fully tokenized ticket"},{"id":"payload_a","label":"A — the raw ticket","explain":"Every known identifier is still in the string. This is the paste. The leftover list is the whole catalog."},{"id":"payload_b","label":"B — email tokenized, member ID and phone still raw","explain":"Half a mask is not a mask. The leftover check does not grade effort; it grades whether any known value remains. Two do."},{"id":"payload_d","label":"D — the restored draft, real values written back","explain":"This is what the named human should see after unmask. Sending it to the model is the leak with nicer prose — the leftover list is again the whole catalog."}],"answerIds":["payload_c"],"shuffle":false,"code":"KNOWN = [\n \"ada@harbor-clinic.org\",\n \"415-555-0142\",\n \"MBR-104221\",\n]\nPAYLOADS = [\n (\"A\", \"Member MBR-104221 at ada@harbor-clinic.org — late refill. Call 415-555-0142.\"),\n (\"B\", \"Member MBR-104221 at [EMAIL_2] — late refill. Call 415-555-0142.\"),\n (\"C\", \"Member [MEMBER_1] at [EMAIL_2] — late refill. Call [PHONE_3].\"),\n (\"D\", \"We'll call 415-555-0142 about MBR-104221 and email ada@harbor-clinic.org.\"),\n]\nprint(f\"known identifiers: {', '.join(KNOWN)}\")\nfor k, text in PAYLOADS:\n leftover = [v for v in KNOWN if v in text]\n print(f\"{k}. leftover={leftover if leftover else 'none'}\")\n print(f\" {text}\")\n","runnable":true},{"type":"fill","id":"ops-studio/the-pii-proxy/s04-04-fill-the-leftover-check","xp":3,"hint":[{"level":1,"body":"One `any(...)` over `KNOWN`: each known value, asked whether it appears in `outbound`. Outbound B proves you need the whole catalog — the member ID and email are gone, the phone is not, so it still prints RAW PII.","cost":0},{"level":2,"body":"`any(v in outbound for v in KNOWN)` — flag the send if any known identifier is still sitting in the outbound string. Tokens do not match those values, which is why C prints ok.","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"leftover-raw-pii-blocks-send","prompt":"The leftover check walks every known identifier and asks whether\nit still sits in the outbound string. Fill in the condition.\n\nNote outbound C: every known value has already been replaced\nwith a token — that one is fine. Tokens in the string are not\nthe problem; a real email, phone, or member ID still in it is.\n\nExpected output:\n```\nA: RAW PII\nB: RAW PII\nC: ok\n```\n","code":"KNOWN = [\n \"ada@harbor-clinic.org\",\n \"415-555-0142\",\n \"MBR-104221\",\n]\nOUTBOUNDS = [\n (\"A\", \"Member MBR-104221 at ada@harbor-clinic.org — late refill.\"),\n (\"B\", \"Member [MEMBER_1] at [EMAIL_2] — late refill. Call 415-555-0142.\"),\n (\"C\", \"Member [MEMBER_1] at [EMAIL_2] — late refill. Call [PHONE_3].\"),\n]\n\nfor label, outbound in OUTBOUNDS:\n leftover = ___\n print(f\"{label}: {'RAW PII' if leftover else 'ok'}\")\n","blanks":[{"id":"leftover-condition","accept":["any(v in outbound for v in KNOWN)","any(v in outbound for v in KNOWN)","any(item in outbound for item in KNOWN)","any(item in outbound for item in KNOWN)","any(k in outbound for k in KNOWN)","any(k in outbound for k in KNOWN)","any(value in outbound for value in KNOWN)","any(value in outbound for value in KNOWN)","any([v for v in KNOWN if v in outbound])","any([v for v in KNOWN if v in outbound])","bool([v for v in KNOWN if v in outbound])","bool([v for v in KNOWN if v in outbound])","len([v for v in KNOWN if v in outbound]) > 0","len([v for v in KNOWN if v in outbound]) > 0","len([v for v in KNOWN if v in outbound])>0","len([v for v in KNOWN if v in outbound])>0"],"caseSensitive":true,"normalize":"collapse-ws"}]},{"type":"predict","id":"ops-studio/the-pii-proxy/s05-05-predict-the-two-sides","xp":3,"hint":[{"level":1,"body":"Index 1 is member → `[MEMBER_1]`, 2 is email → `[EMAIL_2]`, 3 is phone → `[PHONE_3]`. The outbound line is the raw sentence with those three replacements. The human line is the model's draft with the same three tokens written back from the map — not the original sentence, a new one.","cost":0},{"level":2,"body":"```\nmodel sees: Call [EMAIL_2] or [PHONE_3] about [MEMBER_1].\nhuman gets: We'll email ada@harbor-clinic.org and call 415-555-0142 about MBR-104221.\n```\n\nNotice the two sides are different sentences. The model never saw the original ticket; it saw tokens and wrote a new line. Unmask does not replay the input. It restores identifiers into whatever the model returned.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"predict-mask-then-unmask","code":"PII = [\n {\"kind\": \"member\", \"value\": \"MBR-104221\"},\n {\"kind\": \"email\", \"value\": \"ada@harbor-clinic.org\"},\n {\"kind\": \"phone\", \"value\": \"415-555-0142\"},\n]\nRAW = \"Call ada@harbor-clinic.org or 415-555-0142 about MBR-104221.\"\n\nmasked = RAW\ntoken_map = {}\nfor i, item in enumerate(PII, 1):\n token = f\"[{item['kind'].upper()}_{i}]\"\n token_map[token] = item[\"value\"]\n masked = masked.replace(item[\"value\"], token)\n\ndraft = \"We'll email [EMAIL_2] and call [PHONE_3] about [MEMBER_1].\"\nrestored = draft\nfor token, value in token_map.items():\n restored = restored.replace(token, value)\n\nprint(f\"model sees: {masked}\")\nprint(f\"human gets: {restored}\")\n","prompt":"Mask walks `PII` in order and builds `[KIND_n]` tokens with `n`\nstarting at 1. The model drafts against those tokens; unmask\nwalks the map and writes the real values back. What prints?\n","grader":{"kind":"stdout-equality","expected":"model sees: Call [EMAIL_2] or [PHONE_3] about [MEMBER_1].\nhuman gets: We'll email ada@harbor-clinic.org and call 415-555-0142 about MBR-104221.\n","normalize":"collapse-trailing-newline"}},{"type":"read","id":"ops-studio/the-pii-proxy/s06-06-unmask-is-a-named-human-act","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":170,"concept":"unmask-needs-a-named-human","body":"$3b","cta":"Got it","runnable":true},{"type":"write","id":"ops-studio/the-pii-proxy/s07-07-write-the-proxy","xp":5,"hint":[{"level":1,"body":"Mask first (one loop, token `[KIND_n]`, keep the map), then leftover against `KNOWN` (not against `pii` — exc-442 is the trap: the phone is known but was never on the ticket's list), then the reviewer check (`auto` counts as no one). Only then unmask `MODEL_DRAFT[id]`. Do not unmask the original text — the model wrote a new sentence.","cost":0},{"level":2,"body":"```python\ndef run_proxy(ticket):\n masked = ticket[\"text\"]\n token_map = {}\n for i, item in enumerate(ticket[\"pii\"], 1):\n token = f\"[{item['kind'].upper()}_{i}]\"\n token_map[token] = item[\"value\"]\n masked = masked.replace(item[\"value\"], token)\n if any(v in masked for v in KNOWN):\n print(f\"{ticket['id']}: BLOCKED — raw PII\")\n return\n if not ticket.get(\"reviewer\") or ticket[\"reviewer\"] == \"auto\":\n print(f\"{ticket['id']}: BLOCKED — no named human\")\n return\n restored = MODEL_DRAFT[ticket[\"id\"]]\n for token, value in token_map.items():\n restored = restored.replace(token, value)\n print(f\"{ticket['id']}: FOR {ticket['reviewer']}: {restored}\")\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":240,"concept":"write-mask-unmask-proxy","prompt":"Write `run_proxy(ticket)` — mask, refuse, unmask. A ticket has\n`id`, `text`, `pii` (list of `{kind, value}`), and `reviewer`.\n\n1. Walk `pii` in order. Token is `[KIND_n]` with `n` starting at\n 1 (`kind` uppercased). Replace each value in the text. Keep\n a token map.\n2. If any value in `KNOWN` is still in the masked text → print\n `{id}: BLOCKED — raw PII` and return.\n3. If `reviewer` is falsy or equal to `\"auto\"` → print\n `{id}: BLOCKED — no named human` and return.\n4. Otherwise take `MODEL_DRAFT[id]`, restore every token from\n the map, and print\n `{id}: FOR {reviewer}: {restored}`.\n\nExpected output:\n```\nexc-441: FOR vera: We'll call 415-555-0142 about MBR-104221 and email ada@harbor-clinic.org.\nexc-442: BLOCKED — raw PII\nexc-443: BLOCKED — no named human\n```\n","starter":"$3c","grader":{"kind":"stdout-equality","expected":"exc-441: FOR vera: We'll call 415-555-0142 about MBR-104221 and email ada@harbor-clinic.org.\nexc-442: BLOCKED — raw PII\nexc-443: BLOCKED — no named human\n","normalize":"collapse-trailing-newline"},"solution":"$3d","hiddenTests":[]},{"type":"checkpoint","id":"ops-studio/the-pii-proxy/s08-08-checkpoint","xp":8,"hint":[{"level":1,"body":"Always print the id header and the outbound line — even on a block. Then leftover against `KNOWN` in catalog order (exc-442 keeps the phone, not the member ID). Only a clean leftover list reaches the reviewer question. Unmask `MODEL_DRAFT`, not the original text. Indent every line under the id by two spaces.","cost":0},{"level":2,"body":"$3e","cost":0}],"personalize":false,"phase":"check","estSeconds":300,"concept":"pii-proxy-gate","prompt":"$3f","starter":"$40","grader":{"kind":"stdout-equality","expected":"exc-441:\n outbound: Member [MEMBER_1] at [EMAIL_2] — late refill. Call [PHONE_3].\n leftover: none\n for vera: We'll call 415-555-0142 about MBR-104221 and email ada@harbor-clinic.org.\nexc-442:\n outbound: Member [MEMBER_1] called from 415-555-0142.\n leftover: 415-555-0142\n verdict: BLOCKED — raw PII\nexc-443:\n outbound: Email [EMAIL_1] about the missed window.\n leftover: none\n verdict: BLOCKED — no named human\n","normalize":"collapse-trailing-newline"},"solution":"$41","passThreshold":0.66}],"xpTotal":24}],"liveLessonCount":4,"liveStepCount":32,"xpTotal":96}},"chapter":"$1c:2:props:tree:detail","lesson":"$1c:2:props:tree:detail:lessons:0","step":"$1c:2:props:tree:detail:lessons:0:steps:2","stepIndex":2,"nav":{"tocNext":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"tocPrev":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"nextByPath":{"developer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"marketer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"designer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"customer-service":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"copywriter":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"data-analyst":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"project-manager":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"hr-specialist":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"operations":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3},"lawyer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":3}},"prevByPath":{"developer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"marketer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"designer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"customer-service":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"copywriter":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"data-analyst":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"project-manager":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"hr-specialist":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"operations":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1},"lawyer":{"chapterSlug":"ops-studio","lessonSlug":"the-sop-is-data","stepIndex":1}}}}]]

The SOP is data — get the process out of Vera's head without letting the model invent it — step 3 of 8