46:T422,# four industries. each one is "AI-native rebuildable" in principle. # but which one wedges in fastest with the smallest team? industries = { "A": { "name": "Boutique law firm (M&A advisory)", "volume": "low (10 deals/year)", "rules": "complex, mostly tacit", "outcome": "high-stakes, irreversible", }, "B": { "name": "Bookkeeping for SMBs", "volume": "high (300+ clients/month)", "rules": "explicit (GAAP + tax code)", "outcome": "structured, recurring", }, "C": { "name": "Public-relations agency", "volume": "medium (40 active campaigns)", "rules": "taste-driven, relationship-heavy", "outcome": "subjective, narrative", }, "D": { "name": "Investment banking (capital markets)", "volume": "low (per deal)", "rules": "regulated, judgment-dense", "outcome": "high-stakes, brand-risky", }, } for k, v in industries.items(): print(f"{k}: {v['name']:35} vol={v['volume']:20} rules={v['rules']}") 47:T20f9,# Case 4-5: Support ops + software dev — and the incumbent trap *(Illustrative scenarios, not actual case studies.)* We've walked through three rebuilds where a small AI-native team can outrun a larger incumbent. Two more, briefly, before we get to the real lesson of this chapter: **why the incumbents can't catch up**. ## Case 4: Support operations A 40-person SaaS support team for a B2B product. Tickets flow in through email, chat, intercom, and a community forum. Median resolution time is 4.5 hours. CSAT is 4.1. The support manager spends Sundays running pivot tables to figure out which agents are slow and which categories of tickets spike. AI-native version: a triage agent reads every inbound, classifies it (bug / billing / feature-request / how-to), pulls the customer account context, drafts the response, and either resolves the ticket (for clearly-resolvable cases) or routes it to the right human with a pre-drafted reply and a one-line summary. The support manager builds dashboards from structured ticket data instead of hand-coding categories. The 40-person team becomes 12 people doing higher-leverage work (writing canned responses, designing escalation logic, training the agent, talking to angry enterprise customers). Resolution time drops to under an hour. CSAT goes to 4.6. The product team gets better signal because every ticket is properly categorized. The wedge: **classification + draft response**. Once those two steps are agent-owned, the entire support workflow reorganizes around them. ## Case 5: Software development (the process-debt case) An under-credited point that's been circulating in the AI-native discourse: > Process debt hits hard in software dev. Using AI to code faster > just makes you hit the QA bottleneck faster. Until humans stop > manually running the tests, the org isn't AI-native yet. This is the most underrated thing in the AI-native discourse. Every dev team has been told "use Copilot, ship faster." They do. Code velocity goes up. Then the bottleneck moves: from code generation to code review, then to QA, then to deployment, then to incident response. The AI-native software team isn't the one that writes code with AI. It's the one whose **entire SDLC** — the pipeline from "idea" to "in prod" — has been rebuilt so agents can operate inside every step. That looks like: - **PR review**: an agent reviews every PR for the obvious things (typing, naming, test coverage, perf regressions) before the human reviewer sees it. Human reviewer only sees the 20% of PRs that need taste-level judgment. - **Test generation**: an agent generates the test cases from the PR diff + prod traffic patterns + prior bug history. Coverage goes up without the human writing tests. - **Deployment gates**: an agent runs the deployment checklist, monitors the canary, decides whether to roll back, and pages the human only when something is genuinely off. - **Incident response**: an agent collects the logs, traces, and metric anomalies for any incident and drafts the postmortem skeleton. The human writes the "what we'll change" section. A 50-person engineering org becomes a 20-person org shipping the same surface area, because the agent absorbs everything that was process tax. The 20 people stop being a code factory and become a system-design team. ## The incumbent trap Every case study above describes a 5-15-person AI-native team outperforming a 40-100-person incumbent. The obvious question is: **why don't the incumbents just do the same thing?** The answer (and the reason Isenberg's writing keeps returning to this point): **process debt**. > The hard part is that incumbents are full of old process debt. > Their data is messy. Their policies conflict. Their teams > protect turf. Their workflows were built around headcount. > Their software stack is stitched together with duct tape and > quarterly planning rituals. Their operating system assumes > humans are the default processors of information. A 200-person HVAC company can't just "go AI-native." To do it, they would need to: 1. **Clean the data.** Customer records are scattered across QuickBooks, a paper filing cabinet, three dispatchers' heads, and one tech's text messages. Cleaning that takes 6-12 months and no one wants to fund it. 2. **Write down the policies.** The owner's head IS the policy. Writing it down is months of meetings he doesn't have time for, and arguments about which version is the "real" policy. 3. **Restructure the team.** The four dispatchers + two AR clerks + two estimators don't have AI-native roles. They have to be reskilled or replaced. That's a political knife-fight. 4. **Replace the software stack.** The CRM, the dispatch whiteboard, QuickBooks, and the paper forms have to be replaced with a single integrated system that agents can read and write to. 5. **Build the eval and feedback loops.** Once agents are in the workflow, the company needs to measure their outputs, catch their mistakes, and improve the prompts. Nobody on the team knows how to do this. The new AI-native company doesn't have any of this debt. They start clean. They wrote the policies BEFORE they hired the dispatchers. They built the data model BEFORE they took the first customer. They designed the workflow assuming agents would run it. Their software stack is one app, not nine. That's the asymmetry. The new entrant doesn't have to undo anything. The incumbent has 22 years of "ask Sarah" to dismantle before they can even start. ## What this case study teaches (and why theory matters) - **The bottleneck isn't AI access — it's organizational legibility.** Every company has access to the same models. The ones who win are the ones who restructured the company so the models have something to work with. - **Process debt compounds in the wrong direction.** Every year an incumbent doesn't pay down process debt, their AI-native competitor gets a year of compounding advantage. By the time the incumbent realizes the gap, it's too late. - **The strategic question for a founder is "what's process-debt- free that I could rebuild from scratch?"** Pick an industry where the incumbents are 30+ years old, the workflow is document-heavy, and the customer pays for an OUTCOME. That's where the next decade's wealth is going. This is why theory matters in this course. You can drill on Python syntax forever and not see what to BUILD with it. The reading is how you find the wedge. The code is how you execute on it. ## The one real anchor case The five industries above are illustrative. Before this lesson ends, here is the one case study in this chapter that has receipts. **Klarna's 2024-2025 customer service rebuild** is the most public, documented agent-native deployment to date. The OpenAI case study and Sebastian Siemiatkowski's May 2025 walk-back are both part of the same story: an aggressive AI-first rebuild, real productivity gains (the AI assistant handled the equivalent workload of roughly 700 full-time agents in its first month), real quality regressions (customer-facing quality dropped enough that Siemiatkowski publicly admitted the company had cut human agents too aggressively), and a public correction (Klarna began re-hiring human agents for cases the AI couldn't handle well). The Klarna story has receipts. The five rebuilds above don't. They are scenarios in the shape of how an agent-native company could go, informed by patterns visible in real deployments like Klarna and Cresta. They are not documentation of companies that have shipped these systems. > Read the Klarna section twice for every illustrative scenario you > read once. The illustrative scenarios are useful for pattern > recognition. The Klarna story is what actually happens when you > ship one. You'll see the Klarna case discussed in more detail in ch00 (how the market is shifting) and ch24 (debugging output — the specific failure modes that caused the walk-back). Both treatments name the gains AND the regressions, because the honest version of this story is the one that holds both at once. The remaining steps of this lesson are interactive. They take the case-study framework and apply it to a structured "process debt audit" you can run against a real business. By the end you'll have a function you can paste into a notebook and use on your own job or your own startup idea.48:T525,hvac_inc = { "name": "HVAC Inc", "age_years": 22, "data_scattered_across_tools": 6, "tacit_knowledge_pct": 65, "paper_processes": 2, "headcount_per_revenue_million": 8, "documented_workflows_pct": 15, } startup_co = { "name": "StartupCo", "age_years": 1, "data_scattered_across_tools": 2, "tacit_knowledge_pct": 30, "paper_processes": 0, "headcount_per_revenue_million": 4, "documented_workflows_pct": 80, } def process_debt_score(company): score = 0 if company.get("age_years", 0) > 10: score += 25 if company.get("data_scattered_across_tools", 0) >= 5: score += 20 if company.get("tacit_knowledge_pct", 0) > 50: score += 20 if company.get("paper_processes", 0) > 0: score += 15 if company.get("headcount_per_revenue_million", 0) > 7: score += 10 if company.get("documented_workflows_pct", 100) < 30: score += 10 if score <= 20: verdict = "AI-native ready" elif score <= 50: verdict = "manageable debt" elif score <= 80: verdict = "heavy debt" else: verdict = "incumbent trap" return {"score": score, "verdict": verdict} print(f"HVAC Inc: {process_debt_score(hvac_inc)}") print(f"StartupCo: {process_debt_score(startup_co)}") 49:T484,candidates = [ {"name": "Boutique law firm", "volume_per_week": 2, "rules_explicit": False, "outcome_structured": False, "incumbent_avg_age_years": 30}, {"name": "Insurance brokerage", "volume_per_week": 200, "rules_explicit": True, "outcome_structured": True, "incumbent_avg_age_years": 35}, {"name": "PR agency", "volume_per_week": 40, "rules_explicit": False, "outcome_structured": False, "incumbent_avg_age_years": 15}, {"name": "Bookkeeping for SMBs", "volume_per_week": 300, "rules_explicit": True, "outcome_structured": True, "incumbent_avg_age_years": 18}, {"name": "VC investing", "volume_per_week": 1, "rules_explicit": False, "outcome_structured": False, "incumbent_avg_age_years": 25}, ] def score_candidate(c): s = 0 if c["volume_per_week"] >= 100: s += 30 if c["rules_explicit"]: s += 25 if c["outcome_structured"]: s += 25 if c["incumbent_avg_age_years"] >= 20: s += 20 return s def pick_industry(candidates): return max(candidates, key=score_candidate)["name"] print(f"best: {pick_industry(candidates)}") 4a:T647,# A runbook a junior can execute lives in the repo Layer 4 is why this lesson exists. The harness can edit files, run commands, and ship a dispatch change. Anything that can change the world needs a way to undo it — written down next to the harness, not in a wiki tab someone has to interpret while the batch is still failing. Run the editor. Two artifacts, same nickname. The wiki line is a page. The dict is a runbook: a signal to read, a threshold that means rollback, a named human if the decision needs one, and a path in the repo for the record. That is the whole job. Detect. Decide rollback versus continue. Name a human if you cannot execute the decision alone. Write the record back. A junior on the on-call seat should be able to follow it from the checkout without inventing a process. This is the ops role's failure path — one seat, one workflow. Not a company-wide rebuild. ## What this lesson is NOT - Not a war story. The numbers are quiet on purpose. - Not a page that says "stay calm" and then stops. - Not a hiring pitch. What it IS: the failure path as data, the same way lesson 03 made the happy path a spec. Rollback is the point. If the runbook cannot tell a junior when to undo the last ship, it is still a wiki page. ## What you'll build `execute_runbook(signals, runbook)` — read the signal, return rollback / continue / escalate, attach the named human only when escalate fires, and produce the writeback record a later checkout can grep. The runbook you write is the one you would want sitting in `incidents/` the next time the nightly batch misses its threshold.4b:T7f6,# Detect, decide, name a human, write it back Four fields. That is the runbook. 1. **Detect.** Which signal, and from where. Here it is `error_rate` on the nightly harness batch — the same loop you built in lesson 01, now missing its threshold. 2. **Decide.** Rollback if the signal is at or above the line. Continue if it is under. No third mood called "wait and see" unless the runbook wrote that mood down as continue. 3. **Named human.** A seat, not a channel. `release-owner` is a person who can unblock a rollback. "The team" is not a page target. 4. **Writeback.** A path in the repo. The decision becomes a line in `incidents/log.jsonl` so the next checkout can see what ran. Slack is not the record. Run the editor. Three quiet incidents, one rule. | id | error_rate | rollback_blocked | decision | |---|---|---|---| | INC-041 | 0.12 | no | **rollback** | | INC-042 | 0.03 | no | **continue** | | INC-043 | 0.15 | yes | **escalate** | INC-043 is the only one that names a human. The signal says rollback, but rollback cannot run (last-good tag missing, lock held, whatever the runbook marked as blocked). The junior does not invent a workaround. They page `release-owner` and write that down. ## Why rollback is the point Continue is allowed. It is just not the default once the signal crosses the line. The expensive mistake at 3am is shipping past a threshold because the wiki said "use your judgment." Judgment is what you do when the runbook has no rule. If the rule exists, you execute it. Escalate is not a vibe. It is the branch where the written action cannot run, so a named human has to. ## What the next steps drill - Predicting the three decisions from the table (step 4) - Filling the threshold comparison from the runbook (step 5) - Replacing a wiki string with the four fields (step 6) - Replacing "the team" with the named human (step 7) - Writing `execute_runbook` so the writeback record falls out of the decision (step 8) - Running three incidents through it (step 9)4c:T451,RUNBOOK = { "detect": {"signal": "error_rate", "rollback_if_at_least": 0.08}, "named_human": "release-owner", "writeback": {"path": "incidents/log.jsonl"}, } incidents = [ {"id": "INC-041", "error_rate": 0.12, "rollback_blocked": False}, {"id": "INC-042", "error_rate": 0.03, "rollback_blocked": False}, {"id": "INC-043", "error_rate": 0.15, "rollback_blocked": True}, ] def execute_runbook(signals, runbook): detect = runbook["detect"] if signals.get("rollback_blocked"): decision = "escalate" elif signals[detect["signal"]] >= detect["rollback_if_at_least"]: decision = "rollback" else: decision = "continue" actor = runbook["named_human"] if decision == "escalate" else "runbook" return { "id": signals["id"], "decision": decision, "signal": signals[detect["signal"]], "actor": actor, "artifact": runbook["writeback"]["path"], } for inc in incidents: rec = execute_runbook(inc, RUNBOOK) print(f"{rec['id']} {rec['decision']} actor={rec['actor']} artifact={rec['artifact']}") 28:["$","$L2b",null,{"tree":{"toc":{"version":"0.1.0","generatedAt":"1970-01-01T00:00:00.000Z","chapters":[{"number":0,"slug":"before-you-build","title":"from worker to builder","blurb":"stop doing repeated work by hand. this chapter is the map: the builder loop, old workplace deliverables turned into site artifacts, and the reusable tool card. if you are starting here, those moves are the lesson. if you already ran a studio, you already used the loop.","status":"live","lessonCount":10,"stepCount":14,"liveLessonCount":10,"liveStepCount":14,"estMinutes":74,"xpTotal":36},{"number":1,"slug":"read-the-script","title":"read the script — your first code-reading lab","blurb":"ai hands you small scripts. this lab covers the three reads that matter: what the names hold, whether the function actually returns, and which branch of the if-chain really fires. if you are starting here, this labels the reads. if you already ran a studio, you already wrote these.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":21,"xpTotal":78},{"number":2,"slug":"read-the-data-shapes","title":"read the data shapes — the json ai just dumped","blurb":"every api response, config file, and llm reply is lists and dicts nested in a tree. read the shape on sight, walk it with a loop, and catch the wrong-layer bug ai loves to ship.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":23,"xpTotal":80},{"number":3,"slug":"read-the-failure","title":"read the failure — cursor wrote this and crashed","blurb":"when python crashes, it tells you exactly what happened and where. most non-engineers panic at the wall of text — and ai 'fixes' the crash by hiding it. learn to read the failure, then catch only what's worth catching.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":24,"xpTotal":83},{"number":4,"slug":"read-the-pipeline","title":"read the pipeline — files in, api calls out","blurb":"every real script reads a file or calls an api — usually both. learn the with-block, the status-code families, and the retry rules: the seatbelts ai forgets to put on.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":26,"xpTotal":83},{"number":7,"slug":"mutation-and-state","title":"mutation — why your code mysteriously breaks","blurb":"when a list inside a function changes the list outside the function, that's mutation. ai does this constantly without flagging it, and it's the bug class that takes the longest to find.","status":"live","lessonCount":2,"stepCount":9,"liveLessonCount":2,"liveStepCount":9,"estMinutes":10,"xpTotal":32},{"number":13,"slug":"llm-apis","title":"llm apis — talking to claude and openai","blurb":"every ai feature you ship eventually calls a model api. learn the messages pattern, how to read the response, and the four lines ai writes every single time.","status":"live","lessonCount":4,"stepCount":27,"liveLessonCount":4,"liveStepCount":27,"estMinutes":45,"xpTotal":87},{"number":14,"slug":"structured-output","title":"structured output — making the model return json","blurb":"free-form text breaks every pipeline. learn the schema-first pattern ai uses to get reliable json back, validate it with pydantic, and catch the model's lies before they hit prod.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":38,"xpTotal":80},{"number":15,"slug":"mcp","title":"mcp — the model context protocol","blurb":"mcp is the new standard for plugging tools and data sources into ai agents. learn what an mcp server actually is, how claude code lists tools, and why this is replacing one-off integrations everywhere.","status":"live","lessonCount":3,"stepCount":26,"liveLessonCount":3,"liveStepCount":26,"estMinutes":40,"xpTotal":85},{"number":16,"slug":"agent-loops","title":"agent loops — tool_use and the request/tool/respond cycle","blurb":"an agent isn't magic. it's a while loop. learn the actual cycle claude code, cursor, and every other agent uses: model returns tool_use, you run the tool, you send the result back, repeat until end_turn.","status":"live","lessonCount":6,"stepCount":48,"liveLessonCount":6,"liveStepCount":48,"estMinutes":71,"xpTotal":154},{"number":17,"slug":"git-and-github","title":"git and github cli — the ai builder's actual workflow","blurb":"cursor and claude code commit on your behalf. reading those commits — and undoing the bad ones — is your job. learn the three-state model, the commands you'll run every day, and what `gh` does that `git` can't.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":36,"xpTotal":79},{"number":18,"slug":"secrets-and-env","title":"secrets — .env, api keys, and what not to commit","blurb":"ai ships keys to github all the time. learn the .env pattern, why os.getenv is non-negotiable, what to do when a key leaks, and the gitignore lines you need on day one.","status":"live","lessonCount":2,"stepCount":16,"liveLessonCount":2,"liveStepCount":16,"estMinutes":16,"xpTotal":58},{"number":19,"slug":"prompting","title":"prompting ai coding tools","blurb":"an AI coding tool writes and edits with you — Cursor is one brand, Claude Code is another. the difference between a one-shot session and a four-hour spiral is almost always the first prompt.","status":"live","lessonCount":5,"stepCount":36,"liveLessonCount":5,"liveStepCount":36,"estMinutes":59,"xpTotal":118},{"number":20,"slug":"agent-traces","title":"reading agent traces and telemetry","blurb":"when an agent fails, the trace tells you exactly where. learn to read tool calls, tool results, and stop reasons — the json breadcrumbs every agent leaves behind.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":30,"xpTotal":88},{"number":21,"slug":"evals","title":"test the agent before you trust it","blurb":"same input, check the output, pass or fail. don't ship on a demo that felt right.","status":"live","lessonCount":5,"stepCount":34,"liveLessonCount":5,"liveStepCount":34,"estMinutes":57,"xpTotal":112},{"number":22,"slug":"context-and-retrieval","title":"context and retrieval — feeding the model real data","blurb":"rag without the overengineering. chunking, embeddings, vector search, and the small set of patterns that make a model answer from your data instead of its training set.","status":"live","lessonCount":5,"stepCount":44,"liveLessonCount":5,"liveStepCount":44,"estMinutes":75,"xpTotal":144},{"number":23,"slug":"production-tradeoffs","title":"production tradeoffs — cost, latency, quality","blurb":"the three numbers every shipped llm feature lives or dies by. token math, caching, streaming, batching, and the small set of decisions that move the product more than a model swap ever will.","status":"live","lessonCount":4,"stepCount":33,"liveLessonCount":4,"liveStepCount":33,"estMinutes":57,"xpTotal":111},{"number":24,"slug":"debugging-output","title":"debugging broken ai output","blurb":"when the model ships a claim nobody sourced. the methodology for narrowing down what went wrong, the four most-common breakage classes, and the discipline that separates 'we shipped a fix' from 'we blamed the model and shrugged'.","status":"live","lessonCount":4,"stepCount":34,"liveLessonCount":4,"liveStepCount":34,"estMinutes":61,"xpTotal":112},{"number":25,"slug":"capstone","title":"path checkpoint — ship the CLI agent","blurb":"wire it all together. the prompt, the call, the validation, the trace, the eval, the MCP tool. a path checkpoint: the smallest end-to-end llm feature you could ship to a real user. on the developer route, harnesses, image, video, and production tooling still follow. retrieval and prompt-cache cost work live in later chapters — extend this build with them when you scale past the demo input set.","status":"live","lessonCount":7,"stepCount":62,"liveLessonCount":7,"liveStepCount":62,"estMinutes":100,"xpTotal":219},{"number":26,"slug":"agent-harnesses","title":"agent harnesses — the layer between you and the raw api","blurb":"claude code, cursor, aider, codex cli — they're all the same four layers wrapped around the same model api. learn what those layers are, what each adds, and what you'd build yourself if you had to. then you build a small harness you can run and break. when the harness changes the world, the rollback lives in the repo — a runbook a junior can execute at 3am.","status":"live","lessonCount":6,"stepCount":46,"liveLessonCount":6,"liveStepCount":46,"estMinutes":87,"xpTotal":151},{"number":27,"slug":"ai-image-generation","title":"ai image generation — from prompt to production asset","blurb":"the 2026 image model landscape, the prompts that work, and the pipeline that turns one good idea into a hundred ready-to-ship images. nano banana 2 for volume, nano banana pro for fidelity, flux, midjourney, ideogram, gpt-image-2. when each wins and what they cost.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":60,"xpTotal":67},{"number":28,"slug":"ai-video-generation","title":"ai video generation — omni, veo 3.1, sora, and the second wave","blurb":"video is the hardest content type to generate, the most expensive, and the most strategically interesting. learn the 2026 model lineup, the camera-control patterns that separate slop from craft, and the cost math that decides whether your idea is viable.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":50,"xpTotal":61},{"number":29,"slug":"programmatic-design","title":"programmatic design — code-driven video and the new design pipeline","blurb":"ai generates raw assets; code stitches them into something shippable. hyperframes, remotion, claude design — when each tool wins, how they combine, and the data-driven workflows that turn one template into a hundred videos.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":46,"xpTotal":66},{"number":30,"slug":"harness-engineering","title":"harness engineering — the discipline behind the model","blurb":"every coding agent is a model plus a harness. the model is bought; the harness is engineered. learn the craft: how to ratchet rules from failures, fight context rot, design long-horizon loops, wire hooks as enforcement, and read the haas shift that's reshaping what you build vs buy.","status":"live","lessonCount":6,"stepCount":52,"liveLessonCount":6,"liveStepCount":52,"estMinutes":126,"xpTotal":143},{"number":31,"slug":"intro-to-terminal","title":"intro to terminal — the text way to drive your computer","blurb":"the terminal is a text box: type a name, press enter, the computer does it. by the end you can move around your files, make folders, and read files without the mouse. every later tool chapter assumes this.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":26,"xpTotal":50},{"number":32,"slug":"intro-to-claude-cli","title":"intro to claude cli — the door to real building","blurb":"you've used claude in a chat window. the claude cli is the same model with its hands on your actual files. this chapter installs it, signs you in, and runs your first real command. by the end you've watched an ai read, plan, and change things on your machine, and you know when to reach for the cli instead of the chat box.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":48},{"number":33,"slug":"intro-to-codex-cli","title":"intro to openai codex cli — a second tool in the kit","blurb":"you know the claude cli. the openai codex cli does the same job, an ai working in your terminal on your real files, with a different company behind it. this chapter installs it, signs you in, and runs your first command. most of what you already know carries straight over, so this chapter is mostly about what is different and when to reach for which.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":45},{"number":34,"slug":"team-skills","title":"claude skills for teams — playbooks your whole team shares","blurb":"a claude skill is a packaged set of instructions — your team's playbook — that claude loads when it's relevant, so nobody has to re-explain it. this chapter is for people who manage teams. it covers what a skill is, how a team shares and provisions skills, real examples for hr, legal, and ops work, when a skill beats a one-off prompt, and the governance you need before any skill touches real work.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":41,"xpTotal":60},{"number":35,"slug":"dataframes-numpy-pandas","title":"dataframes with numpy and pandas","blurb":"tables are the working surface of applied ml. learn rows, columns, missing values, joins, aggregates, and the dataframe habits ai-generated notebooks assume.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":43,"xpTotal":156},{"number":36,"slug":"sql-for-ml-datasets","title":"sql for ml datasets","blurb":"most training data starts in a database. learn the select, join, filter, aggregate, and leakage traps that decide whether a model is learning signal or nonsense.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":28,"xpTotal":104},{"number":37,"slug":"dataset-pipelines","title":"dataset formats, ingestion, and validation pipelines","blurb":"a dataset is a product surface. build ingestion, validation, partitions, manifests, and checkpoints so the next run is not a mystery.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":38,"slug":"ml-math-and-stats","title":"ml math and statistics that actually show up","blurb":"vectors, probability, distributions, correlation, and uncertainty are not trivia. they are how you read model behavior without worshipping it.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":39,"slug":"supervised-learning-workflows","title":"supervised learning workflows","blurb":"labels, splits, baselines, training, prediction, and evaluation. the supervised workflow is the first complete model loop.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":40,"xpTotal":130},{"number":40,"slug":"unsupervised-learning-and-embeddings","title":"unsupervised learning, embeddings, and recommenders","blurb":"not every useful model has labels. cluster, compare, retrieve, and recommend by turning examples into useful neighborhoods.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":32,"xpTotal":104},{"number":41,"slug":"metrics-and-error-analysis","title":"metrics, slices, and error analysis","blurb":"accuracy is a blunt instrument. learn confusion matrices, precision, recall, thresholds, slices, and failure notes so model quality has evidence.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":42,"slug":"pytorch-tensors-and-autograd","title":"pytorch tensors and autograd","blurb":"read tensor code without flinching. tensors, shapes, broadcasting, gradients, and autograd are the grammar of modern deep learning scripts.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":30,"xpTotal":104},{"number":43,"slug":"training-loops-and-optimizers","title":"training loops, backprop, optimizers, and schedulers","blurb":"the training loop is where models change. learn loss, gradients, optimizer steps, schedules, checkpoints, and the bugs ai ships there.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":44,"slug":"deep-learning-architectures","title":"cnns, transformers, and useful llm internals","blurb":"architecture literacy for builders: convolution, attention, tokens, decoding, kv cache, quantization, and what those choices do to cost and behavior.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":45,"slug":"feature-experiments-registries","title":"feature pipelines, experiment tracking, and registries","blurb":"features, runs, configs, artifacts, and registries are how ml work becomes repeatable instead of a lucky notebook.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":46,"slug":"model-serving-and-mlops","title":"model serving, ci/cd, and mlops","blurb":"serving a model means handling inputs, versions, routes, batch jobs, ci gates, rollback, and production failures deliberately.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":47,"slug":"monitoring-cloud-portfolio","title":"monitoring, drift, cloud scale, and portfolio launch","blurb":"the last mile: logs, drift, alerts, retraining decisions, cloud cost, gpu constraints, architecture docs, demos, and role stories.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":48,"slug":"graphic-design-studio","title":"graphic design studio — prompt, kit, and fifty on-brand assets","blurb":"how a working designer runs image models: prompts written like creative briefs, a brand bible that travels as structured data — with the design tokens that carry it onto screens — and the seeds, reference locks, and QA filters that make fifty generated assets look like one studio shipped them.","status":"live","lessonCount":3,"stepCount":27,"liveLessonCount":3,"liveStepCount":27,"estMinutes":73,"xpTotal":82},{"number":49,"slug":"campaign-studio","title":"campaign studio — one campaign, end to end, with the gate on","blurb":"brief to voice-locked copy variants to batch plan to calendar to the numbers pass. the whole campaign as data your scripts can check — and an approval gate nothing skips. built for the person who owns the send button.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":59,"xpTotal":69},{"number":50,"slug":"support-studio","title":"support studio — the reply drafter you can trust","blurb":"how a working support agent runs AI: a policy pack the drafter can't talk around, a banned-promises linter that holds every draft, golden eval sets that make each macro earn autonomy one intent at a time, and the escalation line where money, law, and anger stay human.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":61,"xpTotal":71},{"number":51,"slug":"copy-studio","title":"copy studio — voice from evidence, variants at scale, claims that clear","blurb":"a client's voice measured from their own published copy, variant batches that pass the voice gate or die with a named reason, and a claim check that blocks the ship button while any fact is unsourced. corrections-log culture, as code.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":66,"xpTotal":72},{"number":52,"slug":"analytics-studio","title":"analytics studio — messy export to defensible memo","blurb":"how a working analyst runs AI: a cleaning log that accounts for every dropped row, sample-first checks that catch the plausible-but-wrong aggregation, a memo where every number carries the address of the cell that made it, SR 11-7-style challenge against the AI's recommendation, and the rerun test — same data, same code, same memo, or it isn't done.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":61,"xpTotal":74},{"number":53,"slug":"delivery-studio","title":"delivery studio — direct the build without becoming the bottleneck","blurb":"how a working delivery lead runs AI builders: specs with acceptance checks written before the build, a review that runs the checks instead of admiring the demo, status roll-ups computed from acceptance records instead of standup vibes, the not-ready call made on evidence you can read out loud to a steering committee, and an hours-and-dollars receipt — before/after on a named function, finance's loaded rate, a named owner, the accepted thing that shipped — small enough to defend in a staff meeting.","status":"live","lessonCount":4,"stepCount":32,"liveLessonCount":4,"liveStepCount":32,"estMinutes":74,"xpTotal":96},{"number":54,"slug":"people-studio","title":"people studio — draft fast, decide human, prove everything","blurb":"how a working hr specialist runs AI: a data firewall that keeps employee records out of prompts by code instead of willpower, recruiting briefs where every claim cites a resume line, policy drafts that cannot publish without a named legal reviewer, and the adverse-action paper trail that survives a discrimination claim.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":56,"xpTotal":72},{"number":55,"slug":"ops-studio","title":"ops studio — tribal knowledge to running automation","blurb":"how a working ops lead gets the process out of one person's head and into automation that can be trusted: SOPs captured as versioned data with the invented steps hunted down, recurring checklists with an explicit auto-run vs named-approval line, exception queues instead of silent failures, a receipt against the export for every number in the weekly report, and a PII proxy that masks member identifiers before the model and restores them only for the named human who sends.","status":"live","lessonCount":4,"stepCount":32,"liveLessonCount":4,"liveStepCount":32,"estMinutes":80,"xpTotal":96},{"number":56,"slug":"legal-studio","title":"legal studio — the cite-check gate and the log that survives","blurb":"how a working lawyer runs AI: a citation-verification drill where every authority gets opened at the source (the fabricated ones always look right), clause extraction that's worthless without a pinpoint, privilege rules written as a routing table instead of remembered under deadline, and the review log you'd want in the record when opposing counsel moves for sanctions.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":60,"xpTotal":69},{"number":57,"slug":"agent-studio","title":"agent studio — ship one agent feature before you stand up","blurb":"one sitting, one agent feature, end to end: a tool schema that makes the dangerous call a validation error, a loop that meters its own spend, golden cases wired before the router works, a trace you debug from instead of vibes, and the ratchet that turns tonight's incident into tomorrow's eval. practice at production tempo.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":66,"xpTotal":73}]},"accessToc":{"chapters":[{"slug":"before-you-build","lessons":[{"slug":"from-worker-to-builder","steps":[{"id":"before-you-build/from-worker-to-builder/s01-01-intro"}]},{"slug":"upgrade-dull-artifacts-into-interactive-sites","steps":[{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s01-01-intro"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s02-02-build-brief"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s03-03-your-artifact-brief"}]},{"slug":"why-building-matters-now","steps":[{"id":"before-you-build/why-building-matters-now/s01-01-intro"}]},{"slug":"the-builder-loop","steps":[{"id":"before-you-build/the-builder-loop/s01-01-intro"}]},{"slug":"your-first-builder-brief","steps":[{"id":"before-you-build/your-first-builder-brief/s01-01-intro"}]},{"slug":"what-the-ai-is-doing","steps":[{"id":"before-you-build/what-the-ai-is-doing/s01-01-intro"}]},{"slug":"build-your-first-reusable-tool","steps":[{"id":"before-you-build/build-your-first-reusable-tool/s01-01-intro"}]},{"slug":"from-prompt-to-project","steps":[{"id":"before-you-build/from-prompt-to-project/s01-01-intro"}]},{"slug":"why-code-appears-later","steps":[{"id":"before-you-build/why-code-appears-later/s01-01-intro"}]},{"slug":"use-it-again","steps":[{"id":"before-you-build/use-it-again/s01-01-intro"},{"id":"before-you-build/use-it-again/s02-02-which-fix-compounds"},{"id":"before-you-build/use-it-again/s03-03-save-version-two"}]}]},{"slug":"read-the-script","lessons":[{"slug":"names-and-values","steps":[{"id":"read-the-script/names-and-values/s01-01-intro"},{"id":"read-the-script/names-and-values/s02-02-spot-the-type"},{"id":"read-the-script/names-and-values/s03-03-assignment"},{"id":"read-the-script/names-and-values/s04-04-predict-the-score"},{"id":"read-the-script/names-and-values/s05-05-fill-the-name"},{"id":"read-the-script/names-and-values/s06-06-fix-the-typo"},{"id":"read-the-script/names-and-values/s07-07-write-greeting"},{"id":"read-the-script/names-and-values/s08-08-checkpoint"}]},{"slug":"functions-and-the-hallucinated-return","steps":[{"id":"read-the-script/functions-and-the-hallucinated-return/s01-01-intro"},{"id":"read-the-script/functions-and-the-hallucinated-return/s02-02-which-returns"},{"id":"read-the-script/functions-and-the-hallucinated-return/s03-03-return-vs-print"},{"id":"read-the-script/functions-and-the-hallucinated-return/s04-04-predict-the-result"},{"id":"read-the-script/functions-and-the-hallucinated-return/s05-05-fix-the-missing-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s06-06-fix-print-vs-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s07-07-write-square"},{"id":"read-the-script/functions-and-the-hallucinated-return/s08-08-checkpoint"}]},{"slug":"decisions-if-elif","steps":[{"id":"read-the-script/decisions-if-elif/s01-01-intro"},{"id":"read-the-script/decisions-if-elif/s02-02-truthiness-quiz"},{"id":"read-the-script/decisions-if-elif/s03-03-eq-vs-is"},{"id":"read-the-script/decisions-if-elif/s04-04-predict-the-empty-list"},{"id":"read-the-script/decisions-if-elif/s05-05-fix-the-zero-bug"},{"id":"read-the-script/decisions-if-elif/s06-06-fix-the-elif-order"},{"id":"read-the-script/decisions-if-elif/s07-07-checkpoint"},{"id":"read-the-script/decisions-if-elif/s08-08-going-deeper"}]}]},{"slug":"read-the-data-shapes","lessons":[{"slug":"the-shapes-ai-dumps-on-you","steps":[{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s01-01-intro"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s02-02-which-is-which"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s03-03-nested-paths"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s04-04-predict-the-deep-key"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s05-05-fill-the-index"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s06-06-fix-the-keyerror"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s07-07-fix-the-list-vs-dict"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s08-08-checkpoint"}]},{"slug":"walking-the-shape","steps":[{"id":"read-the-data-shapes/walking-the-shape/s01-01-intro"},{"id":"read-the-data-shapes/walking-the-shape/s02-02-which-loop-shape"},{"id":"read-the-data-shapes/walking-the-shape/s03-03-walking-the-shape"},{"id":"read-the-data-shapes/walking-the-shape/s04-04-predict-the-list-loop"},{"id":"read-the-data-shapes/walking-the-shape/s05-05-fill-the-loop-variable"},{"id":"read-the-data-shapes/walking-the-shape/s06-06-fix-the-dict-loop"},{"id":"read-the-data-shapes/walking-the-shape/s07-07-write-sum-loop"},{"id":"read-the-data-shapes/walking-the-shape/s08-08-checkpoint"}]},{"slug":"comprehensions-and-the-wrong-layer-bug","steps":[{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s01-01-intro"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s02-02-which-comp-shape"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s03-03-with-filter"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s04-04-predict-the-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s05-05-fix-the-wrong-layer"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s06-06-write-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s07-07-checkpoint"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s08-08-going-deeper"}]}]},{"slug":"read-the-failure","lessons":[{"slug":"read-bottom-up","steps":[{"id":"read-the-failure/read-bottom-up/s01-01-anatomy"},{"id":"read-the-failure/read-bottom-up/s02-02-which-error"},{"id":"read-the-failure/read-bottom-up/s03-03-the-five-fingerprints"},{"id":"read-the-failure/read-bottom-up/s04-04-predict-the-class"},{"id":"read-the-failure/read-bottom-up/s05-05-fix-nameerror"},{"id":"read-the-failure/read-bottom-up/s06-06-fix-attributeerror"},{"id":"read-the-failure/read-bottom-up/s07-07-write-defensive"},{"id":"read-the-failure/read-bottom-up/s08-08-checkpoint"}]},{"slug":"try-except-what-ai-misuses","steps":[{"id":"read-the-failure/try-except-what-ai-misuses/s01-01-intro"},{"id":"read-the-failure/try-except-what-ai-misuses/s02-02-which-block-runs"},{"id":"read-the-failure/try-except-what-ai-misuses/s03-03-naming-the-error"},{"id":"read-the-failure/try-except-what-ai-misuses/s04-04-predict-the-branch"},{"id":"read-the-failure/try-except-what-ai-misuses/s05-05-fill-the-keyword"},{"id":"read-the-failure/try-except-what-ai-misuses/s06-06-fix-the-bare-except"},{"id":"read-the-failure/try-except-what-ai-misuses/s07-07-write-safe-lookup"},{"id":"read-the-failure/try-except-what-ai-misuses/s08-08-checkpoint"}]},{"slug":"fail-loudly","steps":[{"id":"read-the-failure/fail-loudly/s01-01-intro"},{"id":"read-the-failure/fail-loudly/s02-02-which-raise-fires"},{"id":"read-the-failure/fail-loudly/s03-03-predict-the-raise"},{"id":"read-the-failure/fail-loudly/s04-04-fill-the-raise-keyword"},{"id":"read-the-failure/fail-loudly/s05-05-fix-the-silent-swallow"},{"id":"read-the-failure/fail-loudly/s06-06-write-validate-age"},{"id":"read-the-failure/fail-loudly/s07-07-checkpoint"},{"id":"read-the-failure/fail-loudly/s08-08-going-deeper"}]}]},{"slug":"read-the-pipeline","lessons":[{"slug":"files-without-fear","steps":[{"id":"read-the-pipeline/files-without-fear/s01-01-intro"},{"id":"read-the-pipeline/files-without-fear/s02-02-which-mode"},{"id":"read-the-pipeline/files-without-fear/s03-03-pathlib-and-manifests"},{"id":"read-the-pipeline/files-without-fear/s04-04-predict-the-suffix"},{"id":"read-the-pipeline/files-without-fear/s05-05-fill-the-mode"},{"id":"read-the-pipeline/files-without-fear/s06-06-fix-the-wrong-mode"},{"id":"read-the-pipeline/files-without-fear/s07-07-write-manifest-entry"},{"id":"read-the-pipeline/files-without-fear/s08-08-checkpoint"}]},{"slug":"talking-to-apis","steps":[{"id":"read-the-pipeline/talking-to-apis/s01-01-intro"},{"id":"read-the-pipeline/talking-to-apis/s02-02-which-status-family"},{"id":"read-the-pipeline/talking-to-apis/s03-03-parsing-the-json"},{"id":"read-the-pipeline/talking-to-apis/s04-04-predict-the-key"},{"id":"read-the-pipeline/talking-to-apis/s05-05-fill-the-method"},{"id":"read-the-pipeline/talking-to-apis/s06-06-fix-the-status-check"},{"id":"read-the-pipeline/talking-to-apis/s07-07-write-classify-status"},{"id":"read-the-pipeline/talking-to-apis/s08-08-checkpoint"}]},{"slug":"when-the-pipeline-breaks","steps":[{"id":"read-the-pipeline/when-the-pipeline-breaks/s01-01-intro"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s02-02-honor-the-retry-after"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s03-03-predict-the-status-handling"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s04-04-fill-the-raise-for-status"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s05-05-fix-the-missing-check"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s06-06-fix-the-broad-retry"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s07-07-checkpoint"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s08-08-going-deeper"}]}]},{"slug":"mutation-and-state","lessons":[{"slug":"why-it-breaks","steps":[{"id":"mutation-and-state/why-it-breaks/s01-01-intro"},{"id":"mutation-and-state/why-it-breaks/s02-02-which-mutates"},{"id":"mutation-and-state/why-it-breaks/s03-03-shared-reference"},{"id":"mutation-and-state/why-it-breaks/s04-04-fix-the-shared-list"},{"id":"mutation-and-state/why-it-breaks/s05-05-checkpoint"}]},{"slug":"copy-vs-reference","steps":[{"id":"mutation-and-state/copy-vs-reference/s01-01-deep-copy"},{"id":"mutation-and-state/copy-vs-reference/s02-02-predict-the-shallow"},{"id":"mutation-and-state/copy-vs-reference/s03-03-fix-the-shallow-bug"},{"id":"mutation-and-state/copy-vs-reference/s04-04-checkpoint"}]}]},{"slug":"llm-apis","lessons":[{"slug":"before-the-api-what-the-model-is-doing","steps":[{"id":"llm-apis/before-the-api-what-the-model-is-doing/s01-01-intro"}]},{"slug":"the-messages-pattern","steps":[{"id":"llm-apis/the-messages-pattern/s01-01-intro"},{"id":"llm-apis/the-messages-pattern/s02-02-which-role-is-yours"},{"id":"llm-apis/the-messages-pattern/s03-03-reading-the-response"},{"id":"llm-apis/the-messages-pattern/s04-04-predict-the-text"},{"id":"llm-apis/the-messages-pattern/s05-05-fill-the-role"},{"id":"llm-apis/the-messages-pattern/s06-06-fix-the-message-shape"},{"id":"llm-apis/the-messages-pattern/s07-07-fix-the-response-access"},{"id":"llm-apis/the-messages-pattern/s08-08-write-ask-claude"},{"id":"llm-apis/the-messages-pattern/s09-09-checkpoint"}]},{"slug":"reading-the-response","steps":[{"id":"llm-apis/reading-the-response/s01-01-intro"},{"id":"llm-apis/reading-the-response/s02-02-which-block-do-you-read"},{"id":"llm-apis/reading-the-response/s03-03-the-five-stop-reasons"},{"id":"llm-apis/reading-the-response/s04-04-predict-the-text"},{"id":"llm-apis/reading-the-response/s05-05-fill-the-text-extract"},{"id":"llm-apis/reading-the-response/s06-06-fix-the-content-zero"},{"id":"llm-apis/reading-the-response/s07-07-fix-the-stop-assumption"},{"id":"llm-apis/reading-the-response/s08-08-write-the-extractor"},{"id":"llm-apis/reading-the-response/s09-09-checkpoint"}]},{"slug":"the-model-picker","steps":[{"id":"llm-apis/the-model-picker/s01-01-why-model-choice-is-a-product-decision"},{"id":"llm-apis/the-model-picker/s02-02-haiku-territory"},{"id":"llm-apis/the-model-picker/s03-03-which-model-here"},{"id":"llm-apis/the-model-picker/s04-04-sonnet-territory"},{"id":"llm-apis/the-model-picker/s05-05-opus-and-the-thinking-budget"},{"id":"llm-apis/the-model-picker/s06-06-pick-the-model"},{"id":"llm-apis/the-model-picker/s07-07-write-the-model-routing-memo"},{"id":"llm-apis/the-model-picker/s08-08-checkpoint"}]}]},{"slug":"structured-output","lessons":[{"slug":"getting-json-back","steps":[{"id":"structured-output/getting-json-back/s01-01-intro"},{"id":"structured-output/getting-json-back/s02-02-why-validate"},{"id":"structured-output/getting-json-back/s03-03-pydantic-models"},{"id":"structured-output/getting-json-back/s04-04-predict-the-validated"},{"id":"structured-output/getting-json-back/s05-05-fill-the-field-type"},{"id":"structured-output/getting-json-back/s06-06-fix-the-required-field"},{"id":"structured-output/getting-json-back/s07-07-fix-the-type-mismatch"},{"id":"structured-output/getting-json-back/s08-08-write-parse-extraction"},{"id":"structured-output/getting-json-back/s09-09-checkpoint"}]},{"slug":"schemas-eat-prompts","steps":[{"id":"structured-output/schemas-eat-prompts/s01-01-the-trust-boundary"},{"id":"structured-output/schemas-eat-prompts/s02-02-three-shipped-breakages"},{"id":"structured-output/schemas-eat-prompts/s03-03-which-boundary-broke"},{"id":"structured-output/schemas-eat-prompts/s04-04-schema-as-prompt-replacement"},{"id":"structured-output/schemas-eat-prompts/s05-05-where-validation-belongs"},{"id":"structured-output/schemas-eat-prompts/s06-06-the-pydantic-migration-as-war-story"},{"id":"structured-output/schemas-eat-prompts/s07-07-write-the-boundary-audit"},{"id":"structured-output/schemas-eat-prompts/s08-08-checkpoint"}]},{"slug":"dataset-schema-validation","steps":[{"id":"structured-output/dataset-schema-validation/s01-01-validate-at-the-boundary"},{"id":"structured-output/dataset-schema-validation/s02-02-the-silent-drop"},{"id":"structured-output/dataset-schema-validation/s03-03-predict-the-missing-fields"},{"id":"structured-output/dataset-schema-validation/s04-04-fill-the-membership-test"},{"id":"structured-output/dataset-schema-validation/s05-05-fix-the-range-check"},{"id":"structured-output/dataset-schema-validation/s06-06-write-validate-batch"},{"id":"structured-output/dataset-schema-validation/s07-07-claims-with-receipts"},{"id":"structured-output/dataset-schema-validation/s08-08-checkpoint"}]}]},{"slug":"mcp","lessons":[{"slug":"what-mcp-is","steps":[{"id":"mcp/what-mcp-is/s01-01-intro"},{"id":"mcp/what-mcp-is/s02-02-which-side-is-the-server"},{"id":"mcp/what-mcp-is/s03-03-the-tool-list"},{"id":"mcp/what-mcp-is/s04-04-predict-the-tool-result"},{"id":"mcp/what-mcp-is/s05-05-fill-the-tool-name"},{"id":"mcp/what-mcp-is/s06-06-fix-the-missing-argument"},{"id":"mcp/what-mcp-is/s07-07-fix-the-result-shape"},{"id":"mcp/what-mcp-is/s08-08-write-call-tool"},{"id":"mcp/what-mcp-is/s09-09-checkpoint"}]},{"slug":"writing-a-tiny-mcp-server","steps":[{"id":"mcp/writing-a-tiny-mcp-server/s01-01-intro"},{"id":"mcp/writing-a-tiny-mcp-server/s02-02-which-message-is-call-tool"},{"id":"mcp/writing-a-tiny-mcp-server/s03-03-the-tool-registry"},{"id":"mcp/writing-a-tiny-mcp-server/s04-04-predict-the-list-tools-output"},{"id":"mcp/writing-a-tiny-mcp-server/s05-05-fill-the-input-schema"},{"id":"mcp/writing-a-tiny-mcp-server/s06-06-fix-the-missing-error-response"},{"id":"mcp/writing-a-tiny-mcp-server/s07-07-fix-the-validation-gap"},{"id":"mcp/writing-a-tiny-mcp-server/s08-08-write-the-call-tool-dispatcher"},{"id":"mcp/writing-a-tiny-mcp-server/s09-09-checkpoint"}]},{"slug":"why-mcp-won","steps":[{"id":"mcp/why-mcp-won/s01-01-the-graveyard-of-tool-protocols"},{"id":"mcp/why-mcp-won/s02-02-what-openai-plugins-got-wrong"},{"id":"mcp/why-mcp-won/s03-03-which-failure-mode"},{"id":"mcp/why-mcp-won/s04-04-the-mcp-bet"},{"id":"mcp/why-mcp-won/s05-05-the-network-effect"},{"id":"mcp/why-mcp-won/s06-06-spot-the-mcp-wedge"},{"id":"mcp/why-mcp-won/s07-07-write-the-protocol-postmortem"},{"id":"mcp/why-mcp-won/s08-08-checkpoint"}]}]},{"slug":"agent-loops","lessons":[{"slug":"the-loop","steps":[{"id":"agent-loops/the-loop/s01-01-intro"},{"id":"agent-loops/the-loop/s02-02-which-stop-reason"},{"id":"agent-loops/the-loop/s03-03-tool-result-shape"},{"id":"agent-loops/the-loop/s04-04-predict-the-next-call"},{"id":"agent-loops/the-loop/s05-05-fill-the-stop-condition"},{"id":"agent-loops/the-loop/s06-06-fix-the-tool-id"},{"id":"agent-loops/the-loop/s07-07-fix-the-loop-exit"},{"id":"agent-loops/the-loop/s08-08-write-the-loop"},{"id":"agent-loops/the-loop/s09-09-checkpoint"}]},{"slug":"multi-step-tools","steps":[{"id":"agent-loops/multi-step-tools/s01-01-intro"},{"id":"agent-loops/multi-step-tools/s02-02-which-tool-runs"},{"id":"agent-loops/multi-step-tools/s03-03-tool-registry"},{"id":"agent-loops/multi-step-tools/s04-04-predict-the-chain"},{"id":"agent-loops/multi-step-tools/s05-05-fill-the-dispatch"},{"id":"agent-loops/multi-step-tools/s06-06-fix-the-missing-tool"},{"id":"agent-loops/multi-step-tools/s07-07-fix-the-parallel-ids"},{"id":"agent-loops/multi-step-tools/s08-08-write-multi-tool-agent"},{"id":"agent-loops/multi-step-tools/s09-09-checkpoint"}]},{"slug":"routing","steps":[{"id":"agent-loops/routing/s01-01-intro"},{"id":"agent-loops/routing/s02-02-when-to-route"},{"id":"agent-loops/routing/s03-03-classifier-output"},{"id":"agent-loops/routing/s04-04-predict-the-route"},{"id":"agent-loops/routing/s05-05-fill-the-dispatch"},{"id":"agent-loops/routing/s06-06-fix-the-fallback"},{"id":"agent-loops/routing/s07-07-fix-the-loose-classifier"},{"id":"agent-loops/routing/s08-08-write-the-router"},{"id":"agent-loops/routing/s09-09-checkpoint"}]},{"slug":"evaluator-optimizer","steps":[{"id":"agent-loops/evaluator-optimizer/s01-01-intro"},{"id":"agent-loops/evaluator-optimizer/s02-02-when-it-earns-its-keep"},{"id":"agent-loops/evaluator-optimizer/s03-03-judge-output-shape"},{"id":"agent-loops/evaluator-optimizer/s04-04-predict-the-iterations"},{"id":"agent-loops/evaluator-optimizer/s05-05-fill-the-exit"},{"id":"agent-loops/evaluator-optimizer/s06-06-fix-the-judge-parsing"},{"id":"agent-loops/evaluator-optimizer/s07-07-fix-the-missing-cap"},{"id":"agent-loops/evaluator-optimizer/s08-08-write-the-evaluator-optimizer"},{"id":"agent-loops/evaluator-optimizer/s09-09-checkpoint"}]},{"slug":"why-every-framework-is-thirty-lines","steps":[{"id":"agent-loops/why-every-framework-is-thirty-lines/s01-01-the-frameworks-converged"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s02-02-langgraph-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s03-03-the-work-is-a-graph"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s04-04-which-framework-fits"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s05-05-vercel-ai-sdk-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s06-06-the-five-canonical-patterns"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s07-07-when-to-stay-framework-free"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s08-08-write-the-tradeoff-memo"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s09-09-checkpoint"}]},{"slug":"the-industry-map","steps":[{"id":"agent-loops/the-industry-map/s01-01-the-five-layers"},{"id":"agent-loops/the-industry-map/s02-02-which-layer-failed"},{"id":"agent-loops/the-industry-map/s03-03-the-shape-of-the-stack"}]}]},{"slug":"git-and-github","lessons":[{"slug":"the-three-states","steps":[{"id":"git-and-github/the-three-states/s01-01-intro"},{"id":"git-and-github/the-three-states/s02-02-which-command-stages"},{"id":"git-and-github/the-three-states/s03-03-status-and-diff"},{"id":"git-and-github/the-three-states/s04-04-predict-the-status"},{"id":"git-and-github/the-three-states/s05-05-fill-the-command"},{"id":"git-and-github/the-three-states/s06-06-fix-the-staged-files"},{"id":"git-and-github/the-three-states/s07-07-fix-the-pr-command"},{"id":"git-and-github/the-three-states/s08-08-write-status-summary"},{"id":"git-and-github/the-three-states/s09-09-checkpoint"}]},{"slug":"three-git-disasters-ai-shipped","steps":[{"id":"git-and-github/three-git-disasters-ai-shipped/s01-01-why-postmortems-teach-better"},{"id":"git-and-github/three-git-disasters-ai-shipped/s02-02-uber-2016-credentials-in-repo"},{"id":"git-and-github/three-git-disasters-ai-shipped/s03-03-which-control-would-have-caught-it"},{"id":"git-and-github/three-git-disasters-ai-shipped/s04-04-the-samsung-paste"},{"id":"git-and-github/three-git-disasters-ai-shipped/s05-05-the-cursor-git-add-dot-default"},{"id":"git-and-github/three-git-disasters-ai-shipped/s06-06-spot-the-failure-mode"},{"id":"git-and-github/three-git-disasters-ai-shipped/s07-07-write-your-pre-commit-checklist"},{"id":"git-and-github/three-git-disasters-ai-shipped/s08-08-checkpoint"}]},{"slug":"github-actions-for-model-checks","steps":[{"id":"git-and-github/github-actions-for-model-checks/s01-01-intro"},{"id":"git-and-github/github-actions-for-model-checks/s02-02-green-but-wrong"},{"id":"git-and-github/github-actions-for-model-checks/s03-03-predict-the-run"},{"id":"git-and-github/github-actions-for-model-checks/s04-04-fill-the-trigger"},{"id":"git-and-github/github-actions-for-model-checks/s05-05-fix-the-eval-gate"},{"id":"git-and-github/github-actions-for-model-checks/s06-06-write-run-job"},{"id":"git-and-github/github-actions-for-model-checks/s07-07-checkpoint"}]}]},{"slug":"secrets-and-env","lessons":[{"slug":"keeping-keys-safe","steps":[{"id":"secrets-and-env/keeping-keys-safe/s01-01-intro"},{"id":"secrets-and-env/keeping-keys-safe/s02-02-which-line-leaks"},{"id":"secrets-and-env/keeping-keys-safe/s03-03-loading-env-vars"},{"id":"secrets-and-env/keeping-keys-safe/s04-04-predict-the-key"},{"id":"secrets-and-env/keeping-keys-safe/s05-05-fill-the-getenv"},{"id":"secrets-and-env/keeping-keys-safe/s06-06-fix-the-hardcoded-key"},{"id":"secrets-and-env/keeping-keys-safe/s07-07-fix-the-missing-default"},{"id":"secrets-and-env/keeping-keys-safe/s08-08-write-load-config"},{"id":"secrets-and-env/keeping-keys-safe/s09-09-checkpoint"}]},{"slug":"ml-service-secrets","steps":[{"id":"secrets-and-env/ml-service-secrets/s01-01-intro"},{"id":"secrets-and-env/ml-service-secrets/s02-02-which-log-line-leaks"},{"id":"secrets-and-env/ml-service-secrets/s03-03-predict-the-scan"},{"id":"secrets-and-env/ml-service-secrets/s04-04-fill-the-lookup"},{"id":"secrets-and-env/ml-service-secrets/s05-05-fix-the-logged-key"},{"id":"secrets-and-env/ml-service-secrets/s06-06-write-scan-logs"},{"id":"secrets-and-env/ml-service-secrets/s07-07-checkpoint"}]}]},{"slug":"prompting","lessons":[{"slug":"builder-briefs-before-prompts","steps":[{"id":"prompting/builder-briefs-before-prompts/s01-01-intro"}]},{"slug":"the-prompt-craft","steps":[{"id":"prompting/the-prompt-craft/s01-01-intro"},{"id":"prompting/the-prompt-craft/s02-02-which-prompt-is-better"},{"id":"prompting/the-prompt-craft/s03-03-context-rot"},{"id":"prompting/the-prompt-craft/s04-04-predict-the-output"},{"id":"prompting/the-prompt-craft/s05-05-fill-the-constraint"},{"id":"prompting/the-prompt-craft/s06-06-fix-the-vague-prompt"},{"id":"prompting/the-prompt-craft/s07-07-fix-the-bloated-context"},{"id":"prompting/the-prompt-craft/s08-08-write-the-prompt"},{"id":"prompting/the-prompt-craft/s09-09-checkpoint"}]},{"slug":"few-shot-and-reasoning","steps":[{"id":"prompting/few-shot-and-reasoning/s01-01-intro"},{"id":"prompting/few-shot-and-reasoning/s02-02-when-few-shot-helps"},{"id":"prompting/few-shot-and-reasoning/s03-03-the-cot-trap"},{"id":"prompting/few-shot-and-reasoning/s04-04-predict-the-format-lock"},{"id":"prompting/few-shot-and-reasoning/s05-05-fill-the-example-format"},{"id":"prompting/few-shot-and-reasoning/s06-06-fix-the-cot-on-reasoning"},{"id":"prompting/few-shot-and-reasoning/s07-07-fix-the-format-mismatch"},{"id":"prompting/few-shot-and-reasoning/s08-08-write-the-prompt-builder"},{"id":"prompting/few-shot-and-reasoning/s09-09-checkpoint"}]},{"slug":"agent-config-files","steps":[{"id":"prompting/agent-config-files/s01-01-intro"},{"id":"prompting/agent-config-files/s02-02-which-belongs-in-the-file"},{"id":"prompting/agent-config-files/s03-03-anatomy-of-a-good-file"},{"id":"prompting/agent-config-files/s04-04-predict-which-rule-fires"},{"id":"prompting/agent-config-files/s05-05-fill-the-rule-block"},{"id":"prompting/agent-config-files/s06-06-fix-the-too-vague"},{"id":"prompting/agent-config-files/s07-07-fix-the-stale-pointer"},{"id":"prompting/agent-config-files/s08-08-write-a-claude-md"},{"id":"prompting/agent-config-files/s09-09-checkpoint"}]},{"slug":"what-aged-in-prompting","steps":[{"id":"prompting/what-aged-in-prompting/s01-01-prompting-as-a-moving-target"},{"id":"prompting/what-aged-in-prompting/s02-02-cot-then-vs-now"},{"id":"prompting/what-aged-in-prompting/s03-03-which-still-works"},{"id":"prompting/what-aged-in-prompting/s04-04-the-system-prompt-arms-race"},{"id":"prompting/what-aged-in-prompting/s05-05-spot-the-stale-technique"},{"id":"prompting/what-aged-in-prompting/s06-06-the-prompt-engineer-debate"},{"id":"prompting/what-aged-in-prompting/s07-07-write-the-cheatsheet-update"},{"id":"prompting/what-aged-in-prompting/s08-08-checkpoint"}]}]},{"slug":"agent-traces","lessons":[{"slug":"reading-the-trace","steps":[{"id":"agent-traces/reading-the-trace/s01-01-intro"},{"id":"agent-traces/reading-the-trace/s02-02-which-turn-failed"},{"id":"agent-traces/reading-the-trace/s03-03-the-stop-reasons"},{"id":"agent-traces/reading-the-trace/s04-04-predict-the-stop"},{"id":"agent-traces/reading-the-trace/s05-05-fill-the-tool-name"},{"id":"agent-traces/reading-the-trace/s06-06-fix-the-loop"},{"id":"agent-traces/reading-the-trace/s07-07-fix-the-arg-shape"},{"id":"agent-traces/reading-the-trace/s08-08-write-the-summarizer"},{"id":"agent-traces/reading-the-trace/s09-09-checkpoint"}]},{"slug":"trace-driven-debugging","steps":[{"id":"agent-traces/trace-driven-debugging/s01-01-intro"},{"id":"agent-traces/trace-driven-debugging/s02-02-which-pattern-is-this"},{"id":"agent-traces/trace-driven-debugging/s03-03-the-loop-signature"},{"id":"agent-traces/trace-driven-debugging/s04-04-predict-the-call-count"},{"id":"agent-traces/trace-driven-debugging/s05-05-fill-the-detector"},{"id":"agent-traces/trace-driven-debugging/s06-06-fix-the-tool-routing"},{"id":"agent-traces/trace-driven-debugging/s07-07-fix-the-prompt-bloat"},{"id":"agent-traces/trace-driven-debugging/s08-08-write-the-trace-summary"},{"id":"agent-traces/trace-driven-debugging/s09-09-checkpoint"}]},{"slug":"prediction-ids-and-inference-logs","steps":[{"id":"agent-traces/prediction-ids-and-inference-logs/s01-01-intro"},{"id":"agent-traces/prediction-ids-and-inference-logs/s02-02-pick-the-log-line"},{"id":"agent-traces/prediction-ids-and-inference-logs/s03-03-predict-the-ids"},{"id":"agent-traces/prediction-ids-and-inference-logs/s04-04-fill-the-join-key"},{"id":"agent-traces/prediction-ids-and-inference-logs/s05-05-fix-the-join"},{"id":"agent-traces/prediction-ids-and-inference-logs/s06-06-write-the-logger"},{"id":"agent-traces/prediction-ids-and-inference-logs/s07-07-checkpoint"}]}]},{"slug":"evals","lessons":[{"slug":"check-before-you-trust","steps":[{"id":"evals/check-before-you-trust/s01-01-intro"}]},{"slug":"writing-evals","steps":[{"id":"evals/writing-evals/s01-01-intro"},{"id":"evals/writing-evals/s02-02-which-eval-passes"},{"id":"evals/writing-evals/s03-03-eval-patterns"},{"id":"evals/writing-evals/s04-04-predict-the-pass-rate"},{"id":"evals/writing-evals/s05-05-fill-the-assertion"},{"id":"evals/writing-evals/s06-06-fix-the-flaky-eval"},{"id":"evals/writing-evals/s07-07-fix-the-overstrict-eval"},{"id":"evals/writing-evals/s08-08-write-the-suite"},{"id":"evals/writing-evals/s09-09-checkpoint"}]},{"slug":"llm-as-judge","steps":[{"id":"evals/llm-as-judge/s01-01-intro"},{"id":"evals/llm-as-judge/s02-02-pairwise-or-rubric"},{"id":"evals/llm-as-judge/s03-03-the-four-biases"},{"id":"evals/llm-as-judge/s04-04-predict-the-bias"},{"id":"evals/llm-as-judge/s05-05-fill-the-rubric-call"},{"id":"evals/llm-as-judge/s06-06-fix-the-likert-trap"},{"id":"evals/llm-as-judge/s07-07-fix-the-position-bias"},{"id":"evals/llm-as-judge/s08-08-write-the-rubric-judge"},{"id":"evals/llm-as-judge/s09-09-checkpoint"}]},{"slug":"the-rise-of-evals-as-a-discipline","steps":[{"id":"evals/the-rise-of-evals-as-a-discipline/s01-01-the-prompt-engineering-era"},{"id":"evals/the-rise-of-evals-as-a-discipline/s02-02-the-eval-turn"},{"id":"evals/the-rise-of-evals-as-a-discipline/s03-03-which-company-survives-a-model-swap"},{"id":"evals/the-rise-of-evals-as-a-discipline/s04-04-case-studies"},{"id":"evals/the-rise-of-evals-as-a-discipline/s05-05-eval-first-vs-prompt-first"},{"id":"evals/the-rise-of-evals-as-a-discipline/s06-06-why-the-eval-engineer-eats-the-prompt-engineer"},{"id":"evals/the-rise-of-evals-as-a-discipline/s07-07-write-the-eval-readiness-audit"},{"id":"evals/the-rise-of-evals-as-a-discipline/s08-08-checkpoint"}]},{"slug":"model-validation-gates","steps":[{"id":"evals/model-validation-gates/s01-01-intro"},{"id":"evals/model-validation-gates/s02-02-spot-the-theater"},{"id":"evals/model-validation-gates/s03-03-predict-the-decision"},{"id":"evals/model-validation-gates/s04-04-fill-the-regression-check"},{"id":"evals/model-validation-gates/s05-05-fix-the-theater-gate"},{"id":"evals/model-validation-gates/s06-06-write-the-gate-runner"},{"id":"evals/model-validation-gates/s07-07-checkpoint"}]}]},{"slug":"context-and-retrieval","lessons":[{"slug":"chunking-that-respects-structure","steps":[{"id":"context-and-retrieval/chunking-that-respects-structure/s01-01-intro"},{"id":"context-and-retrieval/chunking-that-respects-structure/s02-02-which-content-shape-breaks"},{"id":"context-and-retrieval/chunking-that-respects-structure/s03-03-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s04-04-predict-the-shred"},{"id":"context-and-retrieval/chunking-that-respects-structure/s05-05-fill-the-separator-list"},{"id":"context-and-retrieval/chunking-that-respects-structure/s06-06-fix-the-mid-sentence-cut"},{"id":"context-and-retrieval/chunking-that-respects-structure/s07-07-fix-the-missing-overlap"},{"id":"context-and-retrieval/chunking-that-respects-structure/s08-08-write-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s09-09-checkpoint"}]},{"slug":"embedding-that-fits-the-budget","steps":[{"id":"context-and-retrieval/embedding-that-fits-the-budget/s01-01-intro"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s02-02-which-model-fits"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s03-03-the-vector-space"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s04-04-predict-the-closest"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s05-05-fill-the-cosine-formula"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s06-06-fix-the-wrong-embedding-model"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s07-07-fix-the-missing-normalization"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s08-08-write-the-similarity-ranker"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s09-09-checkpoint"}]},{"slug":"retrieval-that-finds-the-right-thing","steps":[{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s01-01-intro"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s02-02-which-result-is-good"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s03-03-the-distance-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s04-04-predict-the-top-k"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s05-05-fill-the-cutoff"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s06-06-fix-the-no-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s07-07-fix-the-dup-results"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s08-08-write-the-ranker"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s09-09-checkpoint"}]},{"slug":"rag-vs-long-context-vs-fine-tune","steps":[{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s01-01-the-three-way-fork"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s02-02-rag-when-you-need-freshness"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s03-03-which-fork-for-this-scenario"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s04-04-long-context-when-the-corpus-fits"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s05-05-fine-tune-when-style-or-format-is-the-product"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s06-06-which-fork-dies-first"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s07-07-graph-vector-or-both"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s08-08-the-rubric"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s09-09-write-the-fork-decision"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s10-10-checkpoint"}]},{"slug":"retrieval-metrics-and-vector-db-shape","steps":[{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s01-01-the-eval-set"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s02-02-filter-or-search-first"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s03-03-predict-recall-precision"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s04-04-fill-the-reciprocal"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s05-05-fix-the-late-filter"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s06-06-write-recall-at-k"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s07-07-checkpoint"}]}]},{"slug":"production-tradeoffs","lessons":[{"slug":"prompt-caching-correctly","steps":[{"id":"production-tradeoffs/prompt-caching-correctly/s01-01-intro"},{"id":"production-tradeoffs/prompt-caching-correctly/s02-02-which-piece-caches"},{"id":"production-tradeoffs/prompt-caching-correctly/s03-03-the-cache-breakpoints"},{"id":"production-tradeoffs/prompt-caching-correctly/s04-04-predict-the-savings"},{"id":"production-tradeoffs/prompt-caching-correctly/s05-05-fill-the-cache-control"},{"id":"production-tradeoffs/prompt-caching-correctly/s06-06-fix-the-cache-buster"},{"id":"production-tradeoffs/prompt-caching-correctly/s07-07-fix-the-ttl-default"},{"id":"production-tradeoffs/prompt-caching-correctly/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/prompt-caching-correctly/s09-09-checkpoint"}]},{"slug":"read-the-token-bill","steps":[{"id":"production-tradeoffs/read-the-token-bill/s01-01-intro"},{"id":"production-tradeoffs/read-the-token-bill/s02-02-which-prompt-costs-more"},{"id":"production-tradeoffs/read-the-token-bill/s03-03-the-shape-of-a-bill"},{"id":"production-tradeoffs/read-the-token-bill/s04-04-predict-the-monthly-spend"},{"id":"production-tradeoffs/read-the-token-bill/s05-05-fill-the-cost-formula"},{"id":"production-tradeoffs/read-the-token-bill/s06-06-fix-the-pricing-ratio-error"},{"id":"production-tradeoffs/read-the-token-bill/s07-07-fix-the-units-error"},{"id":"production-tradeoffs/read-the-token-bill/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/read-the-token-bill/s09-09-checkpoint"}]},{"slug":"the-model-price-war","steps":[{"id":"production-tradeoffs/the-model-price-war/s01-01-the-price-curve"},{"id":"production-tradeoffs/the-model-price-war/s02-02-three-bets-and-how-they-aged"},{"id":"production-tradeoffs/the-model-price-war/s03-03-which-startup-survived"},{"id":"production-tradeoffs/the-model-price-war/s04-04-the-incumbent-trap"},{"id":"production-tradeoffs/the-model-price-war/s05-05-features-now-viable"},{"id":"production-tradeoffs/the-model-price-war/s06-06-pricing-for-a-war-you-cant-predict"},{"id":"production-tradeoffs/the-model-price-war/s07-07-write-the-cost-model"},{"id":"production-tradeoffs/the-model-price-war/s08-08-checkpoint"}]},{"slug":"batch-realtime-costs","steps":[{"id":"production-tradeoffs/batch-realtime-costs/s01-01-who-is-waiting"},{"id":"production-tradeoffs/batch-realtime-costs/s02-02-which-lane"},{"id":"production-tradeoffs/batch-realtime-costs/s03-03-predict-the-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s04-04-fill-the-miss-rate"},{"id":"production-tradeoffs/batch-realtime-costs/s05-05-fix-the-free-discount"},{"id":"production-tradeoffs/batch-realtime-costs/s06-06-write-monthly-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s07-07-checkpoint"}]}]},{"slug":"debugging-output","lessons":[{"slug":"read-the-trace-not-the-chat","steps":[{"id":"debugging-output/read-the-trace-not-the-chat/s01-01-intro"},{"id":"debugging-output/read-the-trace-not-the-chat/s02-02-where-to-look-first"},{"id":"debugging-output/read-the-trace-not-the-chat/s03-03-the-four-failure-classes"},{"id":"debugging-output/read-the-trace-not-the-chat/s04-04-predict-the-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s05-05-fill-the-find-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s06-06-fix-the-fix-the-symptom"},{"id":"debugging-output/read-the-trace-not-the-chat/s07-07-fix-the-blame-the-model"},{"id":"debugging-output/read-the-trace-not-the-chat/s08-08-write-the-trace-summary"},{"id":"debugging-output/read-the-trace-not-the-chat/s09-09-checkpoint"}]},{"slug":"the-four-breakage-classes","steps":[{"id":"debugging-output/the-four-breakage-classes/s01-01-intro"},{"id":"debugging-output/the-four-breakage-classes/s02-02-classify-the-refund-bug"},{"id":"debugging-output/the-four-breakage-classes/s03-03-the-trace-anatomy"},{"id":"debugging-output/the-four-breakage-classes/s04-04-predict-the-class"},{"id":"debugging-output/the-four-breakage-classes/s05-05-fill-the-class-name"},{"id":"debugging-output/the-four-breakage-classes/s06-06-fix-the-misclassification"},{"id":"debugging-output/the-four-breakage-classes/s07-07-fix-the-parse-mangle"},{"id":"debugging-output/the-four-breakage-classes/s08-08-write-the-classifier"},{"id":"debugging-output/the-four-breakage-classes/s09-09-checkpoint"}]},{"slug":"five-real-postmortems","steps":[{"id":"debugging-output/five-real-postmortems/s01-01-the-postmortem-template"},{"id":"debugging-output/five-real-postmortems/s02-02-air-canada-refund-lawsuit"},{"id":"debugging-output/five-real-postmortems/s03-03-dpd-and-nyc-mycity"},{"id":"debugging-output/five-real-postmortems/s04-04-classify-the-five"},{"id":"debugging-output/five-real-postmortems/s05-05-recruiter-json-mangle"},{"id":"debugging-output/five-real-postmortems/s06-06-glean-style-retrieval-miss"},{"id":"debugging-output/five-real-postmortems/s07-07-which-fix-catches-the-most"},{"id":"debugging-output/five-real-postmortems/s08-08-write-the-postmortem"},{"id":"debugging-output/five-real-postmortems/s09-09-checkpoint"}]},{"slug":"prediction-failure-triage","steps":[{"id":"debugging-output/prediction-failure-triage/s01-01-reproduce-classify-route"},{"id":"debugging-output/prediction-failure-triage/s02-02-first-move"},{"id":"debugging-output/prediction-failure-triage/s03-03-predict-the-route"},{"id":"debugging-output/prediction-failure-triage/s04-04-fill-the-owners"},{"id":"debugging-output/prediction-failure-triage/s05-05-fix-the-paraphrase-repro"},{"id":"debugging-output/prediction-failure-triage/s06-06-write-the-router"},{"id":"debugging-output/prediction-failure-triage/s07-07-checkpoint"}]}]},{"slug":"capstone","lessons":[{"slug":"pick-a-project-that-ships","steps":[{"id":"capstone/pick-a-project-that-ships/s01-01-the-builders-graveyard"},{"id":"capstone/pick-a-project-that-ships/s02-02-which-project-survives-contact-with-reality"},{"id":"capstone/pick-a-project-that-ships/s03-03-the-three-shapes-that-ship"},{"id":"capstone/pick-a-project-that-ships/s04-04-the-wedge-checklist"},{"id":"capstone/pick-a-project-that-ships/s05-05-spot-the-missing-wedge"},{"id":"capstone/pick-a-project-that-ships/s06-06-scope-down-the-capstone"},{"id":"capstone/pick-a-project-that-ships/s07-07-checkpoint"}]},{"slug":"build-a-cli-agent","steps":[{"id":"capstone/build-a-cli-agent/s01-01-intro"},{"id":"capstone/build-a-cli-agent/s02-02-which-piece-is-missing"},{"id":"capstone/build-a-cli-agent/s03-03-the-tool-loop"},{"id":"capstone/build-a-cli-agent/s04-04-predict-the-final-answer"},{"id":"capstone/build-a-cli-agent/s05-05-fill-the-stop-check"},{"id":"capstone/build-a-cli-agent/s06-06-fix-the-tool-dispatch"},{"id":"capstone/build-a-cli-agent/s07-07-fix-the-message-append"},{"id":"capstone/build-a-cli-agent/s08-08-the-logging-pattern"},{"id":"capstone/build-a-cli-agent/s09-09-write-the-tool-runner"},{"id":"capstone/build-a-cli-agent/s10-10-fix-the-loop-exit"},{"id":"capstone/build-a-cli-agent/s11-11-write-the-agent"},{"id":"capstone/build-a-cli-agent/s12-12-checkpoint"}]},{"slug":"wire-the-real-model","steps":[{"id":"capstone/wire-the-real-model/s01-01-intro"},{"id":"capstone/wire-the-real-model/s02-02-where-the-key-lives"},{"id":"capstone/wire-the-real-model/s03-03-the-sdk-response-shape"},{"id":"capstone/wire-the-real-model/s04-04-predict-the-block-type"},{"id":"capstone/wire-the-real-model/s05-05-fill-the-key-load"},{"id":"capstone/wire-the-real-model/s06-06-fix-the-hardcoded-key"},{"id":"capstone/wire-the-real-model/s07-07-fix-the-block-access"},{"id":"capstone/wire-the-real-model/s08-08-write-the-shape-adapter"},{"id":"capstone/wire-the-real-model/s09-09-checkpoint"}]},{"slug":"validate-tool-inputs","steps":[{"id":"capstone/validate-tool-inputs/s01-01-intro"},{"id":"capstone/validate-tool-inputs/s02-02-which-input-needs-validation"},{"id":"capstone/validate-tool-inputs/s03-03-the-validator-surface"},{"id":"capstone/validate-tool-inputs/s04-04-predict-the-error"},{"id":"capstone/validate-tool-inputs/s05-05-fill-the-validate-call"},{"id":"capstone/validate-tool-inputs/s06-06-fix-the-untyped-tool"},{"id":"capstone/validate-tool-inputs/s07-07-fix-the-error-leak"},{"id":"capstone/validate-tool-inputs/s08-08-write-the-validator"},{"id":"capstone/validate-tool-inputs/s09-09-checkpoint"}]},{"slug":"add-evals-and-traces","steps":[{"id":"capstone/add-evals-and-traces/s01-01-intro"},{"id":"capstone/add-evals-and-traces/s02-02-which-eval-fits"},{"id":"capstone/add-evals-and-traces/s03-03-the-trace-shape"},{"id":"capstone/add-evals-and-traces/s04-04-predict-the-pass-rate"},{"id":"capstone/add-evals-and-traces/s05-05-fill-the-trace-append"},{"id":"capstone/add-evals-and-traces/s06-06-fix-the-trace-mutation"},{"id":"capstone/add-evals-and-traces/s07-07-fix-the-tautological-eval"},{"id":"capstone/add-evals-and-traces/s08-08-write-the-eval-runner"},{"id":"capstone/add-evals-and-traces/s09-09-checkpoint"}]},{"slug":"wire-an-mcp-tool","steps":[{"id":"capstone/wire-an-mcp-tool/s01-01-intro"},{"id":"capstone/wire-an-mcp-tool/s02-02-when-to-use-mcp"},{"id":"capstone/wire-an-mcp-tool/s03-03-the-mcp-protocol"},{"id":"capstone/wire-an-mcp-tool/s04-04-predict-the-tool-list"},{"id":"capstone/wire-an-mcp-tool/s05-05-fill-the-tools-call"},{"id":"capstone/wire-an-mcp-tool/s06-06-fix-the-result-shape"},{"id":"capstone/wire-an-mcp-tool/s07-07-fix-the-iserror-flag"},{"id":"capstone/wire-an-mcp-tool/s08-08-write-the-mcp-bridge"},{"id":"capstone/wire-an-mcp-tool/s09-09-checkpoint"}]},{"slug":"portfolio-ml-system-option","steps":[{"id":"capstone/portfolio-ml-system-option/s01-01-the-other-capstone"},{"id":"capstone/portfolio-ml-system-option/s02-02-what-reviewers-open"},{"id":"capstone/portfolio-ml-system-option/s03-03-predict-the-gate"},{"id":"capstone/portfolio-ml-system-option/s04-04-fill-the-gate"},{"id":"capstone/portfolio-ml-system-option/s05-05-fix-the-train-accuracy-gate"},{"id":"capstone/portfolio-ml-system-option/s06-06-write-the-scope-check"},{"id":"capstone/portfolio-ml-system-option/s07-07-checkpoint"}]}]},{"slug":"agent-harnesses","lessons":[{"slug":"project-workspaces-not-product-names","steps":[{"id":"agent-harnesses/project-workspaces-not-product-names/s01-01-intro"}]},{"slug":"what-a-harness-is","steps":[{"id":"agent-harnesses/what-a-harness-is/s01-01-intro"},{"id":"agent-harnesses/what-a-harness-is/s02-02-which-is-a-harness"},{"id":"agent-harnesses/what-a-harness-is/s03-03-the-four-layers"},{"id":"agent-harnesses/what-a-harness-is/s04-04-predict-the-layer"},{"id":"agent-harnesses/what-a-harness-is/s05-05-fill-the-dispatch"},{"id":"agent-harnesses/what-a-harness-is/s06-06-fix-the-no-retry"},{"id":"agent-harnesses/what-a-harness-is/s07-07-fix-the-hardcoded-registry"},{"id":"agent-harnesses/what-a-harness-is/s08-08-write-mini-harness"},{"id":"agent-harnesses/what-a-harness-is/s09-09-checkpoint"}]},{"slug":"building-a-harness","steps":[{"id":"agent-harnesses/building-a-harness/s01-01-intro"},{"id":"agent-harnesses/building-a-harness/s02-02-which-files-belong"},{"id":"agent-harnesses/building-a-harness/s03-03-the-build-order"},{"id":"agent-harnesses/building-a-harness/s04-04-predict-the-failed-layer"},{"id":"agent-harnesses/building-a-harness/s05-05-fill-the-harness-spec"},{"id":"agent-harnesses/building-a-harness/s06-06-fix-the-prompt-policy"},{"id":"agent-harnesses/building-a-harness/s07-07-fix-the-uncapped-allowlist"},{"id":"agent-harnesses/building-a-harness/s08-08-write-build-harness-run"},{"id":"agent-harnesses/building-a-harness/s09-09-checkpoint"}]},{"slug":"architecting-an-ai-native-workflow","steps":[{"id":"agent-harnesses/architecting-an-ai-native-workflow/s01-01-intro"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s02-02-which-workflow-fits"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s03-03-mapping-the-workflow"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s04-04-predict-the-readiness-score"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s05-05-fill-the-required-fields"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s06-06-fix-the-implicit-policy"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s07-07-fix-the-wrong-metric"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s08-08-write-the-workflow-validator"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s09-09-checkpoint"}]},{"slug":"five-industries-walked-through","steps":[{"id":"agent-harnesses/five-industries-walked-through/s01-01-the-case-study-method"},{"id":"agent-harnesses/five-industries-walked-through/s02-02-home-services"},{"id":"agent-harnesses/five-industries-walked-through/s03-03-which-task-belongs-to-the-agent"},{"id":"agent-harnesses/five-industries-walked-through/s04-04-insurance-brokerage"},{"id":"agent-harnesses/five-industries-walked-through/s05-05-recruiting-firm"},{"id":"agent-harnesses/five-industries-walked-through/s06-06-which-industry-wedges-fastest"},{"id":"agent-harnesses/five-industries-walked-through/s07-07-the-incumbent-trap"},{"id":"agent-harnesses/five-industries-walked-through/s08-08-write-the-process-debt-audit"},{"id":"agent-harnesses/five-industries-walked-through/s09-09-checkpoint"}]},{"slug":"incident-runbook-from-the-repo","steps":[{"id":"agent-harnesses/incident-runbook-from-the-repo/s01-01-intro"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s02-02-which-is-a-runbook"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s03-03-detect-decide-writeback"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s04-04-predict-the-decision"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s05-05-fill-the-decision-rule"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s06-06-fix-the-wiki-page"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s07-07-fix-the-unnamed-human"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s08-08-write-the-runbook"},{"id":"agent-harnesses/incident-runbook-from-the-repo/s09-09-checkpoint"}]}]},{"slug":"ai-image-generation","lessons":[{"slug":"the-image-model-landscape","steps":[{"id":"ai-image-generation/the-image-model-landscape/s01-01-the-six-families"},{"id":"ai-image-generation/the-image-model-landscape/s02-02-flux-vs-midjourney"},{"id":"ai-image-generation/the-image-model-landscape/s03-03-pick-the-model"},{"id":"ai-image-generation/the-image-model-landscape/s04-04-nano-banana-and-the-batch-economics"},{"id":"ai-image-generation/the-image-model-landscape/s05-05-text-in-image"},{"id":"ai-image-generation/the-image-model-landscape/s06-06-which-model-for-text"},{"id":"ai-image-generation/the-image-model-landscape/s07-07-the-decision-tree"},{"id":"ai-image-generation/the-image-model-landscape/s08-08-write-pick-image-model"},{"id":"ai-image-generation/the-image-model-landscape/s09-09-checkpoint"}]},{"slug":"prompting-for-real-output","steps":[{"id":"ai-image-generation/prompting-for-real-output/s01-01-why-the-girl-with-red-hair-fails"},{"id":"ai-image-generation/prompting-for-real-output/s02-02-the-eight-knobs"},{"id":"ai-image-generation/prompting-for-real-output/s03-03-fix-the-vague-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s04-04-controlnet-and-reference"},{"id":"ai-image-generation/prompting-for-real-output/s05-05-which-knob-is-missing"},{"id":"ai-image-generation/prompting-for-real-output/s06-06-failure-modes"},{"id":"ai-image-generation/prompting-for-real-output/s07-07-write-score-image-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s08-08-checkpoint"}]},{"slug":"the-image-pipeline","steps":[{"id":"ai-image-generation/the-image-pipeline/s01-01-the-six-stages"},{"id":"ai-image-generation/the-image-pipeline/s02-02-batch-and-filter"},{"id":"ai-image-generation/the-image-pipeline/s03-03-which-stage-is-broken"},{"id":"ai-image-generation/the-image-pipeline/s04-04-format-and-platform"},{"id":"ai-image-generation/the-image-pipeline/s05-05-fill-the-pipeline"},{"id":"ai-image-generation/the-image-pipeline/s06-06-write-cost-for-batch"},{"id":"ai-image-generation/the-image-pipeline/s07-07-checkpoint"}]}]},{"slug":"ai-video-generation","lessons":[{"slug":"the-video-model-lineup","steps":[{"id":"ai-video-generation/the-video-model-lineup/s01-01-the-landscape"},{"id":"ai-video-generation/the-video-model-lineup/s02-02-sora-and-veo"},{"id":"ai-video-generation/the-video-model-lineup/s03-03-higgsfield-and-camera-control"},{"id":"ai-video-generation/the-video-model-lineup/s04-04-the-second-wave"},{"id":"ai-video-generation/the-video-model-lineup/s05-05-text-vs-image-vs-video"},{"id":"ai-video-generation/the-video-model-lineup/s06-06-which-lane"},{"id":"ai-video-generation/the-video-model-lineup/s07-07-pick-the-model"},{"id":"ai-video-generation/the-video-model-lineup/s08-08-write-pick-video-model"},{"id":"ai-video-generation/the-video-model-lineup/s09-09-checkpoint"}]},{"slug":"camera-control-and-motion","steps":[{"id":"ai-video-generation/camera-control-and-motion/s01-01-the-static-camera-problem"},{"id":"ai-video-generation/camera-control-and-motion/s02-02-shot-vocabulary"},{"id":"ai-video-generation/camera-control-and-motion/s03-03-which-shot-is-which"},{"id":"ai-video-generation/camera-control-and-motion/s04-04-camera-moves"},{"id":"ai-video-generation/camera-control-and-motion/s05-05-the-i2v-keyframe-pattern"},{"id":"ai-video-generation/camera-control-and-motion/s06-06-spot-the-generic-prompt"},{"id":"ai-video-generation/camera-control-and-motion/s07-07-write-audit-shot-list"},{"id":"ai-video-generation/camera-control-and-motion/s08-08-checkpoint"}]},{"slug":"the-cost-math","steps":[{"id":"ai-video-generation/the-cost-math/s01-01-the-unit-economics"},{"id":"ai-video-generation/the-cost-math/s02-02-the-retake-rate-reality"},{"id":"ai-video-generation/the-cost-math/s03-03-which-tier-for-this-budget"},{"id":"ai-video-generation/the-cost-math/s04-04-the-60-second-promo"},{"id":"ai-video-generation/the-cost-math/s05-05-write-cost-per-minute"},{"id":"ai-video-generation/the-cost-math/s06-06-checkpoint"}]}]},{"slug":"programmatic-design","lessons":[{"slug":"why-programmatic","steps":[{"id":"programmatic-design/why-programmatic/s01-01-intro"},{"id":"programmatic-design/why-programmatic/s02-02-the-three-jobs"},{"id":"programmatic-design/why-programmatic/s03-03-which-task-is-code"},{"id":"programmatic-design/why-programmatic/s04-04-parametric-vs-one-shot"},{"id":"programmatic-design/why-programmatic/s05-05-the-classifier"},{"id":"programmatic-design/why-programmatic/s06-06-write-classify-video-task"},{"id":"programmatic-design/why-programmatic/s07-07-checkpoint"}]},{"slug":"hyperframes-and-remotion","steps":[{"id":"programmatic-design/hyperframes-and-remotion/s01-01-the-two-tools"},{"id":"programmatic-design/hyperframes-and-remotion/s02-02-hyperframes-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s03-03-hyperframes-cli"},{"id":"programmatic-design/hyperframes-and-remotion/s04-04-remotion-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s05-05-the-decision-tree"},{"id":"programmatic-design/hyperframes-and-remotion/s06-06-pick-the-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s07-07-claude-design-html"},{"id":"programmatic-design/hyperframes-and-remotion/s08-08-write-pick-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s09-09-checkpoint"}]},{"slug":"the-ai-native-design-pipeline","steps":[{"id":"programmatic-design/the-ai-native-design-pipeline/s01-01-the-seven-steps"},{"id":"programmatic-design/the-ai-native-design-pipeline/s02-02-where-ai-fits-each-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s03-03-which-model-which-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s04-04-website-to-hyperframes"},{"id":"programmatic-design/the-ai-native-design-pipeline/s05-05-the-cost-model"},{"id":"programmatic-design/the-ai-native-design-pipeline/s06-06-score-the-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s07-07-write-score-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s08-08-checkpoint"}]}]},{"slug":"harness-engineering","lessons":[{"slug":"the-harness-engineering-mindset","steps":[{"id":"harness-engineering/the-harness-engineering-mindset/s01-01-the-model-is-only-one-input"},{"id":"harness-engineering/the-harness-engineering-mindset/s02-02-the-six-pieces-of-a-harness"},{"id":"harness-engineering/the-harness-engineering-mindset/s03-03-which-piece-is-missing"},{"id":"harness-engineering/the-harness-engineering-mindset/s04-04-the-skill-issue-reframe"},{"id":"harness-engineering/the-harness-engineering-mindset/s05-05-pick-the-fix"},{"id":"harness-engineering/the-harness-engineering-mindset/s06-06-the-harness-gap"},{"id":"harness-engineering/the-harness-engineering-mindset/s07-07-write-the-harness-inventory"},{"id":"harness-engineering/the-harness-engineering-mindset/s08-08-checkpoint"}]},{"slug":"the-ratchet","steps":[{"id":"harness-engineering/the-ratchet/s01-01-failures-are-signals-not-flukes"},{"id":"harness-engineering/the-ratchet/s02-02-every-line-traces-to-a-failure"},{"id":"harness-engineering/the-ratchet/s03-03-which-rule-earned-its-line"},{"id":"harness-engineering/the-ratchet/s04-04-working-backwards-from-behavior"},{"id":"harness-engineering/the-ratchet/s05-05-the-ratchet-log"},{"id":"harness-engineering/the-ratchet/s06-06-spot-the-cruft"},{"id":"harness-engineering/the-ratchet/s07-07-write-the-ratchet-update"},{"id":"harness-engineering/the-ratchet/s08-08-audit-the-ratchet"},{"id":"harness-engineering/the-ratchet/s09-09-checkpoint"}]},{"slug":"context-engineering","steps":[{"id":"harness-engineering/context-engineering/s01-01-the-context-rot"},{"id":"harness-engineering/context-engineering/s02-02-compaction"},{"id":"harness-engineering/context-engineering/s03-03-tool-call-offloading"},{"id":"harness-engineering/context-engineering/s04-04-progressive-disclosure"},{"id":"harness-engineering/context-engineering/s05-05-which-technique-fits"},{"id":"harness-engineering/context-engineering/s06-06-the-memory-hierarchy"},{"id":"harness-engineering/context-engineering/s07-07-classify-the-data"},{"id":"harness-engineering/context-engineering/s08-08-write-the-context-budget"},{"id":"harness-engineering/context-engineering/s09-09-checkpoint"}]},{"slug":"long-horizon-execution","steps":[{"id":"harness-engineering/long-horizon-execution/s01-01-the-40-step-problem"},{"id":"harness-engineering/long-horizon-execution/s02-02-loops-intercepting-early-stops"},{"id":"harness-engineering/long-horizon-execution/s03-03-planning-as-a-separate-step"},{"id":"harness-engineering/long-horizon-execution/s04-04-splits-generation-vs-evaluation"},{"id":"harness-engineering/long-horizon-execution/s05-05-which-pattern-fixes-the-failure"},{"id":"harness-engineering/long-horizon-execution/s06-06-hooks-as-the-enforcement-layer"},{"id":"harness-engineering/long-horizon-execution/s07-07-the-type-check-backpressure-pattern"},{"id":"harness-engineering/long-horizon-execution/s08-08-write-the-hook-router"},{"id":"harness-engineering/long-horizon-execution/s09-09-write-the-completion-guard"},{"id":"harness-engineering/long-horizon-execution/s10-10-checkpoint"}]},{"slug":"the-haas-shift","steps":[{"id":"harness-engineering/the-haas-shift/s01-01-llm-apis-to-harness-apis"},{"id":"harness-engineering/the-haas-shift/s02-02-the-convergence"},{"id":"harness-engineering/the-haas-shift/s03-03-which-haas-fits"},{"id":"harness-engineering/the-haas-shift/s04-04-harnesses-dont-shrink-they-move"},{"id":"harness-engineering/the-haas-shift/s05-05-the-overfitting-feedback-loop"},{"id":"harness-engineering/the-haas-shift/s06-06-mcp-security-as-prompt-injection"},{"id":"harness-engineering/the-haas-shift/s07-07-which-component-is-debt"},{"id":"harness-engineering/the-haas-shift/s08-08-write-the-build-vs-buy"},{"id":"harness-engineering/the-haas-shift/s09-09-checkpoint"}]},{"slug":"pipeline-boundaries","steps":[{"id":"harness-engineering/pipeline-boundaries/s01-01-the-boundary-is-an-artifact"},{"id":"harness-engineering/pipeline-boundaries/s02-02-spot-the-risky-handoff"},{"id":"harness-engineering/pipeline-boundaries/s03-03-predict-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s04-04-fill-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s05-05-fix-the-overtrust"},{"id":"harness-engineering/pipeline-boundaries/s06-06-write-the-advance-check"},{"id":"harness-engineering/pipeline-boundaries/s07-07-checkpoint"}]}]},{"slug":"intro-to-terminal","lessons":[{"slug":"what-the-terminal-is","steps":[{"id":"intro-to-terminal/what-the-terminal-is/s01-01-not-a-cockpit"},{"id":"intro-to-terminal/what-the-terminal-is/s02-02-what-it-actually-is"},{"id":"intro-to-terminal/what-the-terminal-is/s03-03-open-it"},{"id":"intro-to-terminal/what-the-terminal-is/s04-04-which-app"},{"id":"intro-to-terminal/what-the-terminal-is/s05-05-the-prompt"},{"id":"intro-to-terminal/what-the-terminal-is/s06-06-what-a-command-is"},{"id":"intro-to-terminal/what-the-terminal-is/s07-07-checkpoint"}]},{"slug":"moving-around","steps":[{"id":"intro-to-terminal/moving-around/s01-01-the-filesystem-is-a-tree"},{"id":"intro-to-terminal/moving-around/s02-02-pwd-and-ls"},{"id":"intro-to-terminal/moving-around/s03-03-where-am-i"},{"id":"intro-to-terminal/moving-around/s04-04-read-this-listing"},{"id":"intro-to-terminal/moving-around/s05-05-cd"},{"id":"intro-to-terminal/moving-around/s06-06-follow-the-moves"},{"id":"intro-to-terminal/moving-around/s07-07-pick-the-command"},{"id":"intro-to-terminal/moving-around/s08-08-checkpoint"}]},{"slug":"making-things","steps":[{"id":"intro-to-terminal/making-things/s01-01-mkdir-and-touch"},{"id":"intro-to-terminal/making-things/s02-02-folder-or-file"},{"id":"intro-to-terminal/making-things/s03-03-cat-and-echo"},{"id":"intro-to-terminal/making-things/s04-04-reading-a-file"},{"id":"intro-to-terminal/making-things/s05-05-rm-carefully"},{"id":"intro-to-terminal/making-things/s06-06-the-no-undo-rule"},{"id":"intro-to-terminal/making-things/s07-07-string-it-together"},{"id":"intro-to-terminal/making-things/s08-08-checkpoint"}]}]},{"slug":"intro-to-claude-cli","lessons":[{"slug":"what-the-claude-cli-is","steps":[{"id":"intro-to-claude-cli/what-the-claude-cli-is/s01-01-the-glass-is-gone"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s02-02-chat-vs-cli"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s03-03-which-tool"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s04-04-the-agent-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s05-05-read-the-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s06-06-permission-and-limits"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-claude-cli/install-and-sign-in/s01-01-two-ways-to-install"},{"id":"intro-to-claude-cli/install-and-sign-in/s02-02-run-the-installer"},{"id":"intro-to-claude-cli/install-and-sign-in/s03-03-install-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s04-04-sign-in"},{"id":"intro-to-claude-cli/install-and-sign-in/s05-05-what-sign-in-did"},{"id":"intro-to-claude-cli/install-and-sign-in/s06-06-auth-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s07-07-account-not-api-key"},{"id":"intro-to-claude-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-claude-cli/your-first-session/s01-01-start-in-a-folder"},{"id":"intro-to-claude-cli/your-first-session/s02-02-your-first-instruction"},{"id":"intro-to-claude-cli/your-first-session/s03-03-watch-the-loop"},{"id":"intro-to-claude-cli/your-first-session/s04-04-read-what-it-did"},{"id":"intro-to-claude-cli/your-first-session/s05-05-one-shot-or-interactive"},{"id":"intro-to-claude-cli/your-first-session/s06-06-chat-or-cli-decision"},{"id":"intro-to-claude-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-claude-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"intro-to-codex-cli","lessons":[{"slug":"another-tool-same-idea","steps":[{"id":"intro-to-codex-cli/another-tool-same-idea/s01-01-why-a-second-tool"},{"id":"intro-to-codex-cli/another-tool-same-idea/s02-02-what-carries-over"},{"id":"intro-to-codex-cli/another-tool-same-idea/s03-03-what-carries-over-check"},{"id":"intro-to-codex-cli/another-tool-same-idea/s04-04-what-is-different"},{"id":"intro-to-codex-cli/another-tool-same-idea/s05-05-spot-the-difference"},{"id":"intro-to-codex-cli/another-tool-same-idea/s06-06-when-to-reach-for-which"},{"id":"intro-to-codex-cli/another-tool-same-idea/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-codex-cli/install-and-sign-in/s01-01-the-install-paths"},{"id":"intro-to-codex-cli/install-and-sign-in/s02-02-installing-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s03-03-verify-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s04-04-install-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s05-05-sign-in"},{"id":"intro-to-codex-cli/install-and-sign-in/s06-06-account-vs-api-key"},{"id":"intro-to-codex-cli/install-and-sign-in/s07-07-auth-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-codex-cli/your-first-session/s01-01-start-and-instruct"},{"id":"intro-to-codex-cli/your-first-session/s02-02-the-loop-again"},{"id":"intro-to-codex-cli/your-first-session/s03-03-read-what-it-did"},{"id":"intro-to-codex-cli/your-first-session/s04-04-usage-patterns"},{"id":"intro-to-codex-cli/your-first-session/s05-05-usage-scenario"},{"id":"intro-to-codex-cli/your-first-session/s06-06-running-both"},{"id":"intro-to-codex-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-codex-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"team-skills","lessons":[{"slug":"what-a-skill-is","steps":[{"id":"team-skills/what-a-skill-is/s01-01-the-re-explaining-problem"},{"id":"team-skills/what-a-skill-is/s02-02-a-skill-defined"},{"id":"team-skills/what-a-skill-is/s03-03-a-real-skill"},{"id":"team-skills/what-a-skill-is/s04-04-what-a-skill-is-check"},{"id":"team-skills/what-a-skill-is/s05-05-skill-vs-prompt-vs-agent"},{"id":"team-skills/what-a-skill-is/s06-06-skill-or-agent"},{"id":"team-skills/what-a-skill-is/s07-07-checkpoint"}]},{"slug":"skills-as-a-team-thing","steps":[{"id":"team-skills/skills-as-a-team-thing/s01-01-the-team-problem"},{"id":"team-skills/skills-as-a-team-thing/s02-02-org-wide-skills"},{"id":"team-skills/skills-as-a-team-thing/s03-03-sharing-and-the-directory"},{"id":"team-skills/skills-as-a-team-thing/s04-04-deployment-check"},{"id":"team-skills/skills-as-a-team-thing/s05-05-collaborating-through-skills"},{"id":"team-skills/skills-as-a-team-thing/s06-06-sharing-scenario"},{"id":"team-skills/skills-as-a-team-thing/s07-07-checkpoint"}]},{"slug":"role-playbooks","steps":[{"id":"team-skills/role-playbooks/s01-01-hr-skills"},{"id":"team-skills/role-playbooks/s02-02-legal-skills"},{"id":"team-skills/role-playbooks/s03-03-operations-skills"},{"id":"team-skills/role-playbooks/s04-04-match-the-skill"},{"id":"team-skills/role-playbooks/s05-05-skill-or-one-off"},{"id":"team-skills/role-playbooks/s06-06-skill-or-one-off-check"},{"id":"team-skills/role-playbooks/s07-07-checkpoint"}]},{"slug":"governance-and-trust","steps":[{"id":"team-skills/governance-and-trust/s01-01-a-skill-can-do-real-things"},{"id":"team-skills/governance-and-trust/s02-02-treat-it-like-software"},{"id":"team-skills/governance-and-trust/s03-03-evaluate-before-you-deploy"},{"id":"team-skills/governance-and-trust/s04-04-access-control"},{"id":"team-skills/governance-and-trust/s05-05-governance-scenario"},{"id":"team-skills/governance-and-trust/s06-06-a-rollout-checklist"},{"id":"team-skills/governance-and-trust/s07-07-checkpoint"}]}]},{"slug":"dataframes-numpy-pandas","lessons":[{"slug":"ml-package-map","steps":[{"id":"dataframes-numpy-pandas/ml-package-map/s01-01-intro"},{"id":"dataframes-numpy-pandas/ml-package-map/s02-02-match-the-tool"},{"id":"dataframes-numpy-pandas/ml-package-map/s03-03-predict-sys-modules"},{"id":"dataframes-numpy-pandas/ml-package-map/s04-04-fill-the-registry-key"},{"id":"dataframes-numpy-pandas/ml-package-map/s05-05-fix-the-swapped-map"},{"id":"dataframes-numpy-pandas/ml-package-map/s06-06-write-find-unused"},{"id":"dataframes-numpy-pandas/ml-package-map/s07-07-checkpoint"}]},{"slug":"array-and-series-shapes","steps":[{"id":"dataframes-numpy-pandas/array-and-series-shapes/s01-01-intro"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s07-07-checkpoint"}]},{"slug":"dataframe-selection-and-cleaning","steps":[{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s01-01-intro"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s07-07-checkpoint"}]},{"slug":"missing-values-and-types","steps":[{"id":"dataframes-numpy-pandas/missing-values-and-types/s01-01-intro"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s07-07-checkpoint"}]},{"slug":"groupby-joins-and-features","steps":[{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s01-01-intro"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataframe","steps":[{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s01-01-intro"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s07-07-checkpoint"}]}]},{"slug":"sql-for-ml-datasets","lessons":[{"slug":"select-filter-aggregate","steps":[{"id":"sql-for-ml-datasets/select-filter-aggregate/s01-01-intro"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s07-07-checkpoint"}]},{"slug":"joins-and-label-windows","steps":[{"id":"sql-for-ml-datasets/joins-and-label-windows/s01-01-intro"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s07-07-checkpoint"}]},{"slug":"sql-quality-checks","steps":[{"id":"sql-for-ml-datasets/sql-quality-checks/s01-01-intro"},{"id":"sql-for-ml-datasets/sql-quality-checks/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/sql-quality-checks/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/sql-quality-checks/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/sql-quality-checks/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/sql-quality-checks/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/sql-quality-checks/s07-07-checkpoint"}]},{"slug":"mission-sql-feature-query-lab","steps":[{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s01-01-intro"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s07-07-checkpoint"}]}]},{"slug":"dataset-pipelines","lessons":[{"slug":"csv-jsonl-parquet-tradeoffs","steps":[{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s01-01-intro"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s02-02-choose-the-risk"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s03-03-predict-the-checks"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s04-04-fill-the-return"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s06-06-write-count-ready"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s07-07-checkpoint"}]},{"slug":"pagination-retries-checkpoints","steps":[{"id":"dataset-pipelines/pagination-retries-checkpoints/s01-01-intro"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s04-04-fill-the-return"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s06-06-write-count-ready"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s07-07-checkpoint"}]},{"slug":"schema-drift-and-contracts","steps":[{"id":"dataset-pipelines/schema-drift-and-contracts/s01-01-intro"},{"id":"dataset-pipelines/schema-drift-and-contracts/s02-02-choose-the-risk"},{"id":"dataset-pipelines/schema-drift-and-contracts/s03-03-predict-the-checks"},{"id":"dataset-pipelines/schema-drift-and-contracts/s04-04-fill-the-return"},{"id":"dataset-pipelines/schema-drift-and-contracts/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/schema-drift-and-contracts/s06-06-write-count-ready"},{"id":"dataset-pipelines/schema-drift-and-contracts/s07-07-checkpoint"}]},{"slug":"pipeline-orchestration-shape","steps":[{"id":"dataset-pipelines/pipeline-orchestration-shape/s01-01-intro"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s04-04-fill-the-return"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s06-06-write-count-ready"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataset-pipeline","steps":[{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s01-01-intro"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s02-02-choose-the-risk"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s03-03-predict-the-checks"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s04-04-fill-the-return"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s06-06-write-count-ready"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s07-07-checkpoint"}]}]},{"slug":"ml-math-and-stats","lessons":[{"slug":"vectors-matrices-dot-products","steps":[{"id":"ml-math-and-stats/vectors-matrices-dot-products/s01-01-intro"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s04-04-fill-the-return"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s06-06-write-count-ready"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s07-07-checkpoint"}]},{"slug":"probability-and-baselines","steps":[{"id":"ml-math-and-stats/probability-and-baselines/s01-01-intro"},{"id":"ml-math-and-stats/probability-and-baselines/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/probability-and-baselines/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/probability-and-baselines/s04-04-fill-the-return"},{"id":"ml-math-and-stats/probability-and-baselines/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/probability-and-baselines/s06-06-write-count-ready"},{"id":"ml-math-and-stats/probability-and-baselines/s07-07-checkpoint"}]},{"slug":"distributions-sampling-variance","steps":[{"id":"ml-math-and-stats/distributions-sampling-variance/s01-01-intro"},{"id":"ml-math-and-stats/distributions-sampling-variance/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/distributions-sampling-variance/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/distributions-sampling-variance/s04-04-fill-the-return"},{"id":"ml-math-and-stats/distributions-sampling-variance/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/distributions-sampling-variance/s06-06-write-variance"},{"id":"ml-math-and-stats/distributions-sampling-variance/s07-07-checkpoint"}]},{"slug":"bias-variance-and-leakage","steps":[{"id":"ml-math-and-stats/bias-variance-and-leakage/s01-01-intro"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s04-04-fill-the-return"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s06-06-write-diagnose"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s07-07-checkpoint"}]},{"slug":"mission-statistical-sanity-check","steps":[{"id":"ml-math-and-stats/mission-statistical-sanity-check/s01-01-intro"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s04-04-fill-the-return"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s06-06-write-count-ready"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s07-07-checkpoint"}]}]},{"slug":"supervised-learning-workflows","lessons":[{"slug":"labels-features-splits","steps":[{"id":"supervised-learning-workflows/labels-features-splits/s01-01-intro"},{"id":"supervised-learning-workflows/labels-features-splits/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/labels-features-splits/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/labels-features-splits/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/labels-features-splits/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/labels-features-splits/s06-06-write-split-audit"},{"id":"supervised-learning-workflows/labels-features-splits/s07-07-checkpoint"}]},{"slug":"baselines-before-fancy-models","steps":[{"id":"supervised-learning-workflows/baselines-before-fancy-models/s01-01-intro"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s06-06-write-baseline-comparison"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s07-07-checkpoint"}]},{"slug":"training-and-prediction-loop","steps":[{"id":"supervised-learning-workflows/training-and-prediction-loop/s01-01-intro"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s06-06-write-train"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s07-07-checkpoint"}]},{"slug":"overfitting-and-regularization","steps":[{"id":"supervised-learning-workflows/overfitting-and-regularization/s01-01-intro"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s06-06-write-overfit-audit"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s07-07-checkpoint"}]},{"slug":"mission-baseline-model-showdown","steps":[{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s01-01-intro"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s06-06-write-classifier-brief"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s07-07-checkpoint"}]}]},{"slug":"unsupervised-learning-and-embeddings","lessons":[{"slug":"clustering-without-labels","steps":[{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s06-06-write-cluster-summary"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s07-07-checkpoint"}]},{"slug":"dimensionality-and-distance","steps":[{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s06-06-write-euclidean"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s07-07-checkpoint"}]},{"slug":"embeddings-as-features","steps":[{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s06-06-write-retrieval-receipt"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s07-07-checkpoint"}]},{"slug":"mission-tiny-recommender","steps":[{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s06-06-write-recommendation-card"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s07-07-checkpoint"}]}]},{"slug":"metrics-and-error-analysis","lessons":[{"slug":"confusion-matrix","steps":[{"id":"metrics-and-error-analysis/confusion-matrix/s01-01-intro"},{"id":"metrics-and-error-analysis/confusion-matrix/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/confusion-matrix/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/confusion-matrix/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/confusion-matrix/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/confusion-matrix/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/confusion-matrix/s07-07-checkpoint"}]},{"slug":"precision-recall-thresholds","steps":[{"id":"metrics-and-error-analysis/precision-recall-thresholds/s01-01-intro"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s07-07-checkpoint"}]},{"slug":"slice-based-error-analysis","steps":[{"id":"metrics-and-error-analysis/slice-based-error-analysis/s01-01-intro"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s07-07-checkpoint"}]},{"slug":"regression-metrics-and-residuals","steps":[{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s01-01-intro"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s07-07-checkpoint"}]},{"slug":"mission-confusion-matrix-triage","steps":[{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s01-01-intro"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s07-07-checkpoint"}]}]},{"slug":"pytorch-tensors-and-autograd","lessons":[{"slug":"tensors-and-shapes","steps":[{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s07-07-checkpoint"}]},{"slug":"broadcasting-and-vectorization","steps":[{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s07-07-checkpoint"}]},{"slug":"autograd-mental-model","steps":[{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s07-07-checkpoint"}]},{"slug":"mission-tensor-playground","steps":[{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s07-07-checkpoint"}]}]},{"slug":"training-loops-and-optimizers","lessons":[{"slug":"loss-forward-backward-step","steps":[{"id":"training-loops-and-optimizers/loss-forward-backward-step/s01-01-intro"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s07-07-checkpoint"}]},{"slug":"gradient-descent-by-hand","steps":[{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s01-01-intro"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s07-07-checkpoint"}]},{"slug":"optimizers-and-schedulers","steps":[{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s01-01-intro"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s07-07-checkpoint"}]},{"slug":"checkpoints-and-reproducibility","steps":[{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s01-01-intro"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s07-07-checkpoint"}]},{"slug":"mission-overfit-then-recover","steps":[{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s01-01-intro"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s07-07-checkpoint"}]}]},{"slug":"deep-learning-architectures","lessons":[{"slug":"cnns-and-local-patterns","steps":[{"id":"deep-learning-architectures/cnns-and-local-patterns/s01-01-intro"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s04-04-fill-the-return"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s06-06-write-count-ready"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s07-07-checkpoint"}]},{"slug":"tokenizers-and-context-budget","steps":[{"id":"deep-learning-architectures/tokenizers-and-context-budget/s01-01-intro"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s04-04-fill-the-return"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s06-06-write-count-ready"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s07-07-checkpoint"}]},{"slug":"attention-and-transformer-blocks","steps":[{"id":"deep-learning-architectures/attention-and-transformer-blocks/s01-01-intro"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s04-04-fill-the-return"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s06-06-write-count-ready"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s07-07-checkpoint"}]},{"slug":"decoding-kv-cache-quantization","steps":[{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s01-01-intro"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s04-04-fill-the-return"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s06-06-write-count-ready"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s07-07-checkpoint"}]},{"slug":"mission-architecture-tradeoff-note","steps":[{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s01-01-intro"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s04-04-fill-the-return"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s06-06-write-count-ready"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s07-07-checkpoint"}]}]},{"slug":"feature-experiments-registries","lessons":[{"slug":"feature-pipeline-contracts","steps":[{"id":"feature-experiments-registries/feature-pipeline-contracts/s01-01-intro"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s04-04-fill-the-return"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s06-06-write-count-ready"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s07-07-checkpoint"}]},{"slug":"train-inference-skew","steps":[{"id":"feature-experiments-registries/train-inference-skew/s01-01-intro"},{"id":"feature-experiments-registries/train-inference-skew/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/train-inference-skew/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/train-inference-skew/s04-04-fill-the-return"},{"id":"feature-experiments-registries/train-inference-skew/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/train-inference-skew/s06-06-write-count-ready"},{"id":"feature-experiments-registries/train-inference-skew/s07-07-checkpoint"}]},{"slug":"experiment-tracker-lite","steps":[{"id":"feature-experiments-registries/experiment-tracker-lite/s01-01-intro"},{"id":"feature-experiments-registries/experiment-tracker-lite/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/experiment-tracker-lite/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/experiment-tracker-lite/s04-04-fill-the-return"},{"id":"feature-experiments-registries/experiment-tracker-lite/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/experiment-tracker-lite/s06-06-write-count-ready"},{"id":"feature-experiments-registries/experiment-tracker-lite/s07-07-checkpoint"}]},{"slug":"model-registry-shape","steps":[{"id":"feature-experiments-registries/model-registry-shape/s01-01-intro"},{"id":"feature-experiments-registries/model-registry-shape/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/model-registry-shape/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/model-registry-shape/s04-04-fill-the-return"},{"id":"feature-experiments-registries/model-registry-shape/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/model-registry-shape/s06-06-write-count-ready"},{"id":"feature-experiments-registries/model-registry-shape/s07-07-checkpoint"}]},{"slug":"mission-feature-pipeline-and-tracker","steps":[{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s01-01-intro"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s04-04-fill-the-return"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s06-06-write-count-ready"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s07-07-checkpoint"}]}]},{"slug":"model-serving-and-mlops","lessons":[{"slug":"fastapi-inference-shape","steps":[{"id":"model-serving-and-mlops/fastapi-inference-shape/s01-01-intro"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s07-07-checkpoint"}]},{"slug":"batch-vs-realtime-serving","steps":[{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s01-01-intro"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s07-07-checkpoint"}]},{"slug":"docker-runtime-and-config","steps":[{"id":"model-serving-and-mlops/docker-runtime-and-config/s01-01-intro"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s07-07-checkpoint"}]},{"slug":"github-actions-validation-gates","steps":[{"id":"model-serving-and-mlops/github-actions-validation-gates/s01-01-intro"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s06-06-write-failing-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s07-07-checkpoint"}]},{"slug":"continuous-training-and-rollout","steps":[{"id":"model-serving-and-mlops/continuous-training-and-rollout/s01-01-intro"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s07-07-checkpoint"}]},{"slug":"mission-fastapi-model-server","steps":[{"id":"model-serving-and-mlops/mission-fastapi-model-server/s01-01-intro"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s07-07-checkpoint"}]}]},{"slug":"monitoring-cloud-portfolio","lessons":[{"slug":"structured-logs-and-alerts","steps":[{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s01-01-intro"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s07-07-checkpoint"}]},{"slug":"data-and-concept-drift","steps":[{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s01-01-intro"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s07-07-checkpoint"}]},{"slug":"retraining-triggers","steps":[{"id":"monitoring-cloud-portfolio/retraining-triggers/s01-01-intro"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s07-07-checkpoint"}]},{"slug":"cloud-gpu-kubernetes-cost","steps":[{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s01-01-intro"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s07-07-checkpoint"}]},{"slug":"portfolio-architecture-docs","steps":[{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s01-01-intro"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s07-07-checkpoint"}]},{"slug":"mission-final-portfolio-ml-system","steps":[{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s01-01-intro"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s06-06-write-decide"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s07-07-checkpoint"}]}]},{"slug":"graphic-design-studio","lessons":[{"slug":"prompting-like-an-art-director","steps":[{"id":"graphic-design-studio/prompting-like-an-art-director/s01-01-the-prompt-is-a-creative-brief"},{"id":"graphic-design-studio/prompting-like-an-art-director/s02-02-style-language"},{"id":"graphic-design-studio/prompting-like-an-art-director/s03-03-which-tool-for-the-headline"},{"id":"graphic-design-studio/prompting-like-an-art-director/s04-04-predict-the-scaffold"},{"id":"graphic-design-studio/prompting-like-an-art-director/s05-05-negative-space-and-the-iteration-loop"},{"id":"graphic-design-studio/prompting-like-an-art-director/s06-06-fix-the-mood-word-prompt"},{"id":"graphic-design-studio/prompting-like-an-art-director/s07-07-write-prompt-linter"},{"id":"graphic-design-studio/prompting-like-an-art-director/s08-08-checkpoint"}]},{"slug":"the-design-kit","steps":[{"id":"graphic-design-studio/the-design-kit/s01-01-the-brand-bible-as-data"},{"id":"graphic-design-studio/the-design-kit/s02-02-fill-the-palette-line"},{"id":"graphic-design-studio/the-design-kit/s03-03-banned-looks"},{"id":"graphic-design-studio/the-design-kit/s04-04-predict-the-negatives"},{"id":"graphic-design-studio/the-design-kit/s05-05-which-field-failed"},{"id":"graphic-design-studio/the-design-kit/s06-06-fix-the-crossed-kit"},{"id":"graphic-design-studio/the-design-kit/s07-07-write-contrast-check"},{"id":"graphic-design-studio/the-design-kit/s08-08-tokens-for-the-interface"},{"id":"graphic-design-studio/the-design-kit/s09-09-write-token-audit"},{"id":"graphic-design-studio/the-design-kit/s10-10-checkpoint"}]},{"slug":"fifty-assets-one-brand","steps":[{"id":"graphic-design-studio/fifty-assets-one-brand/s01-01-the-consistency-problem"},{"id":"graphic-design-studio/fifty-assets-one-brand/s02-02-seeds-the-same-dice-twice"},{"id":"graphic-design-studio/fifty-assets-one-brand/s03-03-why-does-the-mascot-drift"},{"id":"graphic-design-studio/fifty-assets-one-brand/s04-04-predict-the-batch-plan"},{"id":"graphic-design-studio/fifty-assets-one-brand/s05-05-the-qa-filter"},{"id":"graphic-design-studio/fifty-assets-one-brand/s06-06-defend-the-pick"},{"id":"graphic-design-studio/fifty-assets-one-brand/s07-07-fix-the-qa-threshold"},{"id":"graphic-design-studio/fifty-assets-one-brand/s08-08-write-palette-distance"},{"id":"graphic-design-studio/fifty-assets-one-brand/s09-09-checkpoint"}]}]},{"slug":"campaign-studio","lessons":[{"slug":"voice-pack-as-data","steps":[{"id":"campaign-studio/voice-pack-as-data/s01-01-the-campaign-and-the-gate"},{"id":"campaign-studio/voice-pack-as-data/s02-02-lint-the-draft"},{"id":"campaign-studio/voice-pack-as-data/s03-03-predict-the-violations"},{"id":"campaign-studio/voice-pack-as-data/s04-04-testable-or-mood"},{"id":"campaign-studio/voice-pack-as-data/s05-05-fill-the-linter"},{"id":"campaign-studio/voice-pack-as-data/s06-06-variants-as-records"},{"id":"campaign-studio/voice-pack-as-data/s07-07-write-lint-copy"},{"id":"campaign-studio/voice-pack-as-data/s08-08-checkpoint"}]},{"slug":"batch-plan-and-calendar","steps":[{"id":"campaign-studio/batch-plan-and-calendar/s01-01-slots-before-prompts"},{"id":"campaign-studio/batch-plan-and-calendar/s02-02-which-field-is-missing"},{"id":"campaign-studio/batch-plan-and-calendar/s03-03-write-batch-plan"},{"id":"campaign-studio/batch-plan-and-calendar/s04-04-the-calendar-is-data"},{"id":"campaign-studio/batch-plan-and-calendar/s05-05-predict-the-queue"},{"id":"campaign-studio/batch-plan-and-calendar/s06-06-fix-the-gate"},{"id":"campaign-studio/batch-plan-and-calendar/s07-07-checkpoint"}]},{"slug":"the-numbers-pass","steps":[{"id":"campaign-studio/the-numbers-pass/s01-01-a-report-is-a-draft-over-data"},{"id":"campaign-studio/the-numbers-pass/s02-02-predict-the-verdict"},{"id":"campaign-studio/the-numbers-pass/s03-03-write-verify-numbers"},{"id":"campaign-studio/the-numbers-pass/s04-04-denominators-and-the-ftc"},{"id":"campaign-studio/the-numbers-pass/s05-05-what-may-not-ship"},{"id":"campaign-studio/the-numbers-pass/s06-06-fill-the-policy-router"},{"id":"campaign-studio/the-numbers-pass/s07-07-checkpoint"}]}]},{"slug":"support-studio","lessons":[{"slug":"the-policy-pack","steps":[{"id":"support-studio/the-policy-pack/s01-01-the-reply-that-binds-the-company"},{"id":"support-studio/the-policy-pack/s02-02-the-pack-as-data"},{"id":"support-studio/the-policy-pack/s03-03-spot-the-banned-promise"},{"id":"support-studio/the-policy-pack/s04-04-predict-the-lint"},{"id":"support-studio/the-policy-pack/s05-05-fix-the-stale-citation"},{"id":"support-studio/the-policy-pack/s06-06-the-review-loop-evidence"},{"id":"support-studio/the-policy-pack/s07-07-write-lint-draft"},{"id":"support-studio/the-policy-pack/s08-08-checkpoint"}]},{"slug":"golden-evals-earn-autonomy","steps":[{"id":"support-studio/golden-evals-earn-autonomy/s01-01-the-klarna-arc"},{"id":"support-studio/golden-evals-earn-autonomy/s02-02-a-macro-is-three-things"},{"id":"support-studio/golden-evals-earn-autonomy/s03-03-inside-the-lane"},{"id":"support-studio/golden-evals-earn-autonomy/s04-04-predict-the-eval-run"},{"id":"support-studio/golden-evals-earn-autonomy/s05-05-fill-the-promotion-rule"},{"id":"support-studio/golden-evals-earn-autonomy/s06-06-write-run-golden-evals"},{"id":"support-studio/golden-evals-earn-autonomy/s07-07-checkpoint"}]},{"slug":"the-escalation-boundary","steps":[{"id":"support-studio/the-escalation-boundary/s01-01-drafting-is-not-spending"},{"id":"support-studio/the-escalation-boundary/s02-02-triggers-as-conditions"},{"id":"support-studio/the-escalation-boundary/s03-03-route-this-ticket"},{"id":"support-studio/the-escalation-boundary/s04-04-predict-the-routing"},{"id":"support-studio/the-escalation-boundary/s05-05-fix-the-refund-hole"},{"id":"support-studio/the-escalation-boundary/s06-06-disclosure-and-the-kill-switch"},{"id":"support-studio/the-escalation-boundary/s07-07-write-route-ticket"},{"id":"support-studio/the-escalation-boundary/s08-08-checkpoint"}]}]},{"slug":"copy-studio","lessons":[{"slug":"the-voice-fingerprint","steps":[{"id":"copy-studio/the-voice-fingerprint/s01-01-the-client-you-inherit"},{"id":"copy-studio/the-voice-fingerprint/s02-02-the-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s03-03-predict-the-ceiling"},{"id":"copy-studio/the-voice-fingerprint/s04-04-the-samples-say-otherwise"},{"id":"copy-studio/the-voice-fingerprint/s05-05-fill-derive-the-rules"},{"id":"copy-studio/the-voice-fingerprint/s06-06-the-pack-with-provenance"},{"id":"copy-studio/the-voice-fingerprint/s07-07-write-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s08-08-checkpoint"}]},{"slug":"variants-that-hold-the-line","steps":[{"id":"copy-studio/variants-that-hold-the-line/s01-01-twelve-by-thursday"},{"id":"copy-studio/variants-that-hold-the-line/s02-02-the-drift-problem"},{"id":"copy-studio/variants-that-hold-the-line/s03-03-predict-the-drift-report"},{"id":"copy-studio/variants-that-hold-the-line/s04-04-which-subject-line-ships"},{"id":"copy-studio/variants-that-hold-the-line/s05-05-fix-the-channel-check"},{"id":"copy-studio/variants-that-hold-the-line/s06-06-the-kill-log"},{"id":"copy-studio/variants-that-hold-the-line/s07-07-write-voice-gate"},{"id":"copy-studio/variants-that-hold-the-line/s08-08-checkpoint"}]},{"slug":"the-claim-check","steps":[{"id":"copy-studio/the-claim-check/s01-01-forty-one-of-seventy-seven"},{"id":"copy-studio/the-claim-check/s02-02-what-counts-as-checkable"},{"id":"copy-studio/the-claim-check/s03-03-predict-the-claim-list"},{"id":"copy-studio/the-claim-check/s04-04-what-closes-a-row"},{"id":"copy-studio/the-claim-check/s05-05-fill-the-ship-decision"},{"id":"copy-studio/the-claim-check/s06-06-the-corrections-log"},{"id":"copy-studio/the-claim-check/s07-07-write-ship-gate"},{"id":"copy-studio/the-claim-check/s08-08-checkpoint"}]}]},{"slug":"analytics-studio","lessons":[{"slug":"the-cleaning-log","steps":[{"id":"analytics-studio/the-cleaning-log/s01-01-where-did-those-rows-go"},{"id":"analytics-studio/the-cleaning-log/s02-02-the-log-as-data"},{"id":"analytics-studio/the-cleaning-log/s03-03-refuse-the-silent-step"},{"id":"analytics-studio/the-cleaning-log/s04-04-predict-the-receipts"},{"id":"analytics-studio/the-cleaning-log/s05-05-fill-the-null-decision"},{"id":"analytics-studio/the-cleaning-log/s06-06-fix-the-vanishing-rows"},{"id":"analytics-studio/the-cleaning-log/s07-07-write-the-null-ledger"},{"id":"analytics-studio/the-cleaning-log/s08-08-checkpoint"}]},{"slug":"every-number-has-an-address","steps":[{"id":"analytics-studio/every-number-has-an-address/s01-01-the-plausible-wrong-number"},{"id":"analytics-studio/every-number-has-an-address/s02-02-sample-first-then-full"},{"id":"analytics-studio/every-number-has-an-address/s03-03-which-check-catches-it"},{"id":"analytics-studio/every-number-has-an-address/s04-04-predict-the-fan-out"},{"id":"analytics-studio/every-number-has-an-address/s05-05-fill-the-trace"},{"id":"analytics-studio/every-number-has-an-address/s06-06-write-audit-the-memo"},{"id":"analytics-studio/every-number-has-an-address/s07-07-checkpoint"}]},{"slug":"challenge-then-rerun","steps":[{"id":"analytics-studio/challenge-then-rerun/s01-01-the-confident-recommendation"},{"id":"analytics-studio/challenge-then-rerun/s02-02-the-triad-as-a-checklist"},{"id":"analytics-studio/challenge-then-rerun/s03-03-name-the-leg"},{"id":"analytics-studio/challenge-then-rerun/s04-04-predict-the-rerun"},{"id":"analytics-studio/challenge-then-rerun/s05-05-fix-the-drifting-memo"},{"id":"analytics-studio/challenge-then-rerun/s06-06-write-score-the-challenge"},{"id":"analytics-studio/challenge-then-rerun/s07-07-checkpoint"}]}]},{"slug":"delivery-studio","lessons":[{"slug":"the-spec-is-the-build","steps":[{"id":"delivery-studio/the-spec-is-the-build/s01-01-the-clarifying-question-is-gone"},{"id":"delivery-studio/the-spec-is-the-build/s02-02-the-ticket-as-data"},{"id":"delivery-studio/the-spec-is-the-build/s03-03-spot-the-untestable-check"},{"id":"delivery-studio/the-spec-is-the-build/s04-04-predict-the-spec-lint"},{"id":"delivery-studio/the-spec-is-the-build/s05-05-fill-the-scope-fence"},{"id":"delivery-studio/the-spec-is-the-build/s06-06-inside-or-outside-the-frontier"},{"id":"delivery-studio/the-spec-is-the-build/s07-07-write-lint-spec"},{"id":"delivery-studio/the-spec-is-the-build/s08-08-checkpoint"}]},{"slug":"acceptance-before-the-build","steps":[{"id":"delivery-studio/acceptance-before-the-build/s01-01-the-ninety-seven-thousand-dollar-skip"},{"id":"delivery-studio/acceptance-before-the-build/s02-02-evidence-against-the-checks"},{"id":"delivery-studio/acceptance-before-the-build/s03-03-which-check-catches-it"},{"id":"delivery-studio/acceptance-before-the-build/s04-04-fix-the-vibes-review"},{"id":"delivery-studio/acceptance-before-the-build/s05-05-predict-the-review-run"},{"id":"delivery-studio/acceptance-before-the-build/s06-06-rework-is-a-list-not-a-feeling"},{"id":"delivery-studio/acceptance-before-the-build/s07-07-write-review-deliverable"},{"id":"delivery-studio/acceptance-before-the-build/s08-08-checkpoint"}]},{"slug":"status-from-artifacts","steps":[{"id":"delivery-studio/status-from-artifacts/s01-01-watermelon-status"},{"id":"delivery-studio/status-from-artifacts/s02-02-derive-the-rag"},{"id":"delivery-studio/status-from-artifacts/s03-03-spot-the-artifact"},{"id":"delivery-studio/status-from-artifacts/s04-04-fill-the-worst-of"},{"id":"delivery-studio/status-from-artifacts/s05-05-predict-the-rollup"},{"id":"delivery-studio/status-from-artifacts/s06-06-the-not-ready-call"},{"id":"delivery-studio/status-from-artifacts/s07-07-write-readiness-gate"},{"id":"delivery-studio/status-from-artifacts/s08-08-checkpoint"}]},{"slug":"the-hours-saved-receipt","steps":[{"id":"delivery-studio/the-hours-saved-receipt/s01-01-the-token-chart-is-not-a-receipt"},{"id":"delivery-studio/the-hours-saved-receipt/s02-02-the-receipt-as-data"},{"id":"delivery-studio/the-hours-saved-receipt/s03-03-which-line-survives-staff-meeting"},{"id":"delivery-studio/the-hours-saved-receipt/s04-04-fix-the-vanity-chart"},{"id":"delivery-studio/the-hours-saved-receipt/s05-05-predict-the-receipt-run"},{"id":"delivery-studio/the-hours-saved-receipt/s06-06-only-accepted-ships-count"},{"id":"delivery-studio/the-hours-saved-receipt/s07-07-write-hours-receipt"},{"id":"delivery-studio/the-hours-saved-receipt/s08-08-checkpoint"}]}]},{"slug":"people-studio","lessons":[{"slug":"the-data-firewall","steps":[{"id":"people-studio/the-data-firewall/s01-01-the-rule-that-cannot-live-in-memory"},{"id":"people-studio/the-data-firewall/s02-02-the-firewall-as-data"},{"id":"people-studio/the-data-firewall/s03-03-which-payload-passes"},{"id":"people-studio/the-data-firewall/s04-04-predict-the-scan"},{"id":"people-studio/the-data-firewall/s05-05-fix-the-leaky-gate"},{"id":"people-studio/the-data-firewall/s06-06-employee-relations-takes-nothing"},{"id":"people-studio/the-data-firewall/s07-07-write-prompt-gate"},{"id":"people-studio/the-data-firewall/s08-08-checkpoint"}]},{"slug":"briefs-with-receipts","steps":[{"id":"people-studio/briefs-with-receipts/s01-01-the-most-automated-desk"},{"id":"people-studio/briefs-with-receipts/s02-02-a-claim-with-a-receipt"},{"id":"people-studio/briefs-with-receipts/s03-03-sourced-or-inference"},{"id":"people-studio/briefs-with-receipts/s04-04-predict-the-receipt-audit"},{"id":"people-studio/briefs-with-receipts/s05-05-fill-the-lookup"},{"id":"people-studio/briefs-with-receipts/s06-06-the-screening-line"},{"id":"people-studio/briefs-with-receipts/s07-07-write-audit-brief"},{"id":"people-studio/briefs-with-receipts/s08-08-checkpoint"}]},{"slug":"the-paper-trail","steps":[{"id":"people-studio/the-paper-trail/s01-01-policy-is-legal-surface"},{"id":"people-studio/the-paper-trail/s02-02-the-boundary-as-a-state-machine"},{"id":"people-studio/the-paper-trail/s03-03-who-signs-off"},{"id":"people-studio/the-paper-trail/s04-04-predict-the-version-log"},{"id":"people-studio/the-paper-trail/s05-05-fix-the-shortcut"},{"id":"people-studio/the-paper-trail/s06-06-the-evidence-trail"},{"id":"people-studio/the-paper-trail/s07-07-write-decision-record"},{"id":"people-studio/the-paper-trail/s08-08-checkpoint"}]}]},{"slug":"ops-studio","lessons":[{"slug":"the-sop-is-data","steps":[{"id":"ops-studio/the-sop-is-data/s01-01-the-two-weeks-notice-problem"},{"id":"ops-studio/the-sop-is-data/s02-02-the-sop-as-data"},{"id":"ops-studio/the-sop-is-data/s03-03-spot-the-invented-step"},{"id":"ops-studio/the-sop-is-data/s04-04-fill-the-sop-lint"},{"id":"ops-studio/the-sop-is-data/s05-05-predict-the-diff"},{"id":"ops-studio/the-sop-is-data/s06-06-the-diff-is-the-change-control"},{"id":"ops-studio/the-sop-is-data/s07-07-write-lint-sop"},{"id":"ops-studio/the-sop-is-data/s08-08-checkpoint"}]},{"slug":"auto-run-or-named-approval","steps":[{"id":"ops-studio/auto-run-or-named-approval/s01-01-the-checklist-that-lied"},{"id":"ops-studio/auto-run-or-named-approval/s02-02-the-schedule-is-data"},{"id":"ops-studio/auto-run-or-named-approval/s03-03-which-item-may-auto-run"},{"id":"ops-studio/auto-run-or-named-approval/s04-04-fix-the-silent-auto-complete"},{"id":"ops-studio/auto-run-or-named-approval/s05-05-predict-the-exception-queue"},{"id":"ops-studio/auto-run-or-named-approval/s06-06-exceptions-are-a-queue-not-a-shrug"},{"id":"ops-studio/auto-run-or-named-approval/s07-07-write-run-checklist"},{"id":"ops-studio/auto-run-or-named-approval/s08-08-checkpoint"}]},{"slug":"numbers-with-receipts","steps":[{"id":"ops-studio/numbers-with-receipts/s01-01-the-number-that-moves-money"},{"id":"ops-studio/numbers-with-receipts/s02-02-recompute-dont-trust"},{"id":"ops-studio/numbers-with-receipts/s03-03-which-claim-has-no-receipt"},{"id":"ops-studio/numbers-with-receipts/s04-04-predict-the-verification"},{"id":"ops-studio/numbers-with-receipts/s05-05-fill-the-receipt-check"},{"id":"ops-studio/numbers-with-receipts/s06-06-the-anomaly-is-a-question"},{"id":"ops-studio/numbers-with-receipts/s07-07-write-verify-report"},{"id":"ops-studio/numbers-with-receipts/s08-08-checkpoint"}]},{"slug":"the-pii-proxy","steps":[{"id":"ops-studio/the-pii-proxy/s01-01-the-ticket-that-left-the-building"},{"id":"ops-studio/the-pii-proxy/s02-02-mask-then-unmask"},{"id":"ops-studio/the-pii-proxy/s03-03-which-payload-may-leave"},{"id":"ops-studio/the-pii-proxy/s04-04-fill-the-leftover-check"},{"id":"ops-studio/the-pii-proxy/s05-05-predict-the-two-sides"},{"id":"ops-studio/the-pii-proxy/s06-06-unmask-is-a-named-human-act"},{"id":"ops-studio/the-pii-proxy/s07-07-write-the-proxy"},{"id":"ops-studio/the-pii-proxy/s08-08-checkpoint"}]}]},{"slug":"legal-studio","lessons":[{"slug":"the-cite-check-gate","steps":[{"id":"legal-studio/the-cite-check-gate/s01-01-the-brief-that-reads-perfectly"},{"id":"legal-studio/the-cite-check-gate/s02-02-three-verdicts-as-data"},{"id":"legal-studio/the-cite-check-gate/s03-03-which-cite-needs-no-check"},{"id":"legal-studio/the-cite-check-gate/s04-04-predict-the-verification-run"},{"id":"legal-studio/the-cite-check-gate/s05-05-fix-the-verifier-that-asks-again"},{"id":"legal-studio/the-cite-check-gate/s06-06-write-verify-authorities"},{"id":"legal-studio/the-cite-check-gate/s07-07-checkpoint"}]},{"slug":"clause-extraction-with-pinpoints","steps":[{"id":"legal-studio/clause-extraction-with-pinpoints/s01-01-a-claim-without-a-pinpoint-is-a-rumor"},{"id":"legal-studio/clause-extraction-with-pinpoints/s02-02-the-playbook-as-data"},{"id":"legal-studio/clause-extraction-with-pinpoints/s03-03-which-row-is-unusable"},{"id":"legal-studio/clause-extraction-with-pinpoints/s04-04-predict-the-triage"},{"id":"legal-studio/clause-extraction-with-pinpoints/s05-05-fill-the-walk-away-route"},{"id":"legal-studio/clause-extraction-with-pinpoints/s06-06-write-triage-extraction"},{"id":"legal-studio/clause-extraction-with-pinpoints/s07-07-checkpoint"}]},{"slug":"privilege-rules-and-the-review-log","steps":[{"id":"legal-studio/privilege-rules-and-the-review-log/s01-01-privilege-is-a-routing-question"},{"id":"legal-studio/privilege-rules-and-the-review-log/s02-02-the-routing-table"},{"id":"legal-studio/privilege-rules-and-the-review-log/s03-03-which-request-is-the-waiver-risk"},{"id":"legal-studio/privilege-rules-and-the-review-log/s04-04-predict-the-router"},{"id":"legal-studio/privilege-rules-and-the-review-log/s05-05-fix-the-privilege-hole"},{"id":"legal-studio/privilege-rules-and-the-review-log/s06-06-the-log-that-survives-a-motion"},{"id":"legal-studio/privilege-rules-and-the-review-log/s07-07-write-log-ready"},{"id":"legal-studio/privilege-rules-and-the-review-log/s08-08-checkpoint"}]}]},{"slug":"agent-studio","lessons":[{"slug":"the-tool-contract","steps":[{"id":"agent-studio/the-tool-contract/s01-01-the-feature-you-ship-today"},{"id":"agent-studio/the-tool-contract/s02-02-the-schema-is-the-blast-radius"},{"id":"agent-studio/the-tool-contract/s03-03-spot-the-loose-tool"},{"id":"agent-studio/the-tool-contract/s04-04-predict-the-validation"},{"id":"agent-studio/the-tool-contract/s05-05-fix-the-yolo-rerun"},{"id":"agent-studio/the-tool-contract/s06-06-write-validate-call"},{"id":"agent-studio/the-tool-contract/s07-07-checkpoint"}]},{"slug":"the-loop-with-a-meter","steps":[{"id":"agent-studio/the-loop-with-a-meter/s01-01-a-loop-that-cannot-stop-is-an-invoice"},{"id":"agent-studio/the-loop-with-a-meter/s02-02-caps-are-part-of-the-loop"},{"id":"agent-studio/the-loop-with-a-meter/s03-03-why-did-it-stop"},{"id":"agent-studio/the-loop-with-a-meter/s04-04-predict-the-stop-reason"},{"id":"agent-studio/the-loop-with-a-meter/s05-05-fill-the-budget-check"},{"id":"agent-studio/the-loop-with-a-meter/s06-06-the-trace-is-the-receipt"},{"id":"agent-studio/the-loop-with-a-meter/s07-07-write-run-loop"},{"id":"agent-studio/the-loop-with-a-meter/s08-08-checkpoint"}]},{"slug":"evals-before-it-works","steps":[{"id":"agent-studio/evals-before-it-works/s01-01-green-before-it-runs"},{"id":"agent-studio/evals-before-it-works/s02-02-a-golden-case-is-a-past-incident"},{"id":"agent-studio/evals-before-it-works/s03-03-which-case-catches-it"},{"id":"agent-studio/evals-before-it-works/s04-04-predict-the-eval-run"},{"id":"agent-studio/evals-before-it-works/s05-05-fix-the-router-to-green"},{"id":"agent-studio/evals-before-it-works/s06-06-the-ratchet-entry"},{"id":"agent-studio/evals-before-it-works/s07-07-write-lint-the-ratchet"},{"id":"agent-studio/evals-before-it-works/s08-08-checkpoint"}]}]}]},"detail":{"number":26,"slug":"agent-harnesses","title":"agent harnesses — the layer between you and the raw api","blurb":"claude code, cursor, aider, codex cli — they're all the same four layers wrapped around the same model api. learn what those layers are, what each adds, and what you'd build yourself if you had to. then you build a small harness you can run and break. when the harness changes the world, the rollback lives in the repo — a runbook a junior can execute at 3am.","overview":"$2c","status":"live","lessons":[{"slug":"project-workspaces-not-product-names","title":"Project workspaces, not product names","estMinutes":8,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"agent-harnesses/project-workspaces-not-product-names/s01-01-intro","xp":2,"hint":[],"personalize":false,"phase":"warmup","estSeconds":300,"concept":"project-workspaces-not-product-names","body":"$2d","cta":"Got it","runnable":true}],"xpTotal":2},{"slug":"what-a-harness-is","title":"What a harness is — the four layers under every coding agent","estMinutes":12,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"agent-harnesses/what-a-harness-is/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":130,"concept":"harness-vs-raw-call","body":"$2e","cta":"Got it","code":"$2f","runnable":true},{"type":"mc","id":"agent-harnesses/what-a-harness-is/s02-02-which-is-a-harness","xp":2,"hint":[{"level":1,"body":"The discriminator is layer 4: tool dispatch. Can the system DO something (edit a file, run a command, search the web) on the model's behalf? If yes, harness. If no (just chat), wrapper.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"harness-vs-wrapper","prompt":"A \"harness\" wraps the model API with at least: input prep, model\ncall, output parsing, AND tool dispatch (the model can change the\nworld, not just describe it). Which TWO of the systems above are\nharnesses?\n","options":[{"id":"b_and_d","label":"B (Cursor) and D (Aider)"},{"id":"all_four","label":"All four — anything that calls a model is a harness","explain":"A and C are wrappers — they call the model and render output, but the model can't change anything. No tool dispatch = not a harness."},{"id":"a_and_b","label":"A and B","explain":"A is just a script that calls the API. No tools, no input prep beyond a hardcoded prompt — it's a wrapper, not a harness."},{"id":"c_and_d","label":"C and D","explain":"C is a chat UI. The model talks; nothing changes outside the chat window. No tool dispatch."}],"answerIds":["b_and_d"],"shuffle":false,"code":"systems = {\n \"A\": \"A Python script that calls anthropic.messages.create with a hardcoded prompt and prints the result.\",\n \"B\": \"Cursor — reads open files, runs edits, applies diffs, has tools.\",\n \"C\": \"A web chat UI that sends user input to gpt-4o and renders the response.\",\n \"D\": \"Aider — gathers repo context, calls model, applies file edits, integrates with git.\",\n}\nfor k, v in systems.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"agent-harnesses/what-a-harness-is/s03-03-the-four-layers","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":110,"concept":"four-harness-layers","body":"$30","cta":"Got it","code":"# the four layers, named and ordered.\n\nLAYERS = [\n \"1. input prep — load config, gather context, build messages\",\n \"2. model call — messages.create with retries, cache, routing\",\n \"3. output parsing — separate text from tool_use, accumulate\",\n \"4. tool dispatch — run requested tools, append results, loop\",\n]\nfor layer in LAYERS:\n print(layer)\n","runnable":true},{"type":"predict","id":"agent-harnesses/what-a-harness-is/s04-04-predict-the-layer","xp":3,"hint":[{"level":1,"body":"Layer 1 (input prep): `load_claude_md` = 1. Layer 2 (model call + retries): `messages.create` and `with_retries` = 2. Layer 3 (output parsing): `for block in response.content` = 1. Layer 4 (tool dispatch): `TOOLS[name](**args)` = 1.","cost":0}],"personalize":false,"phase":"build","estSeconds":80,"concept":"classify-harness-code-by-layer","code":"# five lines from a real harness. each belongs to one layer.\n\nsnippets = {\n \"load_claude_md(root)\": 1,\n \"client.messages.create(...)\": 2,\n \"for block in response.content:\": 3,\n \"TOOLS[name](**args)\": 4,\n \"with_retries(call, max_attempts=3)\": 2,\n}\n\nlayer_counts = [0, 0, 0, 0, 0] # index 1..4 used\nfor code, layer in snippets.items():\n layer_counts[layer] += 1\n\nfor i in range(1, 5):\n print(f\"layer {i}: {layer_counts[i]}\")\n","prompt":"Read the editor. Five snippets, classified into the four layers.\nThe script counts how many snippets land in each layer. What\ndoes it print?\n","grader":{"kind":"stdout-equality","expected":"layer 1: 1\nlayer 2: 2\nlayer 3: 1\nlayer 4: 1\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"agent-harnesses/what-a-harness-is/s05-05-fill-the-dispatch","xp":3,"hint":[{"level":1,"body":"Two pieces: registry lookup `TOOLS[name]`, then call with `**input` to unpack the dict as keyword arguments.","cost":0},{"level":2,"body":"`TOOLS[tool_use[\"name\"]](**tool_use[\"input\"])` — the canonical layer-4 dispatch.","cost":0}],"personalize":false,"phase":"build","estSeconds":65,"concept":"tool-registry-dispatch","prompt":"The harness has a tool registry (a dict) and a `tool_use` block\nfrom the model. Fill in the dispatch line: look up the tool by\nname and call it with the input dict unpacked.\n\nExpected output:\n```\nresults for tokyo\n```\n","code":"TOOLS = {\n \"search\": lambda q: f\"results for {q}\",\n \"read\": lambda path: f\"\",\n}\n\ntool_use = {\"name\": \"search\", \"input\": {\"q\": \"tokyo\"}}\n\nresult = ___\nprint(result)\n","blanks":[{"id":"dispatch","accept":["TOOLS[tool_use[\"name\"]](**tool_use[\"input\"])","TOOLS[tool_use['name']](**tool_use['input'])"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"agent-harnesses/what-a-harness-is/s06-06-fix-the-no-retry","xp":4,"hint":[{"level":1,"body":"Wrap the call in a loop `for attempt in range(max_attempts)`. Try, except TransientError — if not the last attempt, continue. Else re-raise. Else (success) return result.","cost":0},{"level":2,"body":"Replace the body of `call_with_retries`:\n\n```python\nfor attempt in range(max_attempts):\n try:\n return fn()\n except TransientError:\n if attempt == max_attempts - 1:\n raise\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"retries-around-model-call","brokenCode":"call_count = {\"n\": 0}\n\nclass TransientError(Exception):\n transient = True\n\ndef flaky_call():\n call_count[\"n\"] += 1\n if call_count[\"n\"] < 3:\n raise TransientError(\"503 service unavailable\")\n return \"succeeded on attempt 3\"\n\ndef call_with_retries(fn, max_attempts=3):\n # bug: no retry. one attempt, raises on first failure.\n return fn()\n\nprint(call_with_retries(flaky_call))\n","prompt":"The harness's layer-2 code calls the model exactly once. If the\ncall hits a transient 429 (rate limit) or 503 (service\nunavailable), it raises and the user sees a crash. Production\nharnesses retry transient failures with exponential backoff.\n\nFix `call_with_retries(fn, max_attempts=3)` to retry up to 3\ntimes on transient errors (any exception with `transient=True`\nattribute). Return the first successful result. Re-raise on\nthe final attempt failure.\n\nExpected output:\n```\nsucceeded on attempt 3\n```\n","grader":{"kind":"stdout-equality","expected":"succeeded on attempt 3","normalize":"collapse-trailing-newline"},"bugLines":[13,14],"revealAfter":5},{"type":"fix","id":"agent-harnesses/what-a-harness-is/s07-07-fix-the-hardcoded-registry","xp":4,"hint":[{"level":1,"body":"Add a `tools` parameter to `dispatch`, drop the hardcoded TOOLS dict, change the lookup to use the parameter, and pass `external_tools` from the call site.","cost":0},{"level":2,"body":"```python\ndef dispatch(tool_use, tools):\n return tools[tool_use[\"name\"]](**tool_use[\"input\"])\n\nresult = dispatch({\"name\": \"search\", \"input\": {\"q\": \"tokyo\"}}, external_tools)\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"inject-tool-registry","brokenCode":"def dispatch(tool_use):\n # bug: tools hardcoded INSIDE the function. adding tools = edits to this body.\n TOOLS = {\n \"search\": lambda q: f\"hits for {q}\",\n }\n return TOOLS[tool_use[\"name\"]](**tool_use[\"input\"])\n\nexternal_tools = {\n \"search\": lambda q: f\"hits for {q}\",\n}\n\nresult = dispatch({\"name\": \"search\", \"input\": {\"q\": \"tokyo\"}})\nprint(result)\n","prompt":"The harness has the tool registry hardcoded INSIDE the loop.\nAdding a new tool means editing the loop function — not a\nmodule-level dict. After three new tools, the loop body is\nunreadable.\n\nFix `dispatch(tool_use, tools)` to take the registry as an\nargument instead of hardcoding it. Then pass the registry in\nfrom outside.\n\nExpected output:\n```\nhits for tokyo\n```\n","grader":{"kind":"stdout-equality","expected":"hits for tokyo","normalize":"collapse-trailing-newline"},"bugLines":[1,7],"revealAfter":4},{"type":"write","id":"agent-harnesses/what-a-harness-is/s08-08-write-mini-harness","xp":5,"hint":[{"level":1,"body":"Same loop shape as previous chapter agent loops, with explicit layer comments. Layer 1: build messages. Loop: layer 2 (call), layer 3 (split content into text/tool_use), branch on stop_reason. On tool_use: layer 4 (dispatch each, append assistant + user turns).","cost":0},{"level":2,"body":"Three counters/buckets: messages list, tool_calls_total int, iter index from `range(1, max_iters + 1)`. End_turn returns the joined text and stats. Cap returns the literal \"capped\".","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"minimal-four-layer-harness","prompt":"Write `mini_harness(user_input, tools, fake_model, max_iters)`\nthat implements the four layers in under 50 lines:\n\n- **Layer 1**: build `messages = [{\"role\": \"user\", \"content\": user_input}]`.\n- Loop up to `max_iters` times:\n - **Layer 2**: call `fake_model(messages)` (returns a dict\n with `stop_reason` and `content`).\n - **Layer 3**: filter `content` for text blocks (`type==\"text\"`)\n and tool_use blocks (`type==\"tool_use\"`).\n - If `stop_reason == \"end_turn\"`: return\n `{\"text\": , \"iters\": , \"tool_calls\": }`.\n - **Layer 4**: for each tool_use block, dispatch\n `tools[block[\"name\"]](**block[\"input\"])`. Track total tool\n runs. Append assistant + user(tool_results) turns.\n- On cap: return `{\"text\": \"capped\", \"iters\": max_iters, \"tool_calls\": }`.\n\nTwo cases run for you. Expected output:\n```\ntext='Tokyo guide.' iters=2 tool_calls=1\ntext='capped' iters=2 tool_calls=2\n```\n","starter":"$31","grader":{"kind":"stdout-equality","expected":"text='Tokyo guide.' iters=2 tool_calls=1\ntext='capped' iters=2 tool_calls=2\n","normalize":"collapse-trailing-newline"},"solution":"$32","hiddenTests":[]},{"type":"checkpoint","id":"agent-harnesses/what-a-harness-is/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"harness-with-all-four-layers","prompt":"$33","starter":"$34","grader":{"kind":"stdout-equality","expected":"text='Found ramen.' iters=2 tool_calls=1 tokens=130 system_used=True\ntext='capped' iters=2 tool_calls=2 tokens=180 system_used=False\n","normalize":"collapse-trailing-newline"},"solution":"$35","passThreshold":0.66}],"xpTotal":32},{"slug":"building-a-harness","title":"building a harness — scaffold, tools, loop, gate","estMinutes":18,"prerequisites":["what-a-harness-is"],"status":"live","steps":[{"type":"read","id":"agent-harnesses/building-a-harness/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":140,"concept":"build-a-harness-you-can-break","body":"$36","cta":"Got it","code":"# agent = model + harness\n# the model samples. the harness delivers context and talks to the world.\n\nproduct = {\n \"you_see\": \"a chat box and a diff\",\n \"you_cannot\": [\"step the loop\", \"break one layer\", \"rerun last night's case\"],\n}\n\nyours = {\n \"files\": [\"loop.py\", \"tools.py\", \"prompts.md\", \"evals.jsonl\", \"runbook.md\"],\n \"you_can\": [\"step the loop\", \"break one layer\", \"rerun last night's case\"],\n}\n\nprint(\"agent = model + harness\")\nprint(\"product hides:\", \", \".join(product[\"you_cannot\"]))\nprint(\"your repo keeps:\", \", \".join(yours[\"files\"]))\n","runnable":true},{"type":"mc","id":"agent-harnesses/building-a-harness/s02-02-which-files-belong","xp":2,"hint":[{"level":1,"body":"five files that let you step the loop and fail a run. two files that do not. the model-string note is the one you never start with.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"minimal-harness-repo-files","prompt":"a minimal harness repo is five isolated files you can\nstep and break: loop, tools, prompts, evals, runbook.\neach file is one concern, so the next harness inherits\nthe pieces that already work. vendor notes and\nscreenshots are noise.\n\nwhich option is the minimal repo?\n","options":[{"id":"a_through_e","label":"A–E (loop.py, tools.py, prompts.md, evals.jsonl, runbook.md)"},{"id":"all_seven","label":"A–G — ship the screenshots and the model-string note too","explain":"F and G are noise. starting from the model string is the build-order bug. a screenshot is not a gate."},{"id":"a_c_f","label":"A, C, and F — loop, prompts, and the model pick","explain":"F is the thing you write last, if ever. you still need tools, evals, and a runbook."},{"id":"b_d_g","label":"B, D, and G — tools, evals, and a screenshot of the first run","explain":"a screenshot is not a file you can rerun. you still need the loop, the prompts, and the runbook."}],"answerIds":["a_through_e"],"shuffle":false,"code":"files = {\n \"A\": \"loop.py — prep, call, parse, dispatch, cap\",\n \"B\": \"tools.py — registry + allowlist\",\n \"C\": \"prompts.md — templates, not policy\",\n \"D\": \"evals.jsonl — cases the gate runs\",\n \"E\": \"runbook.md — detect / decide / writeback\",\n \"F\": \"model-picks.md — which vendor string we feel lucky about this week\",\n \"G\": \"screenshots/first-run.png\",\n}\nfor k, v in files.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"agent-harnesses/building-a-harness/s03-03-the-build-order","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":140,"concept":"harness-build-order","body":"$37","cta":"Got it","code":"# never start with the model string.\n\nBUILD_ORDER = [\n \"1. contract — HarnessSpec: name, trigger, inputs, decisions, writeback, eval_gate\",\n \"2. tools — registry + allowlist. the model may only name these.\",\n \"3. loop — prep → call → parse → dispatch, with a cap\",\n \"4. gate — eval_gate / evals.jsonl. fail the run or do not ship\",\n \"5. runbook — detect / decide / writeback when the loop changes the world\",\n]\nfor line in BUILD_ORDER:\n print(line)\n","runnable":true},{"type":"predict","id":"agent-harnesses/building-a-harness/s04-04-predict-the-failed-layer","xp":3,"hint":[{"level":1,"body":"prep built messages. call came back tool_use. parse lost the tool_use blocks. dispatch sat idle. the first `ok is False` is the answer.","cost":0}],"personalize":false,"phase":"build","estSeconds":80,"concept":"classify-broken-harness-log","code":"# a broken harness log. one layer failed. the rest never got a fair turn.\n\nlog = [\n {\"layer\": \"prep\", \"ok\": True, \"detail\": \"messages built from spec+inputs\"},\n {\"layer\": \"call\", \"ok\": True, \"detail\": \"stop_reason=tool_use\"},\n {\"layer\": \"parse\", \"ok\": False, \"detail\": \"treated tool_use as text\"},\n {\"layer\": \"dispatch\", \"ok\": None, \"detail\": \"never ran — no tool_use blocks\"},\n]\n\nfailed = next(row[\"layer\"] for row in log if row[\"ok\"] is False)\nprint(failed)\n","prompt":"read the editor. four log rows from a harness that returned\ntext and never called a tool, even though the model asked\nfor one. which layer failed? the script prints that name.\n","grader":{"kind":"stdout-equality","expected":"parse\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"agent-harnesses/building-a-harness/s05-05-fill-the-harness-spec","xp":3,"hint":[{"level":1,"body":"six strings in a list. contract first, gate last. writeback sits in front of the gate — that is the path the result hits.","cost":0},{"level":2,"body":"`['name', 'trigger', 'inputs', 'decisions', 'writeback', 'eval_gate']` — single quotes match the expected output.","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"harness-spec-fields","prompt":"fill the list of the SIX HarnessSpec fields, in build order:\nname, trigger, inputs, decisions, writeback, eval_gate.\n\n`decisions` is deterministic policy — route, approve,\ncompact, swap — not a paragraph you hope the model\nfollows.\n\nexpected output:\n```\n['name', 'trigger', 'inputs', 'decisions', 'writeback', 'eval_gate']\n```\n","code":"SPEC_FIELDS = ___\nprint(SPEC_FIELDS)\n","blanks":[{"id":"spec_fields","accept":["['name', 'trigger', 'inputs', 'decisions', 'writeback', 'eval_gate']","[\"name\", \"trigger\", \"inputs\", \"decisions\", \"writeback\", \"eval_gate\"]"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"agent-harnesses/building-a-harness/s06-06-fix-the-prompt-policy","xp":4,"hint":[{"level":1,"body":"policy, guardrails, swaps, compaction — those fire in code. the prompt can stay as tone. `decide` has to implement the rule itself — `FLAG` in the note means `ROUTE`, else `KEEP`.","cost":0},{"level":2,"body":"```python\ndef decide(note):\n return \"ROUTE\" if \"FLAG\" in note else \"KEEP\"\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"policy-in-code-not-prompt","brokenCode":"prompt = (\n \"triage the note. if it mentions FLAG, return ROUTE. \"\n \"otherwise return KEEP. please follow this carefully.\"\n)\n\ndef decide(note):\n # bug: policy lives in the prompt. decide() is a shrug.\n return \"ask the model\"\n\nitem = {\"note\": \"FLAG on the incoming item\"}\nprint(decide(item[\"note\"]))\n","prompt":"the harness stuffed the routing rule into the prompt.\ndeterministic logic does not belong there — policy,\nguardrails, a model swap, compaction of history. a\nparagraph the model can ignore is not a decision, and\nyou cannot unit-test it.\n\nfix `decide(note)` so the rule is `HarnessSpec.decisions`\nin code: if `\"FLAG\"` is in the note, return `\"ROUTE\"`.\notherwise return `\"KEEP\"`. leave the prompt as tone,\nnot policy.\n\nexpected output:\n```\nROUTE\n```\n","grader":{"kind":"stdout-equality","expected":"ROUTE","normalize":"collapse-trailing-newline"},"bugLines":[7,8],"revealAfter":4},{"type":"fix","id":"agent-harnesses/building-a-harness/s07-07-fix-the-uncapped-allowlist","xp":4,"hint":[{"level":1,"body":"swap `while True` for `for i in range(1, max_iters + 1)`. before dispatch, if `block[\"name\"]` is not in `allowlist`, increment skipped and continue.","cost":0},{"level":2,"body":"```python\ndef run_loop(fake_model, tools, allowlist, max_iters):\n messages = [{\"role\": \"user\", \"content\": \"go\"}]\n tool_calls = 0\n skipped = 0\n for i in range(1, max_iters + 1):\n response = fake_model(messages)\n if response[\"stop_reason\"] == \"end_turn\":\n return {\"iters\": i, \"tool_calls\": tool_calls, \"skipped\": skipped}\n for block in response[\"content\"]:\n if block[\"type\"] != \"tool_use\":\n continue\n if block[\"name\"] not in allowlist:\n skipped += 1\n continue\n tools[block[\"name\"]](**block[\"input\"])\n tool_calls += 1\n return {\"iters\": max_iters, \"tool_calls\": tool_calls, \"skipped\": skipped}\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"cap-and-tool-allowlist","brokenCode":"$38","prompt":"the loop has no cap and no allowlist. the model asked for\n`wipe` twice. `wipe` is not a tool you expose. without a\ncap the loop would keep going; without an allowlist it\nKeyErrors on the unknown name.\n\nfix `run_loop` so it (1) iterates at most `max_iters` times\nand (2) skips any tool_use whose name is not in `allowlist`.\nreturn `{\"iters\": , \"tool_calls\": , \"skipped\": }`.\n\nexpected output:\n```\niters=2 tool_calls=0 skipped=2\n```\n","grader":{"kind":"stdout-equality","expected":"iters=2 tool_calls=0 skipped=2","normalize":"collapse-trailing-newline"},"bugLines":[14,19,26],"revealAfter":5},{"type":"write","id":"agent-harnesses/building-a-harness/s08-08-write-build-harness-run","xp":5,"hint":[{"level":1,"body":"prep is one user message built from spec+inputs. the loop is lesson 01 plus an allowlist check before dispatch. gate fails first on allowlist violation or a hit cap, then requires must_call + must_include.","cost":0},{"level":2,"body":"skip unknown names — do not look them up, and set a violated flag so the gate fails. the second case asks for `wipe`; if you dispatch it you crash. if you skip it, tool_calls stays 0 and gate fails because the allowlist was violated.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"build-harness-run-end-to-end","prompt":"$39","starter":"$3a","grader":{"kind":"stdout-equality","expected":"text='ROUTE night.' iters=2 tool_calls=1 gate=pass\ntext='done.' iters=2 tool_calls=0 gate=fail\n","normalize":"collapse-trailing-newline"},"solution":"$3b","hiddenTests":[]},{"type":"checkpoint","id":"agent-harnesses/building-a-harness/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"name-layers-and-ship-gate","prompt":"final drill. name the four layers you just built, in order,\nand the gate you refuse to ship without.\n\nset `LAYERS` to `[\"prep\", \"call\", \"parse\", \"dispatch\"]`\nand `GATE` to `\"eval_gate\"`. print the layers joined by\nspaces, then the gate on the next line.\n\nexpected output:\n```\nprep call parse dispatch\neval_gate\n```\n","starter":"# name the four layers + the gate you refuse to ship without.\n\nLAYERS = []\nGATE = \"\"\n\nprint(\" \".join(LAYERS))\nprint(GATE)\n","grader":{"kind":"stdout-equality","expected":"prep call parse dispatch\neval_gate\n","normalize":"collapse-trailing-newline"},"solution":"# name the four layers + the gate you refuse to ship without.\n\nLAYERS = [\"prep\", \"call\", \"parse\", \"dispatch\"]\nGATE = \"eval_gate\"\n\nprint(\" \".join(LAYERS))\nprint(GATE)\n","passThreshold":0.66}],"xpTotal":32},{"slug":"architecting-an-ai-native-workflow","title":"Architecting an AI-native workflow — a 5-step playbook in code","estMinutes":15,"prerequisites":["building-a-harness"],"status":"live","steps":[{"type":"read","id":"agent-harnesses/architecting-an-ai-native-workflow/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":160,"concept":"ai-assisted-vs-ai-native","body":"$3c","cta":"Got it","code":"# the difference between \"ai-assisted\" and \"ai-native\" as a data shape.\nai_assisted_company = {\n \"question\": \"Where can we add AI to save time?\",\n \"ai_lives\": \"at the edges (chatgpt tab, marketing copilot)\",\n \"workflow_design\": \"human-driven, AI sprinkled on top\",\n \"human_role\": \"search engine, router, copywriter\",\n \"metric\": \"hours saved\",\n}\n\nai_native_company = {\n \"question\": \"How should this workflow exist if agents do the first 80%?\",\n \"ai_lives\": \"at the center (agents in the operating fabric)\",\n \"workflow_design\": \"structured, documented, permissioned, instrumented\",\n \"human_role\": \"reviewer of ambiguity, supervisor of judgment\",\n \"metric\": \"resolution_time, conversion_rate, revenue_per_employee\",\n}\n\nfor k in ai_assisted_company:\n a = ai_assisted_company[k]\n n = ai_native_company[k]\n print(f\"{k:18} assisted: {a}\")\n print(f\"{k:18} native: {n}\")\n print()\n","runnable":true},{"type":"mc","id":"agent-harnesses/architecting-an-ai-native-workflow/s02-02-which-workflow-fits","xp":2,"hint":[{"level":1,"body":"Three signals you want: high volume, rules already exist (even informally), and humans currently doing more coordination than judgment. One workflow hits all three.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":60,"concept":"pick-the-right-workflow","prompt":"Step 1 of the playbook: pick a narrow workflow with\nobvious economic value. The criteria: **volume is high, rules\nexist, humans are doing too much coordination.**\n\nFour candidate workflows. Which ONE is the strongest fit?\n","options":[{"id":"a","label":"A — founder vision setting (vol=1, no rules, 5h coord)","explain":"Low volume + no rules + tacit knowledge. This is exactly the work the playbook says you DON'T automate. Strategy is judgment, not workflow."},{"id":"b","label":"B — support ticket triage (vol=4200, rules exist, 80h coord)"},{"id":"c","label":"C — acquiring a competitor (vol=0, no rules)","explain":"Zero volume = not a workflow, it's a project. Agents pay off on repetition; one-off deals don't compound."},{"id":"d","label":"D — quarterly board prep (vol=1, rules exist, 12h coord)","explain":"Rules exist but volume is one per quarter. Cost of structuring exceeds savings. Wait until you have something that runs daily."}],"answerIds":["b"],"shuffle":false,"code":"workflows = {\n \"A\": {\"name\": \"Founder vision setting\", \"volume_per_week\": 1, \"rules_exist\": False, \"human_coord_hours\": 5},\n \"B\": {\"name\": \"Support ticket triage\", \"volume_per_week\": 4200, \"rules_exist\": True, \"human_coord_hours\": 80},\n \"C\": {\"name\": \"Acquiring a competitor\", \"volume_per_week\": 0, \"rules_exist\": False, \"human_coord_hours\": 0},\n \"D\": {\"name\": \"Quarterly board prep\", \"volume_per_week\": 1, \"rules_exist\": True, \"human_coord_hours\": 12},\n}\nfor k, v in workflows.items():\n print(f\"{k}: {v['name']:30} vol={v['volume_per_week']:4} rules={v['rules_exist']} coord_h={v['human_coord_hours']}\")\n","runnable":true},{"type":"read","id":"agent-harnesses/architecting-an-ai-native-workflow/s03-03-mapping-the-workflow","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":140,"concept":"workflow-spec-as-data","body":"$3d","cta":"Got it","code":"$3e","runnable":true},{"type":"predict","id":"agent-harnesses/architecting-an-ai-native-workflow/s04-04-predict-the-readiness-score","xp":3,"hint":[{"level":1,"body":"Count `name`, `trigger`, `inputs`, `decisions`. That's 4 present out of 7 required. `4/7 = 0.5714...`, rounded to 2 decimals is `0.57`.","cost":0},{"level":2,"body":"Output is two lines: `present: 4/7` and `score: 0.57` (three spaces after `score:` because the format string aligns columns).","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"readiness-score-from-fields","code":"REQUIRED = [\"name\", \"trigger\", \"inputs\", \"decisions\", \"success_criteria\", \"error_sites\", \"tacit_knowledge\"]\n\npartial_spec = {\n \"name\": \"Outbound prospecting\",\n \"trigger\": \"new lead in CRM\",\n \"inputs\": [\"lead.company\", \"lead.role\", \"lead.intent_signals\"],\n \"decisions\": {\"classify_segment\": {\"reversible\": True, \"needs_approval\": False}},\n # success_criteria: missing\n # error_sites: missing\n # tacit_knowledge: missing\n}\n\npresent = [f for f in REQUIRED if f in partial_spec]\nscore = round(len(present) / len(REQUIRED), 2)\n\nprint(f\"present: {len(present)}/{len(REQUIRED)}\")\nprint(f\"score: {score}\")\n","prompt":"The spec has 4 of the 7 required fields. The validator computes\na readiness score as `present / required`, rounded to 2 decimals.\n\nWhat does the script print?\n","grader":{"kind":"stdout-equality","expected":"present: 4/7\nscore: 0.57\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"agent-harnesses/architecting-an-ai-native-workflow/s05-05-fill-the-required-fields","xp":3,"hint":[{"level":1,"body":"Seven strings in a list, in the order: name, trigger, inputs, decisions, success_criteria, error_sites, tacit_knowledge.","cost":0},{"level":2,"body":"`['name', 'trigger', 'inputs', 'decisions', 'success_criteria', 'error_sites', 'tacit_knowledge']` — single quotes match the expected output.","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"required-spec-fields","prompt":"Fill the list of the SEVEN required spec fields a workflow needs\nto be machine-readable. Order matches the lesson: name, trigger,\ninputs, decisions, success_criteria, error_sites, tacit_knowledge.\n\nExpected output:\n```\n['name', 'trigger', 'inputs', 'decisions', 'success_criteria', 'error_sites', 'tacit_knowledge']\n```\n","code":"REQUIRED = ___\nprint(REQUIRED)\n","blanks":[{"id":"required_list","accept":["['name', 'trigger', 'inputs', 'decisions', 'success_criteria', 'error_sites', 'tacit_knowledge']","[\"name\", \"trigger\", \"inputs\", \"decisions\", \"success_criteria\", \"error_sites\", \"tacit_knowledge\"]"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"agent-harnesses/architecting-an-ai-native-workflow/s06-06-fix-the-implicit-policy","xp":4,"hint":[{"level":1,"body":"Replace the string with a dict. Two keys: `auto_approve_under` and `requires_human_above`, both pointing at the same number (100). The agent now has a rule to follow.","cost":0},{"level":2,"body":"`workflow[\"refund_policy\"] = {\"auto_approve_under\": 100, \"requires_human_above\": 100}`. Print prints the dict in insertion order (Python 3.7+).","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"implicit-to-explicit-rule","brokenCode":"workflow = {\n \"name\": \"Refund processing\",\n # bug: \"ask the support lead\" is not a policy. an agent cannot\n # ground in tribal knowledge. encode the rule explicitly.\n \"refund_policy\": \"ask the support lead\",\n}\n\nprint(workflow[\"refund_policy\"])\n","prompt":"A canonical implicit-policy failure: \"the sales process is 'talk\nto Sarah, she knows how we do enterprise.'\" That's tribal\nknowledge. An agent can't ask Sarah. The fix is to make Sarah's\nknowledge explicit as a data rule.\n\nThe spec below has `refund_policy = \"ask the support lead\"`. An\nagent reading this has nothing to ground in. Replace it with an\nexplicit rule dict containing `auto_approve_under` (dollar\nthreshold below which the agent can refund without human review)\nand `requires_human_above` (the same threshold). Use $100.\n\nExpected output:\n```\n{'auto_approve_under': 100, 'requires_human_above': 100}\n```\n","grader":{"kind":"stdout-equality","expected":"{'auto_approve_under': 100, 'requires_human_above': 100}","normalize":"collapse-trailing-newline"},"bugLines":[5],"revealAfter":4},{"type":"fix","id":"agent-harnesses/architecting-an-ai-native-workflow/s07-07-fix-the-wrong-metric","xp":4,"hint":[{"level":1,"body":"Replace the list with `['resolution_time_minutes', 'csat']`. Both show up in the customer's experience and on the company's dashboard.","cost":0},{"level":2,"body":"`workflow[\"metrics\"] = [\"resolution_time_minutes\", \"csat\"]`. Hours saved is what you tell investors before they ask for revenue numbers; resolution_time and csat are what they ask for next.","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"business-metric-not-hours-saved","brokenCode":"workflow = {\n \"name\": \"Support triage\",\n # bug: \"hours saved\" is a fake metric. business impact is\n # resolution_time, conversion_rate, csat, revenue_per_employee.\n \"metrics\": [\"hours_saved_per_week\"],\n}\n\nprint(workflow[\"metrics\"])\n","prompt":"Step 5 of the playbook is the one most teams botch: measure\nbusiness impact, NOT \"hours saved.\" Hours saved is a vanity metric\na PM invents to justify a project; it doesn't show up on the\nincome statement.\n\nThe spec below tracks `hours_saved_per_week`. Replace it with two\nreal business metrics: `resolution_time_minutes` (median time to\nresolve a ticket) and `csat` (customer satisfaction, 1-5).\n\nExpected output (the two business metrics):\n```\n['resolution_time_minutes', 'csat']\n```\n","grader":{"kind":"stdout-equality","expected":"['resolution_time_minutes', 'csat']","normalize":"collapse-trailing-newline"},"bugLines":[5],"revealAfter":4},{"type":"write","id":"agent-harnesses/architecting-an-ai-native-workflow/s08-08-write-the-workflow-validator","xp":5,"hint":[{"level":1,"body":"Five returned keys. Build `missing` as a sorted list so output is deterministic. `has_explicit_policies` defaults to True when there are no policies (empty dict).","cost":0},{"level":2,"body":"`has_explicit_policies = all(isinstance(v, dict) for v in policies.values())` if policies else True. `business_metrics_only = not any(m in VANITY for m in metrics)`. `ready` is the AND of three signals.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"workflow-spec-validator","prompt":"Write `is_ai_native_ready(spec)` that returns a dict with:\n\n- `score`: float, fraction of REQUIRED fields present (rounded to 2)\n- `missing`: list of required field names not in the spec\n- `has_explicit_policies`: True iff EVERY value in\n `spec.get(\"policies\", {})` is a dict (not a string — \"ask Sarah\"\n is a string and disqualifies)\n- `business_metrics_only`: True iff `spec.get(\"metrics\", [])` contains\n NONE of these vanity strings: `\"hours_saved\"`, `\"hours_saved_per_week\"`,\n `\"engagement\"`, `\"vibes\"`\n- `ready`: True iff score >= 0.85 AND has_explicit_policies AND\n business_metrics_only\n\nTwo specs run. Expected output:\n```\n{'score': 1.0, 'missing': [], 'has_explicit_policies': True, 'business_metrics_only': True, 'ready': True}\n{'score': 0.43, 'missing': ['decisions', 'error_sites', 'success_criteria', 'tacit_knowledge'], 'has_explicit_policies': False, 'business_metrics_only': False, 'ready': False}\n```\n","starter":"REQUIRED = [\"name\", \"trigger\", \"inputs\", \"decisions\", \"success_criteria\", \"error_sites\", \"tacit_knowledge\"]\nVANITY = {\"hours_saved\", \"hours_saved_per_week\", \"engagement\", \"vibes\"}\n\ngood_spec = {\n \"name\": \"Support triage\",\n \"trigger\": \"ticket.created\",\n \"inputs\": [\"ticket.body\", \"customer.history\"],\n \"decisions\": {\"classify\": {\"reversible\": True, \"needs_approval\": False}},\n \"success_criteria\": \"first-response < 10min\",\n \"error_sites\": [\"wrong-account retrieval\"],\n \"tacit_knowledge\": \"tone with angry-but-right customers\",\n \"policies\": {\"refund\": {\"auto_approve_under\": 100}},\n \"metrics\": [\"resolution_time_minutes\", \"csat\"],\n}\n\nvibe_spec = {\n \"name\": \"Sales outreach\",\n \"trigger\": \"lead.created\",\n \"inputs\": [\"lead.company\"],\n \"policies\": {\"who_to_contact\": \"ask Sarah, she knows\"},\n \"metrics\": [\"hours_saved_per_week\"],\n}\n\n# define is_ai_native_ready(spec) below\n\nprint(is_ai_native_ready(good_spec))\nprint(is_ai_native_ready(vibe_spec))\n","grader":{"kind":"stdout-equality","expected":"{'score': 1.0, 'missing': [], 'has_explicit_policies': True, 'business_metrics_only': True, 'ready': True}\n{'score': 0.43, 'missing': ['decisions', 'error_sites', 'success_criteria', 'tacit_knowledge'], 'has_explicit_policies': False, 'business_metrics_only': False, 'ready': False}\n","normalize":"collapse-trailing-newline"},"solution":"$3f","hiddenTests":[]},{"type":"checkpoint","id":"agent-harnesses/architecting-an-ai-native-workflow/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"triage-three-workflow-specs","prompt":"Final drill. Three workflow specs from three teams who want to\ngo AI-native. Score each one with `is_ai_native_ready` from step\n08. Print a one-line verdict per spec: `READY` or\n`NOT READY: `.\n\nReason is the FIRST failing check, in this priority order:\n1. score < 0.85 -> `\"incomplete spec (score=X)\"`\n2. not has_explicit_policies -> `\"implicit policy\"`\n3. not business_metrics_only -> `\"vanity metric\"`\n\nThree specs run. Expected output:\n```\nSupport: READY\nSales: NOT READY: implicit policy\nOnboard: NOT READY: incomplete spec (score=0.57)\n```\n","starter":"$40","grader":{"kind":"stdout-equality","expected":"Support: READY\nSales: NOT READY: implicit policy\nOnboard: NOT READY: incomplete spec (score=0.57)\n","normalize":"collapse-trailing-newline"},"solution":"$41","passThreshold":0.66}],"xpTotal":31},{"slug":"five-industries-walked-through","title":"Five industries walked through — what AI-native looks like in the wild","estMinutes":20,"prerequisites":["architecting-an-ai-native-workflow"],"status":"live","steps":[{"type":"read","id":"agent-harnesses/five-industries-walked-through/s01-01-the-case-study-method","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":180,"concept":"case-study-method","body":"$42","cta":"Got it","runnable":true},{"type":"read","id":"agent-harnesses/five-industries-walked-through/s02-02-home-services","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":200,"concept":"case-study-home-services","body":"$43","cta":"Got it","runnable":true},{"type":"mc","id":"agent-harnesses/five-industries-walked-through/s03-03-which-task-belongs-to-the-agent","xp":2,"hint":[{"level":1,"body":"The general rule (echoing Isenberg's writing on AI-native services): agents handle structured work, humans handle taste, trust, judgment, relationships, exceptions. Firing someone and lawyer-negotiation are both 100% judgment + relationships.","cost":0}],"personalize":false,"phase":"build","estSeconds":55,"concept":"agent-vs-human-task-split","prompt":"Five tasks from the home-services rebuild. Three of them belong to\nthe agent (structured work). Two of them belong to a human (taste,\ntrust, judgment, relationships, exceptions).\n\nWhich TWO should stay with humans?\n","options":[{"id":"2_and_4","label":"2 (fire a tech) and 4 (negotiate with a lawyer)"},{"id":"1_and_5","label":"1 (classify call) and 5 (generate invoice)","explain":"Both are structured tasks with explicit rules. Classification is a canonical agent job; invoice drafting is mechanical given a job-done signal. Humans don't need to do either."},{"id":"3_and_4","label":"3 (look up history) and 4 (negotiate with a lawyer)","explain":"Customer-history lookup is exactly what agents are best at — read structured records and synthesize. Don't waste a human on it."},{"id":"1_and_2","label":"1 (classify call) and 2 (fire a tech)","explain":"Half right. Firing a tech is judgment. But classifying calls is the agent's whole reason to exist in this workflow."}],"answerIds":["2_and_4"],"shuffle":false,"code":"# five tasks from the home-services workflow. who should own each?\ntasks = [\n \"Classify an incoming call as emergency vs scheduled\",\n \"Decide whether to fire a long-tenured tech for repeated rudeness\",\n \"Look up the customer's prior service history and draft a job brief\",\n \"Negotiate a one-off discount with the customer's lawyer over a disputed invoice\",\n \"Generate the invoice draft when the tech says the job is done\",\n]\nfor i, t in enumerate(tasks, 1):\n print(f\"{i}. {t}\")\n","runnable":true},{"type":"read","id":"agent-harnesses/five-industries-walked-through/s04-04-insurance-brokerage","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":220,"concept":"case-study-insurance-brokerage","body":"$44","cta":"Got it","runnable":true},{"type":"read","id":"agent-harnesses/five-industries-walked-through/s05-05-recruiting-firm","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":200,"concept":"case-study-recruiting","body":"$45","cta":"Got it","runnable":true},{"type":"mc","id":"agent-harnesses/five-industries-walked-through/s06-06-which-industry-wedges-fastest","xp":2,"hint":[{"level":1,"body":"The wedge profile is: HIGH volume, EXPLICIT rules (so the agent has something to ground in), STRUCTURED outcomes (recurring shapes, not one-off bespoke work). Only one of the four hits all three.","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"pick-the-fastest-wedge","prompt":"Four candidate industries for an AI-native rebuild. Per the\ncase-study method (high volume + explicit rules + structured\noutcomes), which ONE wedges in fastest?\n","options":[{"id":"a","label":"A — boutique M&A law firm","explain":"Low volume + mostly tacit rules + high-stakes outcomes. This is the OPPOSITE of the wedge profile. The cost of structuring the workflow exceeds the savings; the irreversible-outcome nature means humans can't be removed."},{"id":"b","label":"B — bookkeeping for SMBs"},{"id":"c","label":"C — PR agency","explain":"Taste-driven and relationship-heavy. Agents can draft, but the value the client buys is the human taste behind the pitch. Volume's also medium, not high."},{"id":"d","label":"D — investment banking capital markets","explain":"Low volume + heavily regulated + brand-risk on every output. Worst possible profile for an agent-first rebuild."}],"answerIds":["b"],"shuffle":false,"code":"$46","runnable":true},{"type":"read","id":"agent-harnesses/five-industries-walked-through/s07-07-the-incumbent-trap","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":200,"concept":"process-debt-and-the-incumbent-trap","body":"$47","cta":"Got it","runnable":true},{"type":"write","id":"agent-harnesses/five-industries-walked-through/s08-08-write-the-process-debt-audit","xp":5,"hint":[{"level":1,"body":"Six conditionals, each adds a fixed number of points. Use `.get()` with defaults so missing fields don't crash. Then four-way threshold for the verdict.","cost":0},{"level":2,"body":"HVAC Inc hits all six conditions: 25 (old) + 20 (scattered) + 20 (tacit) + 15 (paper) + 10 (overstaffed) + 10 (undocumented) = 100. StartupCo hits zero conditions, score 0. The verdict thresholds: <=20 ready, <=50 manageable, <=80 heavy, >80 trap.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"process-debt-scoring","prompt":"Write `process_debt_score(company)` that takes a company profile\n(dict) and returns a dict with two fields:\n\n- `score`: integer 0-100, higher means MORE process debt (bad)\n- `verdict`: string, one of:\n - `\"AI-native ready\"` if score <= 20\n - `\"manageable debt\"` if score <= 50\n - `\"heavy debt\"` if score <= 80\n - `\"incumbent trap\"` if score > 80\n\nScore the company on these signals (each adds points to the debt\ntotal):\n\n- `age_years > 10`: add 25 (incumbent business — typical 20+ year history)\n- `data_scattered_across_tools >= 5`: add 20 (CRM + Slack + email + spreadsheets...)\n- `tacit_knowledge_pct > 50`: add 20 (more than half of company knowledge \"in someone's head\")\n- `paper_processes > 0`: add 15 (any paper at all)\n- `headcount_per_revenue_million > 7`: add 10 (overstaffed)\n- `documented_workflows_pct < 30`: add 10 (mostly undocumented)\n\nTwo companies run. Expected output:\n```\nHVAC Inc: {'score': 100, 'verdict': 'incumbent trap'}\nStartupCo: {'score': 0, 'verdict': 'AI-native ready'}\n```\n","starter":"hvac_inc = {\n \"name\": \"HVAC Inc\",\n \"age_years\": 22,\n \"data_scattered_across_tools\": 6,\n \"tacit_knowledge_pct\": 65,\n \"paper_processes\": 2,\n \"headcount_per_revenue_million\": 8,\n \"documented_workflows_pct\": 15,\n}\n\nstartup_co = {\n \"name\": \"StartupCo\",\n \"age_years\": 1,\n \"data_scattered_across_tools\": 2,\n \"tacit_knowledge_pct\": 30,\n \"paper_processes\": 0,\n \"headcount_per_revenue_million\": 4,\n \"documented_workflows_pct\": 80,\n}\n\n# define process_debt_score(company) below\n\nprint(f\"HVAC Inc: {process_debt_score(hvac_inc)}\")\nprint(f\"StartupCo: {process_debt_score(startup_co)}\")\n","grader":{"kind":"stdout-equality","expected":"HVAC Inc: {'score': 100, 'verdict': 'incumbent trap'}\nStartupCo: {'score': 0, 'verdict': 'AI-native ready'}\n","normalize":"collapse-trailing-newline"},"solution":"$48","hiddenTests":[]},{"type":"checkpoint","id":"agent-harnesses/five-industries-walked-through/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":240,"concept":"synthesize-the-case-study-method","prompt":"Final drill. Synthesize the case-study method into one function:\n`pick_industry(candidates)` that takes a list of industry profiles\nand returns the name of the BEST candidate for an AI-native\nrebuild.\n\nEach candidate is a dict with:\n- `name`: string\n- `volume_per_week`: int\n- `rules_explicit`: bool\n- `outcome_structured`: bool\n- `incumbent_avg_age_years`: int\n\nScore each candidate using these rules (higher = better wedge):\n\n- `volume_per_week >= 100`: +30\n- `rules_explicit`: +25\n- `outcome_structured`: +25\n- `incumbent_avg_age_years >= 20`: +20 (incumbent trap is biggest)\n\nReturn the name of the highest-scoring candidate. On a tie, return\nthe FIRST one (Python's `max` with `key=` preserves stable order).\n\nFive candidates run. Expected output:\n```\nbest: Insurance brokerage\n```\n","starter":"candidates = [\n {\"name\": \"Boutique law firm\", \"volume_per_week\": 2, \"rules_explicit\": False, \"outcome_structured\": False, \"incumbent_avg_age_years\": 30},\n {\"name\": \"Insurance brokerage\", \"volume_per_week\": 200, \"rules_explicit\": True, \"outcome_structured\": True, \"incumbent_avg_age_years\": 35},\n {\"name\": \"PR agency\", \"volume_per_week\": 40, \"rules_explicit\": False, \"outcome_structured\": False, \"incumbent_avg_age_years\": 15},\n {\"name\": \"Bookkeeping for SMBs\", \"volume_per_week\": 300, \"rules_explicit\": True, \"outcome_structured\": True, \"incumbent_avg_age_years\": 18},\n {\"name\": \"VC investing\", \"volume_per_week\": 1, \"rules_explicit\": False, \"outcome_structured\": False, \"incumbent_avg_age_years\": 25},\n]\n\n# define pick_industry(candidates) below\n\nprint(f\"best: {pick_industry(candidates)}\")\n","grader":{"kind":"stdout-equality","expected":"best: Insurance brokerage","normalize":"collapse-trailing-newline"},"solution":"$49","passThreshold":0.66}],"xpTotal":22},{"slug":"incident-runbook-from-the-repo","title":"Incident runbook from the repo — detect, decide rollback, write back","estMinutes":14,"prerequisites":["what-a-harness-is"],"status":"live","steps":[{"type":"read","id":"agent-harnesses/incident-runbook-from-the-repo/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":140,"concept":"wiki-vs-repo-runbook","body":"$4a","cta":"Got it","code":"# two artifacts that both get called a \"runbook.\"\n# only one can be executed from the repo at 3am.\n\nwiki = \"if the nightly batch looks bad, use your judgment\"\n\nrunbook = {\n \"detect\": {\"signal\": \"error_rate\", \"rollback_if_at_least\": 0.08},\n \"named_human\": \"release-owner\",\n \"writeback\": \"incidents/log.jsonl\",\n}\n\nprint(\"wiki: a page a junior has to interpret\")\nprint(\n f\"runbook: detect={runbook['detect']['signal']} \"\n f\"rollback_if>={runbook['detect']['rollback_if_at_least']} \"\n f\"human={runbook['named_human']} \"\n f\"writeback={runbook['writeback']}\"\n)\n","runnable":true},{"type":"mc","id":"agent-harnesses/incident-runbook-from-the-repo/s02-02-which-is-a-runbook","xp":2,"hint":[{"level":1,"body":"The discriminator is executability. Can a junior (or a harness) read one artifact and know detect / decide / who / writeback without inventing a process?","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"executable-runbook-shape","prompt":"A runbook a junior can execute from the repo has to be data:\na signal to detect, a rollback-versus-continue rule, a named\nhuman if the decision cannot be executed alone, and a path\nto write the record back. Which ONE of the artifacts above\nis that runbook?\n","options":[{"id":"a","label":"A — the Notion page","explain":"A page a junior has to interpret is still a wiki. 'Maybe' and 'find someone' are not a threshold or a named seat."},{"id":"b","label":"B — the dict in the repo"},{"id":"c","label":"C — last month's Slack thread","explain":"A thread is a story about one night. It is not a rule the next junior can execute, and nothing gets written back to the repo."},{"id":"d","label":"D — 'use your judgment'","explain":"'The team' is not a named human, and judgment is what the runbook is supposed to replace at 3am."}],"answerIds":["b"],"shuffle":false,"code":"artifacts = {\n \"A\": \"A Notion page: 'if the nightly batch looks bad, maybe roll back or find someone.'\",\n \"B\": \"A dict in the repo: detect.error_rate, rollback_if_at_least=0.08, named_human=release-owner, writeback=incidents/log.jsonl.\",\n \"C\": \"Last month's Slack thread, pinned, titled 'what we did last time'.\",\n \"D\": \"A wiki heading: 'Use your judgment. Page the team if unsure.'\",\n}\nfor k, v in artifacts.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"agent-harnesses/incident-runbook-from-the-repo/s03-03-detect-decide-writeback","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":120,"concept":"detect-decide-writeback","body":"$4b","cta":"Got it","code":"# the decision table a junior runs. rollback is the default\n# once the signal crosses the line. escalate only when\n# rollback itself cannot run.\n\nTHRESHOLD = 0.08\nincidents = [\n {\"id\": \"INC-041\", \"error_rate\": 0.12, \"rollback_blocked\": False},\n {\"id\": \"INC-042\", \"error_rate\": 0.03, \"rollback_blocked\": False},\n {\"id\": \"INC-043\", \"error_rate\": 0.15, \"rollback_blocked\": True},\n]\n\ndef decide(inc):\n if inc[\"rollback_blocked\"]:\n return \"escalate\"\n if inc[\"error_rate\"] >= THRESHOLD:\n return \"rollback\"\n return \"continue\"\n\nfor inc in incidents:\n print(f\"{inc['id']} {decide(inc)}\")\n","runnable":true},{"type":"predict","id":"agent-harnesses/incident-runbook-from-the-repo/s04-04-predict-the-decision","xp":3,"hint":[{"level":1,"body":"Blocked is checked first — INC-043 escalates even though 0.15 is over the line. 0.12 >= 0.08 is rollback. 0.03 is continue.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"classify-incident-decision","code":"THRESHOLD = 0.08\nincidents = [\n {\"id\": \"INC-041\", \"error_rate\": 0.12, \"rollback_blocked\": False},\n {\"id\": \"INC-042\", \"error_rate\": 0.03, \"rollback_blocked\": False},\n {\"id\": \"INC-043\", \"error_rate\": 0.15, \"rollback_blocked\": True},\n]\n\ndef decide(inc):\n if inc[\"rollback_blocked\"]:\n return \"escalate\"\n if inc[\"error_rate\"] >= THRESHOLD:\n return \"rollback\"\n return \"continue\"\n\nfor inc in incidents:\n print(f\"{inc['id']} {decide(inc)}\")\n","prompt":"Read the editor. Three incidents, one threshold (0.08),\nrollback blocked only on INC-043. What does the script\nprint?\n","grader":{"kind":"stdout-equality","expected":"INC-041 rollback\nINC-042 continue\nINC-043 escalate\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"agent-harnesses/incident-runbook-from-the-repo/s05-05-fill-the-decision-rule","xp":3,"hint":[{"level":1,"body":"Look up the signal name, then compare it to `rollback_if_at_least`. Do not hardcode `error_rate` or `0.08`.","cost":0},{"level":2,"body":"`signals[runbook[\"detect\"][\"signal\"]] >= runbook[\"detect\"][\"rollback_if_at_least\"]`","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"rollback-threshold-rule","prompt":"Fill the rollback comparison. Read the signal name and the\nthreshold from the runbook — do not hardcode `error_rate`\nor `0.08`. Escalate is already handled.\n\nExpected output:\n```\nrollback\ncontinue\n```\n","code":"runbook = {\n \"detect\": {\"signal\": \"error_rate\", \"rollback_if_at_least\": 0.08},\n}\n\ndef decide(signals, runbook):\n if signals.get(\"rollback_blocked\"):\n return \"escalate\"\n if ___:\n return \"rollback\"\n return \"continue\"\n\nprint(decide({\"error_rate\": 0.12}, runbook))\nprint(decide({\"error_rate\": 0.03}, runbook))\n","blanks":[{"id":"threshold_check","accept":["signals[runbook[\"detect\"][\"signal\"]] >= runbook[\"detect\"][\"rollback_if_at_least\"]","signals[runbook['detect']['signal']] >= runbook['detect']['rollback_if_at_least']"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"agent-harnesses/incident-runbook-from-the-repo/s06-06-fix-the-wiki-page","xp":4,"hint":[{"level":1,"body":"Replace the string with a dict. Four keys: `detect` (the signal name), `rollback_if_at_least` (0.08), `named_human` (`release-owner`), `writeback` (`incidents/log.jsonl`).","cost":0},{"level":2,"body":"```python\nrunbook = {\n \"detect\": \"error_rate\",\n \"rollback_if_at_least\": 0.08,\n \"named_human\": \"release-owner\",\n \"writeback\": \"incidents/log.jsonl\",\n}\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"runbook-as-data-not-prose","brokenCode":"# bug: a wiki sentence is not a runbook. a junior cannot\n# execute prose. encode detect / threshold / human / path.\nrunbook = \"if the nightly batch looks bad, use your judgment\"\n\nprint(runbook[\"detect\"])\nprint(runbook[\"rollback_if_at_least\"])\nprint(runbook[\"named_human\"])\nprint(runbook[\"writeback\"])\n","prompt":"The \"runbook\" is a sentence. A junior cannot execute a\nsentence. Replace it with a dict that has the four fields\na checkout can follow: `detect`, `rollback_if_at_least`,\n`named_human`, `writeback`.\n\nUse `error_rate`, `0.08`, `release-owner`, and\n`incidents/log.jsonl`. The print statements stay as they\nare — they read those four keys.\n\nExpected output:\n```\nerror_rate\n0.08\nrelease-owner\nincidents/log.jsonl\n```\n","grader":{"kind":"stdout-equality","expected":"error_rate\n0.08\nrelease-owner\nincidents/log.jsonl\n","normalize":"collapse-trailing-newline"},"bugLines":[3],"revealAfter":4},{"type":"fix","id":"agent-harnesses/incident-runbook-from-the-repo/s07-07-fix-the-unnamed-human","xp":4,"hint":[{"level":1,"body":"On escalate, return `runbook[\"named_human\"]`. Leave the `return \"runbook\"` branch for rollback and continue.","cost":0},{"level":2,"body":"Replace `return \"the team\"` with `return runbook[\"named_human\"]`.","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"named-human-not-the-team","brokenCode":"runbook = {\"named_human\": \"release-owner\"}\n\ndef actor_for(decision, runbook):\n # bug: \"the team\" is not a named human. escalate has\n # to resolve to a seat the junior can actually page.\n if decision == \"escalate\":\n return \"the team\"\n return \"runbook\"\n\nprint(actor_for(\"escalate\", runbook))\nprint(actor_for(\"rollback\", runbook))\nprint(actor_for(\"continue\", runbook))\n","prompt":"Escalate currently pages `\"the team\"`. That is a channel,\nnot a person. A junior cannot execute \"page the team.\"\nReturn `runbook[\"named_human\"]` on escalate, and\n`\"runbook\"` on every decision the checkout can finish\nalone.\n\nExpected output:\n```\nrelease-owner\nrunbook\nrunbook\n```\n","grader":{"kind":"stdout-equality","expected":"release-owner\nrunbook\nrunbook\n","normalize":"collapse-trailing-newline"},"bugLines":[7],"revealAfter":4},{"type":"write","id":"agent-harnesses/incident-runbook-from-the-repo/s08-08-write-the-runbook","xp":5,"hint":[{"level":1,"body":"Blocked first, then threshold, then continue. Actor is the named human only on escalate. Artifact is the writeback path.","cost":0},{"level":2,"body":"Build a dict with those five keys in that order so `print` matches. Look up the signal with `signals[runbook[\"detect\"][\"signal\"]]`.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"execute-runbook","prompt":"Write `execute_runbook(signals, runbook)` that returns a\nwriteback record — the dict a junior would append to the\nrepo. Fields:\n\n- `id`: `signals[\"id\"]`\n- `decision`: `\"escalate\"` if `signals.get(\"rollback_blocked\")`;\n `\"rollback\"` if the detect signal is >=\n `rollback_if_at_least`; otherwise `\"continue\"`\n- `signal`: the detect signal's value\n- `actor`: `runbook[\"named_human\"]` on escalate, else\n `\"runbook\"`\n- `artifact`: `runbook[\"writeback\"][\"path\"]`\n\nTwo incidents run. Expected output:\n```\n{'id': 'INC-041', 'decision': 'rollback', 'signal': 0.12, 'actor': 'runbook', 'artifact': 'incidents/log.jsonl'}\n{'id': 'INC-043', 'decision': 'escalate', 'signal': 0.15, 'actor': 'release-owner', 'artifact': 'incidents/log.jsonl'}\n```\n","starter":"RUNBOOK = {\n \"detect\": {\"signal\": \"error_rate\", \"rollback_if_at_least\": 0.08},\n \"named_human\": \"release-owner\",\n \"writeback\": {\"path\": \"incidents/log.jsonl\"},\n}\n\ninc_041 = {\"id\": \"INC-041\", \"error_rate\": 0.12, \"rollback_blocked\": False}\ninc_043 = {\"id\": \"INC-043\", \"error_rate\": 0.15, \"rollback_blocked\": True}\n\n# define execute_runbook(signals, runbook) below\n\nprint(execute_runbook(inc_041, RUNBOOK))\nprint(execute_runbook(inc_043, RUNBOOK))\n","grader":{"kind":"stdout-equality","expected":"{'id': 'INC-041', 'decision': 'rollback', 'signal': 0.12, 'actor': 'runbook', 'artifact': 'incidents/log.jsonl'}\n{'id': 'INC-043', 'decision': 'escalate', 'signal': 0.15, 'actor': 'release-owner', 'artifact': 'incidents/log.jsonl'}\n","normalize":"collapse-trailing-newline"},"solution":"RUNBOOK = {\n \"detect\": {\"signal\": \"error_rate\", \"rollback_if_at_least\": 0.08},\n \"named_human\": \"release-owner\",\n \"writeback\": {\"path\": \"incidents/log.jsonl\"},\n}\n\ninc_041 = {\"id\": \"INC-041\", \"error_rate\": 0.12, \"rollback_blocked\": False}\ninc_043 = {\"id\": \"INC-043\", \"error_rate\": 0.15, \"rollback_blocked\": True}\n\ndef execute_runbook(signals, runbook):\n detect = runbook[\"detect\"]\n if signals.get(\"rollback_blocked\"):\n decision = \"escalate\"\n elif signals[detect[\"signal\"]] >= detect[\"rollback_if_at_least\"]:\n decision = \"rollback\"\n else:\n decision = \"continue\"\n actor = runbook[\"named_human\"] if decision == \"escalate\" else \"runbook\"\n return {\n \"id\": signals[\"id\"],\n \"decision\": decision,\n \"signal\": signals[detect[\"signal\"]],\n \"actor\": actor,\n \"artifact\": runbook[\"writeback\"][\"path\"],\n }\n\nprint(execute_runbook(inc_041, RUNBOOK))\nprint(execute_runbook(inc_043, RUNBOOK))\n","hiddenTests":[]},{"type":"checkpoint","id":"agent-harnesses/incident-runbook-from-the-repo/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"three-incident-writebacks","prompt":"Final drill. Run all three incidents through\n`execute_runbook` from step 08. Print one writeback line\nper incident:\n\n`{id} {decision} actor={actor} artifact={artifact}`\n\nSame runbook. Same rules: blocked → escalate and name\n`release-owner`; at or above 0.08 → rollback; else\ncontinue. Actor is `runbook` unless escalate fired.\n\nThree incidents run. Expected output:\n```\nINC-041 rollback actor=runbook artifact=incidents/log.jsonl\nINC-042 continue actor=runbook artifact=incidents/log.jsonl\nINC-043 escalate actor=release-owner artifact=incidents/log.jsonl\n```\n","starter":"RUNBOOK = {\n \"detect\": {\"signal\": \"error_rate\", \"rollback_if_at_least\": 0.08},\n \"named_human\": \"release-owner\",\n \"writeback\": {\"path\": \"incidents/log.jsonl\"},\n}\n\nincidents = [\n {\"id\": \"INC-041\", \"error_rate\": 0.12, \"rollback_blocked\": False},\n {\"id\": \"INC-042\", \"error_rate\": 0.03, \"rollback_blocked\": False},\n {\"id\": \"INC-043\", \"error_rate\": 0.15, \"rollback_blocked\": True},\n]\n\ndef execute_runbook(signals, runbook):\n detect = runbook[\"detect\"]\n if signals.get(\"rollback_blocked\"):\n decision = \"escalate\"\n elif signals[detect[\"signal\"]] >= detect[\"rollback_if_at_least\"]:\n decision = \"rollback\"\n else:\n decision = \"continue\"\n actor = runbook[\"named_human\"] if decision == \"escalate\" else \"runbook\"\n return {\n \"id\": signals[\"id\"],\n \"decision\": decision,\n \"signal\": signals[detect[\"signal\"]],\n \"actor\": actor,\n \"artifact\": runbook[\"writeback\"][\"path\"],\n }\n\n# print one writeback line per incident\n","grader":{"kind":"stdout-equality","expected":"INC-041 rollback actor=runbook artifact=incidents/log.jsonl\nINC-042 continue actor=runbook artifact=incidents/log.jsonl\nINC-043 escalate actor=release-owner artifact=incidents/log.jsonl\n","normalize":"collapse-trailing-newline"},"solution":"$4c","passThreshold":0.66}],"xpTotal":32}],"liveLessonCount":6,"liveStepCount":46,"xpTotal":151}},"chapter":"$28:props:tree:detail","lesson":"$28:props:tree:detail:lessons:2","step":"$28:props:tree:detail:lessons:2:steps:3","stepIndex":3,"nav":{"tocNext":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"tocPrev":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"nextByPath":{"developer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"marketer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"designer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"customer-service":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"copywriter":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"data-analyst":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"project-manager":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"hr-specialist":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"operations":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4},"lawyer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":4}},"prevByPath":{"developer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"marketer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"designer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"customer-service":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"copywriter":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"data-analyst":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"project-manager":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"hr-specialist":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"operations":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2},"lawyer":{"chapterSlug":"agent-harnesses","lessonSlug":"building-a-harness","stepIndex":2}}}}]

building a harness — scaffold, tools, loop, gate — step 4 of 9