# The model is creative. The schema is honest. Lesson 02 wired the real Anthropic SDK shape. The next failure mode that bites in production is the one nobody warns you about: **sometimes the model invents arguments**. It calls your `search` tool with `{"query": "ramen"}` instead of `{"q": "ramen"}`. It passes a number where you expected a string. It calls a tool that doesn't exist. Without validation, your code does one of three bad things: 1. **`KeyError`** when you read `args["q"]` — crash, lose the conversation, page on-call. 2. **Silent wrong behavior** when the tool runs but with garbage — the model gets a confusing result and starts making things up. 3. **`AttributeError` at runtime** five frames deep, no clue what actually went wrong. Validation catches it at the boundary. The schema you give the model in `tools=[...]` is the same schema your code validates against before dispatching. If they ever drift, you find out *before* the tool runs, return a clean error message to the model as the tool result, and the model self-corrects. ## What this looks like in production In production code, validation is one line: ```python from pydantic import BaseModel class SearchInput(BaseModel): q: str # in the dispatch loop: try: args = SearchInput.model_validate(tool_use_block.input) except ValidationError as e: return f"invalid args: {e.errors()[0]['msg']}" ``` Pydantic generates the JSON Schema you send to the model AND the validator you run on the response. Same schema, both sides. We're not using Pydantic in this lesson. The point of the capstone is to *see* what Pydantic does so you understand what you're trusting. The hand-rolled `validate()` function in the editor is the bare-metal equivalent — twelve lines, no install, deterministic behavior. When you swap to real Pydantic in production, you delete this and replace with two lines. ## What you'll build A `dispatch_with_validation(tool_use_block)` that looks up the schema, validates the model's input, runs the tool only on valid input, and returns a `tool_result` block either way — never crashes, never raises, always closes the loop. Then the bug everyone hits: forgetting to convert a `ValidationError` into a tool result, leaving the loop in a broken state.36:T40e,# the model occasionally invents tool args. validation catches it # BEFORE your tool runs and produces an obscure crash. TOOL_SCHEMAS = { "search": {"required": ["q"], "types": {"q": str}}, "read_page": {"required": ["url"], "types": {"url": str}}, } def validate(tool_name, args): schema = TOOL_SCHEMAS.get(tool_name) if schema is None: return False, f"unknown tool: {tool_name}" for field in schema["required"]: if field not in args: return False, f"missing field: {field}" for field, expected in schema["types"].items(): if field in args and not isinstance(args[field], expected): return False, f"wrong type for {field}: expected {expected.__name__}" return True, "ok" print(validate("search", {"q": "ramen"})) # ok print(validate("search", {"query": "ramen"})) # missing field q print(validate("search", {"q": 42})) # wrong type print(validate("frobnicate", {"x": 1})) # unknown tool 37:Taa7,# Three checks every validator does A real Pydantic model encodes three things the runtime checks before your code sees the input. Hand-rolled or framework-generated, the shape is the same: ## 1. Required fields Did every required field arrive? `args["q"]` raises `KeyError` if not, so check up front and return a clean error message instead. ## 2. Types Is each present field the expected Python type? `isinstance(value, expected)` covers most cases — `str`, `int`, `float`, `bool`, `list`, `dict`. For more elaborate types (`list[int]`, optional fields, nested models) production code uses Pydantic's full type system; the bare-metal version covers the cases you actually see in tool inputs. ## 3. Value constraints Even when the type is right, the value can be wrong. `page_size: int` doesn't catch `page_size: 99999999`. Encode the constraint (`1 ≤ page_size ≤ 100`) and check it. In Pydantic, this is `Annotated[int, Field(ge=1, le=100)]` or a `@field_validator`. In the hand-rolled version above, it's a lambda that returns `(ok, message)`. ## What makes validation worth doing Validation isn't free. It runs on every tool call. The reason it's non-negotiable in production: **the alternative is debugging the agent loop from a stack trace five frames deep, while the user is waiting**. A model that called your tool with the wrong field name would otherwise crash silently inside the tool's first DB query — your trace shows "DatabaseError on row 0" and you'd waste an hour chasing the wrong layer. With validation, the trace says `tool_use search → invalid args: missing required field q`. You see exactly where it went wrong, and critically: **the model sees it too**, in the next `tool_result` block, and tries again with corrected input. That's the load-bearing piece. Validation isn't just a guard rail — it's a *teaching channel* back to the model. Return the error as the tool result, the model usually self-corrects on the next turn. Without it, you crash, lose context, and the user starts over. ## Where this fits in the loop ```python for block in response.content: if block.type == "tool_use": verdict = validate(block.input, TOOL_SCHEMAS[block.name]) if verdict["ok"]: output = TOOLS[block.name](**verdict["args"]) else: output = f"VALIDATION ERROR: {verdict['error']}" tool_results.append({ "type": "tool_result", "tool_use_id": block.id, "content": output, }) ``` The shape of the loop doesn't change. One `validate()` call between "the model said run this tool" and "we ran the tool." If the verdict is bad, the tool never runs and the model gets the error message instead.38:T51e,# production validators do three checks. teach all three with one function. def validate(args, schema): # 1. required fields present? for field in schema.get("required", []): if field not in args: return {"ok": False, "error": f"missing required field: {field}"} # 2. types match? for field, expected_type in schema.get("types", {}).items(): if field in args and not isinstance(args[field], expected_type): actual = type(args[field]).__name__ return {"ok": False, "error": f"{field} should be {expected_type.__name__}, got {actual}"} # 3. constraints satisfied? for field, check in schema.get("constraints", {}).items(): if field in args: ok, msg = check(args[field]) if not ok: return {"ok": False, "error": f"{field}: {msg}"} return {"ok": True, "args": args} schema = { "required": ["page_size"], "types": {"page_size": int}, "constraints": {"page_size": lambda v: (1 <= v <= 100, "must be 1..100")}, } print(validate({"page_size": 25}, schema)) # ok print(validate({"page_size": "25"}, schema)) # type error print(validate({"page_size": 999}, schema)) # constraint error print(validate({}, schema)) # missing field 39:T45a,def validate(args, schema): for field in schema.get("required", []): if field not in args: return {"ok": False, "error": f"missing required field: {field}"} for field, expected_type in schema.get("types", {}).items(): if field in args and not isinstance(args[field], expected_type): actual = type(args[field]).__name__ return {"ok": False, "error": f"{field} should be {expected_type.__name__}, got {actual}"} return {"ok": True, "args": args} TOOLS = {"search": lambda q: f"results for {q}"} SCHEMAS = {"search": {"required": ["q"], "types": {"q": str}}} def dispatch_with_validation(tool_name, args): schema = SCHEMAS.get(tool_name) if schema is None: return f"UNKNOWN_TOOL: {tool_name}" verdict = validate(args, schema) if not verdict["ok"]: return f"VALIDATION_ERROR: {verdict['error']}" return TOOLS[tool_name](**verdict["args"]) print(dispatch_with_validation("search", {"q": "ramen"})) print(dispatch_with_validation("search", {"query": "ramen"})) print(dispatch_with_validation("frobnicate", {"x": 1})) 3a:T407,Final drill. Build the agent loop with full validation. Write `run_validated_agent(question, max_iters)` that: - Maintains `messages = [{"role": "user", "content": question}]`. - Loops up to `max_iters` times calling `fake_create(messages)` → a `MockMessage` with `.stop_reason` and `.content` (a list of `MockTextBlock` and/or `MockToolUseBlock`). - On `end_turn`: collect text from text blocks, return `{"ok": True, "answer": , "validation_errors": , "iters": }`. - On `tool_use`: for each tool_use block, call `dispatch_with_validation(block.name, block.input)`. Track how many results came back as `VALIDATION_ERROR:` or `UNKNOWN_TOOL:` — increment a `validation_errors` counter per such result. Append the assistant turn + a user turn with `tool_result` blocks (each with the right `tool_use_id`). - On cap: return `{"ok": False, "error": "capped", "iters": max_iters}`. Expected output: ``` ok=True iters=3 validation_errors=1 answer=Found a 2026 ramen guide. ``` 3b:T85a,def validate(args, schema): for field in schema.get("required", []): if field not in args: return {"ok": False, "error": f"missing required field: {field}"} for field, expected_type in schema.get("types", {}).items(): if field in args and not isinstance(args[field], expected_type): actual = type(args[field]).__name__ return {"ok": False, "error": f"{field} should be {expected_type.__name__}, got {actual}"} return {"ok": True, "args": args} TOOLS = {"search": lambda q: f"results for {q}"} SCHEMAS = {"search": {"required": ["q"], "types": {"q": str}}} def dispatch_with_validation(tool_name, args): schema = SCHEMAS.get(tool_name) if schema is None: return f"UNKNOWN_TOOL: {tool_name}" verdict = validate(args, schema) if not verdict["ok"]: return f"VALIDATION_ERROR: {verdict['error']}" return TOOLS[tool_name](**verdict["args"]) class MockTextBlock: def __init__(self, text): self.type = "text"; self.text = text class MockToolUseBlock: def __init__(self, id, name, input): self.type = "tool_use"; self.id = id; self.name = name; self.input = input class MockMessage: def __init__(self, stop_reason, content): self.stop_reason = stop_reason; self.content = content responses = iter([ # turn 1: model calls search with the WRONG field name (validation error) MockMessage("tool_use", [MockToolUseBlock(id="t1", name="search", input={"query": "ramen"})]), # turn 2: model corrects itself based on the error message MockMessage("tool_use", [MockToolUseBlock(id="t2", name="search", input={"q": "ramen"})]), # turn 3: model wraps up MockMessage("end_turn", [MockTextBlock("Found a 2026 ramen guide.")]), ]) def fake_create(messages): return next(responses) # define run_validated_agent(question, max_iters) below result = run_validated_agent("find ramen", 5) if result.get("ok"): print(f"ok={result['ok']} iters={result['iters']} validation_errors={result['validation_errors']} answer={result['answer']}") else: print(f"ok={result['ok']} error={result['error']}") 3c:Tc50,def validate(args, schema): for field in schema.get("required", []): if field not in args: return {"ok": False, "error": f"missing required field: {field}"} for field, expected_type in schema.get("types", {}).items(): if field in args and not isinstance(args[field], expected_type): actual = type(args[field]).__name__ return {"ok": False, "error": f"{field} should be {expected_type.__name__}, got {actual}"} return {"ok": True, "args": args} TOOLS = {"search": lambda q: f"results for {q}"} SCHEMAS = {"search": {"required": ["q"], "types": {"q": str}}} def dispatch_with_validation(tool_name, args): schema = SCHEMAS.get(tool_name) if schema is None: return f"UNKNOWN_TOOL: {tool_name}" verdict = validate(args, schema) if not verdict["ok"]: return f"VALIDATION_ERROR: {verdict['error']}" return TOOLS[tool_name](**verdict["args"]) class MockTextBlock: def __init__(self, text): self.type = "text"; self.text = text class MockToolUseBlock: def __init__(self, id, name, input): self.type = "tool_use"; self.id = id; self.name = name; self.input = input class MockMessage: def __init__(self, stop_reason, content): self.stop_reason = stop_reason; self.content = content responses = iter([ MockMessage("tool_use", [MockToolUseBlock(id="t1", name="search", input={"query": "ramen"})]), MockMessage("tool_use", [MockToolUseBlock(id="t2", name="search", input={"q": "ramen"})]), MockMessage("end_turn", [MockTextBlock("Found a 2026 ramen guide.")]), ]) def fake_create(messages): return next(responses) def run_validated_agent(question, max_iters): messages = [{"role": "user", "content": question}] validation_errors = 0 for i in range(1, max_iters + 1): response = fake_create(messages) if response.stop_reason == "end_turn": text_parts = [b.text for b in response.content if b.type == "text"] return { "ok": True, "answer": " ".join(text_parts), "validation_errors": validation_errors, "iters": i, } tool_results = [] for block in response.content: if block.type == "tool_use": output = dispatch_with_validation(block.name, block.input) if output.startswith("VALIDATION_ERROR:") or output.startswith("UNKNOWN_TOOL:"): validation_errors += 1 tool_results.append({ "type": "tool_result", "tool_use_id": block.id, "content": output, }) messages.append({"role": "assistant", "content": response.content}) messages.append({"role": "user", "content": tool_results}) return {"ok": False, "error": "capped", "iters": max_iters} result = run_validated_agent("find ramen", 5) if result.get("ok"): print(f"ok={result['ok']} iters={result['iters']} validation_errors={result['validation_errors']} answer={result['answer']}") else: print(f"ok={result['ok']} error={result['error']}") 3d:Tb1a,# An agent without measurement is a bet, not a feature Lesson 03 made the agent reject bad inputs. This lesson makes the agent measurable. Two pieces, both load-bearing in production: ## 1. Trace — record everything, every turn A trace is the structured record of every turn the agent took: which tool was called, what it returned, what `stop_reason` came back, how many tokens flowed in and out. Chapter 20 explained the *shape* of traces. This lesson wires one into your agent. The minimum viable trace, per turn: ```python { "turn": 1, "stop_reason": "tool_use", "tools_called": ["search"], "input_tokens": 412, "output_tokens": 38, "validation_errors": 0, } ``` That's enough to answer the four questions you'll be asked when something breaks: how many turns did it take, what stopped it, what tools ran, what did it cost. Production tracers (LangSmith, Helicone, Phoenix) capture more — span trees, full inputs/outputs, prompt diffs across runs — but the shape above is what every single one of them encodes. ## 2. Eval suite — assert on outputs, automate the loop Chapter 21 ranked four eval patterns. The agent doesn't change which pattern you pick — the agent's *output* is just a string and the suite runs the same way. What changes: the cases. An eval case for an agent is `(question, expected_property)`, where `expected_property` might be: - **exact** — the answer string equals a known reference - **contains** — the answer contains a required substring ("Paris", "$199.99", `2024-04-15`) - **schema** — the answer matches a JSON schema (when the agent returns structured output) - **judge** — an LLM-as-judge (lesson 21-02) verifies semantic correctness For an agent capstone, the practical default is `contains` for fact-recall cases plus an LLM-as-judge for "did it sound like a human did this." Two patterns covers 90% of capstone needs. ## What `assert` you write, and where The eval suite calls the agent with a question, gets back the result + trace, then asserts properties of *both*: ```python result = run_agent("capital of France?") assert "Paris" in result["answer"] # output check assert len(result["trace"]) <= 3 # turn cap assert result["trace"][-1]["stop_reason"] == "end_turn" # didn't error out ``` Three assertions, three different layers. Output correctness, turn budget, exit cleanliness. Real production suites have 50–200 cases; the principle is the same. ## What you'll build A `run_agent_with_trace(question)` that emits a trace per turn, then `run_eval_suite(cases)` that runs the agent on each case and asserts both output and trace properties. Then the bug everyone hits: writing an eval that just re-asserts what the agent did (tautological), instead of checking against an independent ground truth.3e:T492,# one trace record per turn. one eval suite over the whole agent. # together: an agent you can ship without crossing your fingers. def run_agent_with_trace(question, fake_model, tools): messages = [{"role": "user", "content": question}] trace = [] for i in range(5): response = fake_model(messages) trace.append({ "turn": i + 1, "stop_reason": response["stop_reason"], "input_tokens": response["usage"]["input_tokens"], "output_tokens": response["usage"]["output_tokens"], }) if response["stop_reason"] == "end_turn": return {"answer": response["content"][0]["text"], "trace": trace} # tool dispatch elided return {"answer": "capped", "trace": trace} def fake_model(messages): return {"stop_reason": "end_turn", "content": [{"type": "text", "text": "Paris"}], "usage": {"input_tokens": 30, "output_tokens": 5}} result = run_agent_with_trace("capital of France?", fake_model, {}) print(f"answer: {result['answer']}") print(f"turns: {len(result['trace'])}") print(f"tokens: {sum(t['input_tokens'] + t['output_tokens'] for t in result['trace'])}") 3f:Tb8f,# Six fields, no more A real LangSmith run has dozens of fields per span. A real Phoenix trace has tree structure with parent/child relationships. But for the *bare-metal version that teaches what's underneath*, six fields per turn cover every question you'll be asked at 2am: | Field | Question it answers | |---|---| | `turn` | Which step in the loop are we looking at? | | `stop_reason` | Why did the model stop here? | | `tools_called` | What did the model do this turn? | | `input_tokens` | How much did we send? | | `output_tokens` | How much came back? | | `validation_errors` | How many tool calls were rejected before running? | Sum across all turns and you have cost (in/out tokens), latency proxy (turn count), reliability (validation error rate), and tool usage profile. ## Append-only is the contract The trace list is **append-only** during a run. You add records as turns complete. You never modify a record after it's recorded. You never remove a record. You never reorder. This isn't religious. It's because: 1. **Debugging needs a faithful timeline.** If turn 3 looks weird, turn 3's record at the time it happened is what you need — not turn 3 retroactively cleaned up. 2. **Mutation breaks shared traces.** If your eval suite passes the same `trace` list to multiple consumers (a logger, a dashboard, a unit test), mutation by one corrupts the others. 3. **It maps cleanly to streaming exports.** A trace that's append-only is the same shape as a JSONL file growing on disk or a stream of log lines piped to LangSmith. The append-only contract is also why production code uses **deep copies** when adding records — never the live mutable object itself. Step 6 fixes the bug where a beginner appends a reference and watches subsequent turns retroactively edit history. ## Reading the trace, after the fact The eval suite reads the trace to verify behavior: ```python result = run_agent("capital of France?") trace = result["trace"] # how many turns? assert len(trace) <= 3 # did it exit cleanly? assert trace[-1]["stop_reason"] == "end_turn" # how many tools did it call total? total_tools = sum(len(t["tools_called"]) for t in trace) assert total_tools <= 2 # was the answer correct? assert "Paris" in result["answer"] ``` Four assertions, three of them about the *trace*, one about the *answer*. Pure-output evals check what the agent said. Trace evals check *how* the agent got there. Production suites need both — because you can ship an agent that gives the right answer the expensive way, and find out about the bill at the end of the month. ## Why this is the capstone-shaped lesson Chapters 20 and 21 each taught one half of this — chapter 20 gave you trace literacy, chapter 21 gave you eval discipline. This lesson wires them through your own agent code, in your own language. The eval suite you write here is the difference between "I built an agent" and "I built an agent that ships."40:T50b,# Simplified trace fixture: each entry is one model turn. In a real Anthropic # trace, intermediate turns end on `tool_use` (you run the tool, append the # result, loop) and only the final turn ends on `end_turn`. We collapse that # to one stop_reason per entry so the eval logic stays focused on the count # and the last-entry check. AGENT_BEHAVIOR = { "capital of France?": {"answer": "Paris is the capital.", "trace": [{"stop_reason": "end_turn"}]}, "capital of Germany?": {"answer": "Berlin is the capital.", "trace": [{"stop_reason": "tool_use"}, {"stop_reason": "tool_use"}, {"stop_reason": "tool_use"}, {"stop_reason": "end_turn"}]}, # too many turns "capital of Japan?": {"answer": "Tokyo is the capital.", "trace": [{"stop_reason": "end_turn"}]}, } def run_agent(question): return AGENT_BEHAVIOR[question] cases = [ {"question": "capital of France?", "expected": "Paris", "max_turns": 3}, {"question": "capital of Germany?", "expected": "Berlin", "max_turns": 3}, {"question": "capital of Japan?", "expected": "Tokyo", "max_turns": 3}, ] # define run_eval_suite(cases) below result = run_eval_suite(cases) print(f"total={result['total']} passed={result['passed']} failed={result['failed']} pass_rate={result['pass_rate']}") 41:T601,AGENT_BEHAVIOR = { "capital of France?": {"answer": "Paris is the capital.", "trace": [{"stop_reason": "end_turn"}]}, "capital of Germany?": {"answer": "Berlin is the capital.", "trace": [{"stop_reason": "tool_use"}, {"stop_reason": "tool_use"}, {"stop_reason": "tool_use"}, {"stop_reason": "end_turn"}]}, "capital of Japan?": {"answer": "Tokyo is the capital.", "trace": [{"stop_reason": "end_turn"}]}, } def run_agent(question): return AGENT_BEHAVIOR[question] cases = [ {"question": "capital of France?", "expected": "Paris", "max_turns": 3}, {"question": "capital of Germany?", "expected": "Berlin", "max_turns": 3}, {"question": "capital of Japan?", "expected": "Tokyo", "max_turns": 3}, ] def run_eval_suite(cases): passed_count = 0 for c in cases: result = run_agent(c["question"]) ok_answer = c["expected"] in result["answer"] ok_turns = len(result["trace"]) <= c["max_turns"] ok_exit = result["trace"][-1]["stop_reason"] == "end_turn" if ok_answer and ok_turns and ok_exit: passed_count += 1 total = len(cases) failed_count = total - passed_count pass_rate = round(passed_count / total, 2) if total else 0.0 return { "total": total, "passed": passed_count, "failed": failed_count, "pass_rate": pass_rate, } result = run_eval_suite(cases) print(f"total={result['total']} passed={result['passed']} failed={result['failed']} pass_rate={result['pass_rate']}") 42:T4b4,RESPONSES = { "capital of France?": [ {"stop_reason": "end_turn", "text": "Paris is the capital.", "usage": {"input_tokens": 30, "output_tokens": 5}}, ], "capital of Atlantis?": [ {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 30, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 35, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 40, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 45, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 50, "output_tokens": 10}}, ], } def fake_create(question, turn_index): return RESPONSES[question][turn_index] cases = [ {"question": "capital of France?", "expected": "Paris", "max_turns": 3}, {"question": "capital of Atlantis?", "expected": "Poseidonis", "max_turns": 3}, ] # define run_agent(question) below # define run_eval_suite(cases) below result = run_eval_suite(cases) print(f"total={result['total']} passed={result['passed']} failed={result['failed']} pass_rate={result['pass_rate']}") 43:T8a2,RESPONSES = { "capital of France?": [ {"stop_reason": "end_turn", "text": "Paris is the capital.", "usage": {"input_tokens": 30, "output_tokens": 5}}, ], "capital of Atlantis?": [ {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 30, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 35, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 40, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 45, "output_tokens": 10}}, {"stop_reason": "tool_use", "text": "", "usage": {"input_tokens": 50, "output_tokens": 10}}, ], } def fake_create(question, turn_index): return RESPONSES[question][turn_index] cases = [ {"question": "capital of France?", "expected": "Paris", "max_turns": 3}, {"question": "capital of Atlantis?", "expected": "Poseidonis", "max_turns": 3}, ] def run_agent(question): trace = [] for i in range(5): response = fake_create(question, i) record = { "turn": i + 1, "stop_reason": response["stop_reason"], "tokens": response["usage"]["input_tokens"] + response["usage"]["output_tokens"], } trace.append(record) if response["stop_reason"] == "end_turn": return {"answer": response["text"], "trace": trace} return {"answer": "capped", "trace": trace} def run_eval_suite(cases): passed_count = 0 for c in cases: result = run_agent(c["question"]) ok_answer = c["expected"] in result["answer"] ok_turns = len(result["trace"]) <= c["max_turns"] ok_exit = result["trace"][-1]["stop_reason"] == "end_turn" if ok_answer and ok_turns and ok_exit: passed_count += 1 total = len(cases) return { "total": total, "passed": passed_count, "failed": total - passed_count, "pass_rate": round(passed_count / total, 2) if total else 0.0, } result = run_eval_suite(cases) print(f"total={result['total']} passed={result['passed']} failed={result['failed']} pass_rate={result['pass_rate']}") 44:Taae,# Tools that come from somewhere else Lessons 01–04 built an agent whose tools live in your Python code — a `TOOLS` dict mapping name to function. That works until you want to add a tool *without* deploying. Or share a tool across multiple agents. Or let a third party publish a tool you can install like a plugin. That's what MCP — the Model Context Protocol — solves. Chapter 15 introduced it. This lesson wires it through your agent. ## The shape, in 30 seconds **Three primitives** in MCP: tools, resources, prompts. We're using tools. Each MCP server exposes: - `tools/list` — "what tools do you have?" - `tools/call` — "run this tool with these arguments" Both calls are JSON-RPC 2.0. The server runs as a separate process (stdio transport) or over HTTP (Streamable HTTP transport). Your agent doesn't care which — it just makes the call. ## Why use MCP instead of a local `TOOLS` dict? Three real reasons, in increasing order of importance: 1. **Plugin model.** A user installs a Slack MCP server, you instantly have `slack_post_message` and `slack_search` available to your agent. No code changes. 2. **Cross-language.** Your agent is Python; the MCP server can be TypeScript, Go, Rust. Same protocol, doesn't matter. 3. **Sandboxing.** The MCP server runs in its own process with its own filesystem/network access. If the agent makes a tool do something malicious, the blast radius is the server, not your whole app. The cost: one network hop per tool call (or one IPC over stdio). For high-volume tools you'll keep a local fast path; for the long tail (one-off integrations), MCP is the right bet. ## What "wiring it" means in code Your existing tool registry is `TOOLS = {"search": search_fn}` and `SCHEMAS = {"search": {...}}`. You bridge MCP by: 1. **Discovery**: call `tools/list` once at startup, get back a list of tool definitions. 2. **Registration**: convert each MCP tool into your registry format. Names match. `inputSchema` maps to your validator schema. The "function" becomes a closure that calls the MCP server's `tools/call`. 3. **Dispatch**: when the agent's loop wants to run `search`, the closure makes a `tools/call` request and returns the result. The agent loop *doesn't change*. It still calls `TOOLS[name](**args)`. The function just happens to be a network call now. ## What you'll build A `mcp_tools_to_registry(tools_list)` function that converts a JSON-RPC `tools/list` response into your `TOOLS` and `SCHEMAS` dicts. Then a fix for the most common bug: reading `response["content"]` directly when the real shape wraps it inside `response["result"]`. Then a fix for ignoring the `isError` flag that MCP servers set when a tool fails.45:Ta78,# Two endpoints, one wrapping shape The whole protocol is JSON-RPC 2.0. Two methods do all the work: `tools/list` and `tools/call`. Three things to lock in. ## 1. `tools/list` — discovery ```json → {"jsonrpc":"2.0","id":1,"method":"tools/list"} ← {"jsonrpc":"2.0","id":1,"result":{"tools":[ {"name":"search","description":"...","inputSchema":{...}}, ... ]}} ``` Note `inputSchema` (camelCase, not `input_schema`). MCP is Python-team-staffed but the protocol is camelCase on the wire. ## 2. `tools/call` — invocation ```json → {"jsonrpc":"2.0","id":42,"method":"tools/call", "params":{"name":"search","arguments":{"q":"ramen"}}} ← {"jsonrpc":"2.0","id":42,"result":{ "content":[{"type":"text","text":"3 results"}], "isError":false }} ``` Three fields to remember in the response: - **`result.content`** — a *list* of content blocks, like the Anthropic message content shape from chapter 13. Each block has a `type` (`text`, `image`, `resource`). For most tools you'll iterate the list and concatenate the text fields. - **`result.isError`** — boolean. `True` means the tool ran, the protocol succeeded, but the tool reported failure. The `content` in this case is the error message. Treat it like a tool that returned an error string. - **The wrap matters.** Beginners read `response["content"]` expecting the content list directly. The list is one level deeper — inside `response["result"]`. This is the bug step 6 fixes. ## 3. JSON-RPC errors are different from tool errors There are TWO kinds of errors: | Kind | Where it appears | When it fires | |---|---|---| | **Protocol error** | `response["error"]` (no `result` key) | Method doesn't exist, malformed JSON-RPC | | **Tool error** | `response["result"]["isError"] == True` | Tool ran but reported failure | Tool errors are *normal* — the user mistyped a query, an upstream API is down, the model passed bad args. Protocol errors are *broken* — your code or the server has a bug. Step 7 fixes the bug where code ignores `isError` and treats every non-protocol-error response as success — feeding actual error messages to the model as successful tool results, which makes the agent confidently relay nonsense. ## What this maps to in your registry The bridge function turns each MCP tool definition into: ```python TOOLS["search"] = lambda **args: call_mcp("search", args) SCHEMAS["search"] = mcp_tool["inputSchema"] ``` The agent loop doesn't change. It still does `TOOLS[name](**args)` and `validate(args, SCHEMAS[name])`. The only new thing is the closure inside `call_mcp` that makes the JSON-RPC request and unwraps `result.content`/`isError`.46:T5b8,tools_list_response = { "jsonrpc": "2.0", "id": 1, "result": { "tools": [ {"name": "search", "description": "Web search.", "inputSchema": {"required": ["q"]}}, {"name": "read_url", "description": "Fetch a page.", "inputSchema": {"required": ["url"]}}, {"name": "summarize","description": "Compress text.","inputSchema": {"required": ["text"]}}, ], }, } def mcp_call(tool_name, args): return { "jsonrpc": "2.0", "id": 99, "result": { "content": [{"type": "text", "text": f"3 results for {args.get('q', '?')}"}], "isError": False, }, } def bridge_mcp_tools(tools_list_response, mcp_call): tools = {} schemas = {} for tool_def in tools_list_response["result"]["tools"]: name = tool_def["name"] schemas[name] = tool_def["inputSchema"] def make_closure(tool_name): def call(**args): response = mcp_call(tool_name, args) text = response["result"]["content"][0]["text"] if response["result"]["isError"]: text = f"TOOL_ERROR: {text}" return text return call tools[name] = make_closure(name) return tools, schemas tools, schemas = bridge_mcp_tools(tools_list_response, mcp_call) print(f"tool_names: {list(tools.keys())}") print(f"search_result: {tools['search'](q='ramen')}") 47:T50b,Final drill of the capstone. Build the agent loop running with MCP-sourced tools. Write `run_mcp_agent(question, tools_list_response, mcp_call, fake_model, max_iters)` that: - Bridges `tools_list_response` via `bridge_mcp_tools(...)` (provided) into `tools, schemas`. - Loops up to `max_iters` times calling `fake_model(messages, list(tools.keys()))`. - The model returns a dict like `{"stop_reason": "...", "content": [...]}` where each content block is either `{"type": "text", "text": "..."}` or `{"type": "tool_use", "id": "...", "name": "...", "input": {...}}`. - On `end_turn`: collect text blocks, return `{"ok": True, "answer": , "iters": , "tool_calls": , "tool_errors": }`. - On `tool_use`: dispatch each tool_use block through `tools[block["name"]](**block["input"])`. Count any result that starts with `"TOOL_ERROR:"` as a tool error. Append assistant + user turns the standard way. - On cap: return `{"ok": False, "error": "capped", "iters": max_iters}`. Two cases run for you. Expected output: ``` ok=True iters=2 tool_calls=1 tool_errors=0 answer=Found Tokyo's best ramen. ok=True iters=2 tool_calls=1 tool_errors=1 answer=I couldn't search; tool was down. ``` 48:T9af,def bridge_mcp_tools(tools_list_response, mcp_call): tools = {} schemas = {} for tool_def in tools_list_response["result"]["tools"]: name = tool_def["name"] schemas[name] = tool_def["inputSchema"] def make_closure(tool_name): def call(**args): response = mcp_call(tool_name, args) text = response["result"]["content"][0]["text"] if response["result"]["isError"]: text = f"TOOL_ERROR: {text}" return text return call tools[name] = make_closure(name) return tools, schemas tools_list_response = { "jsonrpc": "2.0", "id": 1, "result": {"tools": [{"name": "search", "description": "Web search.", "inputSchema": {"required": ["q"]}}]}, } # mcp_call success on case 1, failure on case 2 case_state = {"mode": "ok"} def mcp_call(tool_name, args): if case_state["mode"] == "ok": return {"jsonrpc": "2.0", "id": 99, "result": {"content": [{"type": "text", "text": f"hits for {args.get('q', '?')}"}], "isError": False}} return {"jsonrpc": "2.0", "id": 99, "result": {"content": [{"type": "text", "text": "search down"}], "isError": True}} ok_responses = iter([ {"stop_reason": "tool_use", "content": [{"type": "tool_use", "id": "t1", "name": "search", "input": {"q": "ramen"}}]}, {"stop_reason": "end_turn", "content": [{"type": "text", "text": "Found Tokyo's best ramen."}]}, ]) fail_responses = iter([ {"stop_reason": "tool_use", "content": [{"type": "tool_use", "id": "f1", "name": "search", "input": {"q": "ramen"}}]}, {"stop_reason": "end_turn", "content": [{"type": "text", "text": "I couldn't search; tool was down."}]}, ]) current = {"iter": ok_responses} def fake_model(messages, available_tool_names): return next(current["iter"]) # define run_mcp_agent(question, tools_list_response, mcp_call, fake_model, max_iters) below case_state["mode"] = "ok" current["iter"] = ok_responses r = run_mcp_agent("find ramen", tools_list_response, mcp_call, fake_model, 5) print(f"ok={r['ok']} iters={r['iters']} tool_calls={r['tool_calls']} tool_errors={r['tool_errors']} answer={r['answer']}") case_state["mode"] = "fail" current["iter"] = fail_responses r = run_mcp_agent("find ramen", tools_list_response, mcp_call, fake_model, 5) print(f"ok={r['ok']} iters={r['iters']} tool_calls={r['tool_calls']} tool_errors={r['tool_errors']} answer={r['answer']}") 49:Tea0,def bridge_mcp_tools(tools_list_response, mcp_call): tools = {} schemas = {} for tool_def in tools_list_response["result"]["tools"]: name = tool_def["name"] schemas[name] = tool_def["inputSchema"] def make_closure(tool_name): def call(**args): response = mcp_call(tool_name, args) text = response["result"]["content"][0]["text"] if response["result"]["isError"]: text = f"TOOL_ERROR: {text}" return text return call tools[name] = make_closure(name) return tools, schemas tools_list_response = { "jsonrpc": "2.0", "id": 1, "result": {"tools": [{"name": "search", "description": "Web search.", "inputSchema": {"required": ["q"]}}]}, } case_state = {"mode": "ok"} def mcp_call(tool_name, args): if case_state["mode"] == "ok": return {"jsonrpc": "2.0", "id": 99, "result": {"content": [{"type": "text", "text": f"hits for {args.get('q', '?')}"}], "isError": False}} return {"jsonrpc": "2.0", "id": 99, "result": {"content": [{"type": "text", "text": "search down"}], "isError": True}} ok_responses = iter([ {"stop_reason": "tool_use", "content": [{"type": "tool_use", "id": "t1", "name": "search", "input": {"q": "ramen"}}]}, {"stop_reason": "end_turn", "content": [{"type": "text", "text": "Found Tokyo's best ramen."}]}, ]) fail_responses = iter([ {"stop_reason": "tool_use", "content": [{"type": "tool_use", "id": "f1", "name": "search", "input": {"q": "ramen"}}]}, {"stop_reason": "end_turn", "content": [{"type": "text", "text": "I couldn't search; tool was down."}]}, ]) current = {"iter": ok_responses} def fake_model(messages, available_tool_names): return next(current["iter"]) def run_mcp_agent(question, tools_list_response, mcp_call, fake_model, max_iters): tools, schemas = bridge_mcp_tools(tools_list_response, mcp_call) messages = [{"role": "user", "content": question}] tool_calls = 0 tool_errors = 0 for i in range(1, max_iters + 1): response = fake_model(messages, list(tools.keys())) if response["stop_reason"] == "end_turn": text_parts = [b["text"] for b in response["content"] if b["type"] == "text"] return { "ok": True, "answer": " ".join(text_parts), "iters": i, "tool_calls": tool_calls, "tool_errors": tool_errors, } tool_results = [] for block in response["content"]: if block["type"] == "tool_use": tool_calls += 1 output = tools[block["name"]](**block["input"]) if output.startswith("TOOL_ERROR:"): tool_errors += 1 tool_results.append({ "type": "tool_result", "tool_use_id": block["id"], "content": output, }) messages.append({"role": "assistant", "content": response["content"]}) messages.append({"role": "user", "content": tool_results}) return {"ok": False, "error": "capped", "iters": max_iters} case_state["mode"] = "ok" current["iter"] = ok_responses r = run_mcp_agent("find ramen", tools_list_response, mcp_call, fake_model, 5) print(f"ok={r['ok']} iters={r['iters']} tool_calls={r['tool_calls']} tool_errors={r['tool_errors']} answer={r['answer']}") case_state["mode"] = "fail" current["iter"] = fail_responses r = run_mcp_agent("find ramen", tools_list_response, mcp_call, fake_model, 5) print(f"ok={r['ok']} iters={r['iters']} tool_calls={r['tool_calls']} tool_errors={r['tool_errors']} answer={r['answer']}") 4a:Tac3,# The other capstone — a tiny ML system, all six stages present You've shipped the agent. Loop, tools, validation, evals, MCP — lessons 01 through 05, done. This last lesson is the fork in the road: if the ML track is where you're headed — model training, pipelines, the "ML engineer" job description — here's the second capstone to build, and the last thing this course will ask of you. It is not "train a model." Anyone can train a model; a tutorial notebook trains a model. **A system is a model wearing a pipeline**, and the pipeline has exactly six stages: 1. **Data pull** — a script that fetches the raw data and snapshots it. Re-runnable, dated, boring. 2. **Validation** — a schema check that *refuses* rows instead of hoping. Chapter 24 taught you where bad predictions come from; most are data problems, and this stage is where you catch them. 3. **Train + eval** — fit the model, then score it on data it never saw. The held-out split is the whole point; the training score is a vanity metric. 4. **Gate** — one comparison: eval score against a threshold you wrote down *before* training. Below the bar, the model does not ship. No exceptions, not even for demo day. 5. **Serving stub** — one function, features in, prediction out. Not a Kubernetes cluster. A function you could put behind a route in an afternoon. 6. **Monitoring hook** — every prediction logged with its input. When a bad prediction lands next Tuesday, this is the trace you'll triage from. ## How this fits in a weekend Scope is the trap, so pre-shrink everything. One public dataset you can download in under a minute. One boring model — logistic regression is a fine choice and a better signal than a half-understood transformer. One metric. Saturday: stages 1-3. Sunday morning: the gate and the stub. Sunday afternoon: the monitoring hook and a README with a single run command. If a stage is threatening to eat the weekend, shrink the data or the model — never delete the stage. ## What reviewers actually look at Here's the part nobody tells you: the reviewer skimming your repo spends about ninety seconds and **never reads your model code**. They've seen a thousand `model.fit()` calls. What they look for is judgment — does the eval use a held-out set, does a gate exist, does anything log at serving time, does the README have one command that runs the whole thing. A mediocre model inside all six stages beats a clever model inside none, every single time. The clever model says you can follow a tutorial. The gate says you can be trusted with a pipeline that touches real users. Run the editor. Six lines of table-of-contents now; by Sunday night, six working stages with your name on the repo.4b:T4d9,REQUIRED = [ "data pull", "validation", "train + eval", "gate", "serving stub", "monitoring hook", ] systems = [ { "name": "churn-predictor", "stages": ["data pull", "validation", "train + eval", "gate", "serving stub", "monitoring hook"], "eval_accuracy": 0.84, "gate_threshold": 0.80, }, { "name": "price-model", "stages": ["data pull", "validation", "train + eval", "gate", "serving stub"], "eval_accuracy": 0.91, "gate_threshold": 0.80, }, { "name": "spam-filter", "stages": ["data pull", "validation", "train + eval", "gate", "serving stub", "monitoring hook"], "eval_accuracy": 0.74, "gate_threshold": 0.80, }, ] def readiness_report(system): present = sum(1 for stage in REQUIRED if stage in system["stages"]) gate = "ship" if system["eval_accuracy"] >= system["gate_threshold"] else "hold" ready = present == len(REQUIRED) and gate == "ship" return {"stages_present": present, "gate": gate, "portfolio_ready": ready} for s in systems: r = readiness_report(s) print(f"{s['name']}: {r['stages_present']}/6 stages, gate {r['gate']}, ready: {r['portfolio_ready']}") 16:[["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"$1e"}}],["$","$L1f",null,{"tree":{"toc":{"version":"0.1.0","generatedAt":"1970-01-01T00:00:00.000Z","chapters":[{"number":0,"slug":"before-you-build","title":"from worker to builder","blurb":"you are not here to become a programmer overnight. you are here to stop doing repeated work by hand. this chapter is the map: the builder loop, old workplace deliverables turned into site artifacts, and your first reusable tool.","status":"live","lessonCount":10,"stepCount":14,"liveLessonCount":10,"liveStepCount":14,"estMinutes":74,"xpTotal":36},{"number":1,"slug":"read-the-script","title":"read the script — your first code-reading lab","blurb":"ai hands you code before you feel ready to read it. this lab covers the three reads that matter first: what the names hold, whether the function actually returns, and which branch of the if-chain really fires.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":21,"xpTotal":78},{"number":2,"slug":"read-the-data-shapes","title":"read the data shapes — the json ai just dumped","blurb":"every api response, config file, and llm reply is lists and dicts nested in a tree. read the shape on sight, walk it with a loop, and catch the wrong-layer bug ai loves to ship.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":23,"xpTotal":80},{"number":3,"slug":"read-the-failure","title":"read the failure — cursor wrote this and crashed","blurb":"when python crashes, it tells you exactly what happened and where. most non-engineers panic at the wall of text — and ai 'fixes' the crash by hiding it. learn to read the failure, then catch only what's worth catching.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":24,"xpTotal":83},{"number":4,"slug":"read-the-pipeline","title":"read the pipeline — files in, api calls out","blurb":"every real script reads a file or calls an api — usually both. learn the with-block, the status-code families, and the retry rules: the seatbelts ai forgets to put on.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":26,"xpTotal":83},{"number":7,"slug":"mutation-and-state","title":"mutation — why your code mysteriously breaks","blurb":"when a list inside a function changes the list outside the function, that's mutation. ai does this constantly without flagging it, and it's the bug class that takes the longest to find.","status":"live","lessonCount":2,"stepCount":9,"liveLessonCount":2,"liveStepCount":9,"estMinutes":10,"xpTotal":32},{"number":13,"slug":"llm-apis","title":"llm apis — talking to claude and openai","blurb":"every ai feature you ship eventually calls a model api. learn the messages pattern, how to read the response, and the four lines ai writes every single time.","status":"live","lessonCount":4,"stepCount":27,"liveLessonCount":4,"liveStepCount":27,"estMinutes":45,"xpTotal":87},{"number":14,"slug":"structured-output","title":"structured output — making the model return json","blurb":"free-form text breaks every pipeline. learn the schema-first pattern ai uses to get reliable json back, validate it with pydantic, and catch the model's lies before they hit prod.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":38,"xpTotal":80},{"number":15,"slug":"mcp","title":"mcp — the model context protocol","blurb":"mcp is the new standard for plugging tools and data sources into ai agents. learn what an mcp server actually is, how claude code lists tools, and why this is replacing one-off integrations everywhere.","status":"live","lessonCount":3,"stepCount":26,"liveLessonCount":3,"liveStepCount":26,"estMinutes":40,"xpTotal":85},{"number":16,"slug":"agent-loops","title":"agent loops — tool_use and the request/tool/respond cycle","blurb":"an agent isn't magic. it's a while loop. learn the actual cycle claude code, cursor, and every other agent uses: model returns tool_use, you run the tool, you send the result back, repeat until end_turn.","status":"live","lessonCount":6,"stepCount":48,"liveLessonCount":6,"liveStepCount":48,"estMinutes":71,"xpTotal":154},{"number":17,"slug":"git-and-github","title":"git and github cli — the ai builder's actual workflow","blurb":"cursor and claude code commit on your behalf. reading those commits — and undoing the bad ones — is your job. learn the three-state model, the commands you'll run every day, and what `gh` does that `git` can't.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":36,"xpTotal":79},{"number":18,"slug":"secrets-and-env","title":"secrets — .env, api keys, and what not to commit","blurb":"ai ships keys to github all the time. learn the .env pattern, why os.getenv is non-negotiable, what to do when a key leaks, and the gitignore lines you need on day one.","status":"live","lessonCount":2,"stepCount":16,"liveLessonCount":2,"liveStepCount":16,"estMinutes":16,"xpTotal":58},{"number":19,"slug":"prompting","title":"prompting cursor and claude code effectively","blurb":"the difference between a one-shot ai session and a four-hour debugging spiral is almost always the first prompt. learn the structure that gets you usable code.","status":"live","lessonCount":5,"stepCount":36,"liveLessonCount":5,"liveStepCount":36,"estMinutes":59,"xpTotal":118},{"number":20,"slug":"agent-traces","title":"reading agent traces and telemetry","blurb":"when an agent fails, the trace tells you exactly where. learn to read tool calls, tool results, and stop reasons — the json breadcrumbs every agent leaves behind.","status":"live","lessonCount":3,"stepCount":25,"liveLessonCount":3,"liveStepCount":25,"estMinutes":30,"xpTotal":88},{"number":21,"slug":"evals","title":"eval-driven ai development","blurb":"if you can't test it, you can't ship it. learn the simple-but-strict eval patterns that separate ai features that work from ones that just feel like they do.","status":"live","lessonCount":5,"stepCount":34,"liveLessonCount":5,"liveStepCount":34,"estMinutes":57,"xpTotal":112},{"number":22,"slug":"context-and-retrieval","title":"context and retrieval — feeding the model real data","blurb":"rag without the overengineering. chunking, embeddings, vector search, and the small set of patterns that make a model answer from your data instead of its training set.","status":"live","lessonCount":5,"stepCount":44,"liveLessonCount":5,"liveStepCount":44,"estMinutes":75,"xpTotal":144},{"number":23,"slug":"production-tradeoffs","title":"production tradeoffs — cost, latency, quality","blurb":"the three numbers every shipped llm feature lives or dies by. token math, caching, streaming, batching, and the small set of decisions that move the product more than a model swap ever will.","status":"live","lessonCount":4,"stepCount":33,"liveLessonCount":4,"liveStepCount":33,"estMinutes":57,"xpTotal":111},{"number":24,"slug":"debugging-output","title":"debugging broken ai output","blurb":"when the model lies to your customer. the methodology for narrowing down what went wrong, the four most-common breakage classes, and the discipline that separates 'we shipped a fix' from 'we blamed the model and shrugged'.","status":"live","lessonCount":4,"stepCount":34,"liveLessonCount":4,"liveStepCount":34,"estMinutes":61,"xpTotal":112},{"number":25,"slug":"capstone","title":"capstone — ship the system","blurb":"wire it all together. the prompt, the call, the validation, the trace, the eval, the MCP tool. less a tutorial demo, more the smallest end-to-end llm feature you could ship to a real user. (retrieval and prompt-cache cost work live in chapters 22-23 — extend the capstone with them when you scale past the demo input set.)","status":"live","lessonCount":7,"stepCount":62,"liveLessonCount":7,"liveStepCount":62,"estMinutes":100,"xpTotal":219},{"number":26,"slug":"agent-harnesses","title":"agent harnesses — the layer between you and the raw api","blurb":"claude code, cursor, aider, codex cli — they're all the same four layers wrapped around the same model api. learn what those layers are, what each adds, and what you'd build yourself if you had to.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":55,"xpTotal":87},{"number":27,"slug":"ai-image-generation","title":"ai image generation — from prompt to production asset","blurb":"the 2026 image model landscape, the prompts that work, and the pipeline that turns one good idea into a hundred ready-to-ship images. nano banana 2 for volume, nano banana pro for fidelity, flux, midjourney, ideogram, gpt-image-2. when each wins and what they cost.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":60,"xpTotal":67},{"number":28,"slug":"ai-video-generation","title":"ai video generation — omni, veo 3.1, sora, and the second wave","blurb":"video is the hardest content type to generate, the most expensive, and the most strategically interesting. learn the 2026 model lineup, the camera-control patterns that separate slop from craft, and the cost math that decides whether your idea is viable.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":50,"xpTotal":61},{"number":29,"slug":"programmatic-design","title":"programmatic design — code-driven video and the new design pipeline","blurb":"ai generates raw assets; code stitches them into something shippable. hyperframes, remotion, claude design — when each tool wins, how they combine, and the data-driven workflows that turn one template into a hundred videos.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":46,"xpTotal":66},{"number":30,"slug":"harness-engineering","title":"harness engineering — the discipline behind the model","blurb":"every coding agent is a model plus a harness. the model is bought; the harness is engineered. learn the craft: how to ratchet rules from failures, fight context rot, design long-horizon loops, wire hooks as enforcement, and read the haas shift that's reshaping what you build vs buy.","status":"live","lessonCount":6,"stepCount":52,"liveLessonCount":6,"liveStepCount":52,"estMinutes":126,"xpTotal":143},{"number":31,"slug":"intro-to-terminal","title":"intro to terminal — the text way to drive your computer","blurb":"you've never opened a terminal. by the end of this chapter you have, and you can move around your files, make folders, and read files without touching the mouse. it's a keyboard shortcut, not a cockpit. every tool in the rest of this course assumes you can do this, so we do it first.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":26,"xpTotal":50},{"number":32,"slug":"intro-to-claude-cli","title":"intro to claude cli — the door to real building","blurb":"you've used claude in a chat window. the claude cli is the same model with its hands on your actual files. this chapter installs it, signs you in, and runs your first real command. by the end you've watched an ai read, plan, and change things on your machine, and you know when to reach for the cli instead of the chat box.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":48},{"number":33,"slug":"intro-to-codex-cli","title":"intro to openai codex cli — a second tool in the kit","blurb":"you know the claude cli. the openai codex cli does the same job, an ai working in your terminal on your real files, with a different company behind it. this chapter installs it, signs you in, and runs your first command. most of what you already know carries straight over, so this chapter is mostly about what is different and when to reach for which.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":32,"xpTotal":45},{"number":34,"slug":"team-skills","title":"claude skills for teams — playbooks your whole team shares","blurb":"a claude skill is a packaged set of instructions — your team's playbook — that claude loads when it's relevant, so nobody has to re-explain it. this chapter is for people who manage teams. it covers what a skill is, how a team shares and provisions skills, real examples for hr, legal, and ops work, when a skill beats a one-off prompt, and the governance you need before any skill touches real work.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":41,"xpTotal":60},{"number":35,"slug":"dataframes-numpy-pandas","title":"dataframes with numpy and pandas","blurb":"tables are the working surface of applied ml. learn rows, columns, missing values, joins, aggregates, and the dataframe habits ai-generated notebooks assume.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":43,"xpTotal":156},{"number":36,"slug":"sql-for-ml-datasets","title":"sql for ml datasets","blurb":"most training data starts in a database. learn the select, join, filter, aggregate, and leakage traps that decide whether a model is learning signal or nonsense.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":28,"xpTotal":104},{"number":37,"slug":"dataset-pipelines","title":"dataset formats, ingestion, and validation pipelines","blurb":"a dataset is a product surface. build ingestion, validation, partitions, manifests, and checkpoints so the next run is not a mystery.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":38,"slug":"ml-math-and-stats","title":"ml math and statistics that actually show up","blurb":"vectors, probability, distributions, correlation, and uncertainty are not trivia. they are how you read model behavior without worshipping it.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":39,"slug":"supervised-learning-workflows","title":"supervised learning workflows","blurb":"labels, splits, baselines, training, prediction, and evaluation. the supervised workflow is the first complete model loop.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":40,"xpTotal":130},{"number":40,"slug":"unsupervised-learning-and-embeddings","title":"unsupervised learning, embeddings, and recommenders","blurb":"not every useful model has labels. cluster, compare, retrieve, and recommend by turning examples into useful neighborhoods.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":32,"xpTotal":104},{"number":41,"slug":"metrics-and-error-analysis","title":"metrics, slices, and error analysis","blurb":"accuracy is a blunt instrument. learn confusion matrices, precision, recall, thresholds, slices, and failure notes so model quality has evidence.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":42,"slug":"pytorch-tensors-and-autograd","title":"pytorch tensors and autograd","blurb":"read tensor code without flinching. tensors, shapes, broadcasting, gradients, and autograd are the grammar of modern deep learning scripts.","status":"live","lessonCount":4,"stepCount":28,"liveLessonCount":4,"liveStepCount":28,"estMinutes":30,"xpTotal":104},{"number":43,"slug":"training-loops-and-optimizers","title":"training loops, backprop, optimizers, and schedulers","blurb":"the training loop is where models change. learn loss, gradients, optimizer steps, schedules, checkpoints, and the bugs ai ships there.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":44,"slug":"deep-learning-architectures","title":"cnns, transformers, and useful llm internals","blurb":"architecture literacy for builders: convolution, attention, tokens, decoding, kv cache, quantization, and what those choices do to cost and behavior.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":45,"slug":"feature-experiments-registries","title":"feature pipelines, experiment tracking, and registries","blurb":"features, runs, configs, artifacts, and registries are how ml work becomes repeatable instead of a lucky notebook.","status":"live","lessonCount":5,"stepCount":35,"liveLessonCount":5,"liveStepCount":35,"estMinutes":37,"xpTotal":130},{"number":46,"slug":"model-serving-and-mlops","title":"model serving, ci/cd, and mlops","blurb":"serving a model means handling inputs, versions, routes, batch jobs, ci gates, rollback, and production failures deliberately.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":47,"slug":"monitoring-cloud-portfolio","title":"monitoring, drift, cloud scale, and portfolio launch","blurb":"the last mile: logs, drift, alerts, retraining decisions, cloud cost, gpu constraints, architecture docs, demos, and role stories.","status":"live","lessonCount":6,"stepCount":42,"liveLessonCount":6,"liveStepCount":42,"estMinutes":44,"xpTotal":156},{"number":48,"slug":"graphic-design-studio","title":"graphic design studio — prompt, kit, and fifty on-brand assets","blurb":"how a working designer runs image models: prompts written like creative briefs, a brand bible that travels as structured data — with the design tokens that carry it onto screens — and the seeds, reference locks, and QA filters that make fifty generated assets look like one studio shipped them.","status":"live","lessonCount":3,"stepCount":27,"liveLessonCount":3,"liveStepCount":27,"estMinutes":73,"xpTotal":82},{"number":49,"slug":"campaign-studio","title":"campaign studio — one campaign, end to end, with the gate on","blurb":"brief to voice-locked copy variants to batch plan to calendar to the numbers pass. the whole campaign as data your scripts can check — and an approval gate nothing skips. built for the person who owns the send button.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":59,"xpTotal":69},{"number":50,"slug":"support-studio","title":"support studio — the reply drafter you can trust","blurb":"how a working support agent runs AI: a policy pack the drafter can't talk around, a banned-promises linter that holds every draft, golden eval sets that make each macro earn autonomy one intent at a time, and the escalation line where money, law, and anger stay human.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":61,"xpTotal":71},{"number":51,"slug":"copy-studio","title":"copy studio — voice from evidence, variants at scale, claims that clear","blurb":"a client's voice measured from their own published copy, variant batches that pass the voice gate or die with a named reason, and a claim check that blocks the ship button while any fact is unsourced. corrections-log culture, as code.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":66,"xpTotal":72},{"number":52,"slug":"analytics-studio","title":"analytics studio — messy export to defensible memo","blurb":"how a working analyst runs AI: a cleaning log that accounts for every dropped row, sample-first checks that catch the plausible-but-wrong aggregation, a memo where every number carries the address of the cell that made it, SR 11-7-style challenge against the AI's recommendation, and the rerun test — same data, same code, same memo, or it isn't done.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":61,"xpTotal":74},{"number":53,"slug":"delivery-studio","title":"delivery studio — direct the build without becoming the bottleneck","blurb":"how a working delivery lead runs AI builders: specs with acceptance checks written before the build, a review that runs the checks instead of admiring the demo, status roll-ups computed from acceptance records instead of standup vibes, and the not-ready call made on evidence you can read out loud to a steering committee.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":56,"xpTotal":72},{"number":54,"slug":"people-studio","title":"people studio — draft fast, decide human, prove everything","blurb":"how a working hr specialist runs AI: a data firewall that keeps employee records out of prompts by code instead of willpower, recruiting briefs where every claim cites a resume line, policy drafts that cannot publish without a named legal reviewer, and the adverse-action paper trail that survives a discrimination claim.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":56,"xpTotal":72},{"number":55,"slug":"ops-studio","title":"ops studio — tribal knowledge to running automation","blurb":"how a working ops lead gets the process out of one person's head and into automation that can be trusted: SOPs captured as versioned data with the invented steps hunted down, recurring checklists with an explicit auto-run vs named-approval line, exception queues instead of silent failures, and a receipt against the export for every number in the weekly report.","status":"live","lessonCount":3,"stepCount":24,"liveLessonCount":3,"liveStepCount":24,"estMinutes":58,"xpTotal":72},{"number":56,"slug":"legal-studio","title":"legal studio — the cite-check gate and the log that survives","blurb":"how a working lawyer runs AI: a citation-verification drill where every authority gets opened at the source (the fabricated ones always look right), clause extraction that's worthless without a pinpoint, privilege rules written as a routing table instead of remembered under deadline, and the review log you'd want in the record when opposing counsel moves for sanctions.","status":"live","lessonCount":3,"stepCount":22,"liveLessonCount":3,"liveStepCount":22,"estMinutes":60,"xpTotal":69},{"number":57,"slug":"agent-studio","title":"agent studio — ship one agent feature before you stand up","blurb":"one sitting, one agent feature, end to end: a tool schema that makes the dangerous call a validation error, a loop that meters its own spend, golden cases wired before the router works, a trace you debug from instead of vibes, and the ratchet that turns tonight's incident into tomorrow's eval. practice at production tempo.","status":"live","lessonCount":3,"stepCount":23,"liveLessonCount":3,"liveStepCount":23,"estMinutes":66,"xpTotal":73}]},"accessToc":{"chapters":[{"slug":"before-you-build","lessons":[{"slug":"from-worker-to-builder","steps":[{"id":"before-you-build/from-worker-to-builder/s01-01-intro"}]},{"slug":"upgrade-dull-artifacts-into-interactive-sites","steps":[{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s01-01-intro"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s02-02-build-brief"},{"id":"before-you-build/upgrade-dull-artifacts-into-interactive-sites/s03-03-your-artifact-brief"}]},{"slug":"why-building-matters-now","steps":[{"id":"before-you-build/why-building-matters-now/s01-01-intro"}]},{"slug":"the-builder-loop","steps":[{"id":"before-you-build/the-builder-loop/s01-01-intro"}]},{"slug":"your-first-builder-brief","steps":[{"id":"before-you-build/your-first-builder-brief/s01-01-intro"}]},{"slug":"what-the-ai-is-doing","steps":[{"id":"before-you-build/what-the-ai-is-doing/s01-01-intro"}]},{"slug":"build-your-first-reusable-tool","steps":[{"id":"before-you-build/build-your-first-reusable-tool/s01-01-intro"}]},{"slug":"from-prompt-to-project","steps":[{"id":"before-you-build/from-prompt-to-project/s01-01-intro"}]},{"slug":"why-code-appears-later","steps":[{"id":"before-you-build/why-code-appears-later/s01-01-intro"}]},{"slug":"use-it-again","steps":[{"id":"before-you-build/use-it-again/s01-01-intro"},{"id":"before-you-build/use-it-again/s02-02-which-fix-compounds"},{"id":"before-you-build/use-it-again/s03-03-save-version-two"}]}]},{"slug":"read-the-script","lessons":[{"slug":"names-and-values","steps":[{"id":"read-the-script/names-and-values/s01-01-intro"},{"id":"read-the-script/names-and-values/s02-02-spot-the-type"},{"id":"read-the-script/names-and-values/s03-03-assignment"},{"id":"read-the-script/names-and-values/s04-04-predict-the-score"},{"id":"read-the-script/names-and-values/s05-05-fill-the-name"},{"id":"read-the-script/names-and-values/s06-06-fix-the-typo"},{"id":"read-the-script/names-and-values/s07-07-write-greeting"},{"id":"read-the-script/names-and-values/s08-08-checkpoint"}]},{"slug":"functions-and-the-hallucinated-return","steps":[{"id":"read-the-script/functions-and-the-hallucinated-return/s01-01-intro"},{"id":"read-the-script/functions-and-the-hallucinated-return/s02-02-which-returns"},{"id":"read-the-script/functions-and-the-hallucinated-return/s03-03-return-vs-print"},{"id":"read-the-script/functions-and-the-hallucinated-return/s04-04-predict-the-result"},{"id":"read-the-script/functions-and-the-hallucinated-return/s05-05-fix-the-missing-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s06-06-fix-print-vs-return"},{"id":"read-the-script/functions-and-the-hallucinated-return/s07-07-write-square"},{"id":"read-the-script/functions-and-the-hallucinated-return/s08-08-checkpoint"}]},{"slug":"decisions-if-elif","steps":[{"id":"read-the-script/decisions-if-elif/s01-01-intro"},{"id":"read-the-script/decisions-if-elif/s02-02-truthiness-quiz"},{"id":"read-the-script/decisions-if-elif/s03-03-eq-vs-is"},{"id":"read-the-script/decisions-if-elif/s04-04-predict-the-empty-list"},{"id":"read-the-script/decisions-if-elif/s05-05-fix-the-zero-bug"},{"id":"read-the-script/decisions-if-elif/s06-06-fix-the-elif-order"},{"id":"read-the-script/decisions-if-elif/s07-07-checkpoint"},{"id":"read-the-script/decisions-if-elif/s08-08-going-deeper"}]}]},{"slug":"read-the-data-shapes","lessons":[{"slug":"the-shapes-ai-dumps-on-you","steps":[{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s01-01-intro"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s02-02-which-is-which"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s03-03-nested-paths"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s04-04-predict-the-deep-key"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s05-05-fill-the-index"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s06-06-fix-the-keyerror"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s07-07-fix-the-list-vs-dict"},{"id":"read-the-data-shapes/the-shapes-ai-dumps-on-you/s08-08-checkpoint"}]},{"slug":"walking-the-shape","steps":[{"id":"read-the-data-shapes/walking-the-shape/s01-01-intro"},{"id":"read-the-data-shapes/walking-the-shape/s02-02-which-loop-shape"},{"id":"read-the-data-shapes/walking-the-shape/s03-03-walking-the-shape"},{"id":"read-the-data-shapes/walking-the-shape/s04-04-predict-the-list-loop"},{"id":"read-the-data-shapes/walking-the-shape/s05-05-fill-the-loop-variable"},{"id":"read-the-data-shapes/walking-the-shape/s06-06-fix-the-dict-loop"},{"id":"read-the-data-shapes/walking-the-shape/s07-07-write-sum-loop"},{"id":"read-the-data-shapes/walking-the-shape/s08-08-checkpoint"}]},{"slug":"comprehensions-and-the-wrong-layer-bug","steps":[{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s01-01-intro"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s02-02-which-comp-shape"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s03-03-with-filter"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s04-04-predict-the-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s05-05-fix-the-wrong-layer"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s06-06-write-comp"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s07-07-checkpoint"},{"id":"read-the-data-shapes/comprehensions-and-the-wrong-layer-bug/s08-08-going-deeper"}]}]},{"slug":"read-the-failure","lessons":[{"slug":"read-bottom-up","steps":[{"id":"read-the-failure/read-bottom-up/s01-01-anatomy"},{"id":"read-the-failure/read-bottom-up/s02-02-which-error"},{"id":"read-the-failure/read-bottom-up/s03-03-the-five-fingerprints"},{"id":"read-the-failure/read-bottom-up/s04-04-predict-the-class"},{"id":"read-the-failure/read-bottom-up/s05-05-fix-nameerror"},{"id":"read-the-failure/read-bottom-up/s06-06-fix-attributeerror"},{"id":"read-the-failure/read-bottom-up/s07-07-write-defensive"},{"id":"read-the-failure/read-bottom-up/s08-08-checkpoint"}]},{"slug":"try-except-what-ai-misuses","steps":[{"id":"read-the-failure/try-except-what-ai-misuses/s01-01-intro"},{"id":"read-the-failure/try-except-what-ai-misuses/s02-02-which-block-runs"},{"id":"read-the-failure/try-except-what-ai-misuses/s03-03-naming-the-error"},{"id":"read-the-failure/try-except-what-ai-misuses/s04-04-predict-the-branch"},{"id":"read-the-failure/try-except-what-ai-misuses/s05-05-fill-the-keyword"},{"id":"read-the-failure/try-except-what-ai-misuses/s06-06-fix-the-bare-except"},{"id":"read-the-failure/try-except-what-ai-misuses/s07-07-write-safe-lookup"},{"id":"read-the-failure/try-except-what-ai-misuses/s08-08-checkpoint"}]},{"slug":"fail-loudly","steps":[{"id":"read-the-failure/fail-loudly/s01-01-intro"},{"id":"read-the-failure/fail-loudly/s02-02-which-raise-fires"},{"id":"read-the-failure/fail-loudly/s03-03-predict-the-raise"},{"id":"read-the-failure/fail-loudly/s04-04-fill-the-raise-keyword"},{"id":"read-the-failure/fail-loudly/s05-05-fix-the-silent-swallow"},{"id":"read-the-failure/fail-loudly/s06-06-write-validate-age"},{"id":"read-the-failure/fail-loudly/s07-07-checkpoint"},{"id":"read-the-failure/fail-loudly/s08-08-going-deeper"}]}]},{"slug":"read-the-pipeline","lessons":[{"slug":"files-without-fear","steps":[{"id":"read-the-pipeline/files-without-fear/s01-01-intro"},{"id":"read-the-pipeline/files-without-fear/s02-02-which-mode"},{"id":"read-the-pipeline/files-without-fear/s03-03-pathlib-and-manifests"},{"id":"read-the-pipeline/files-without-fear/s04-04-predict-the-suffix"},{"id":"read-the-pipeline/files-without-fear/s05-05-fill-the-mode"},{"id":"read-the-pipeline/files-without-fear/s06-06-fix-the-wrong-mode"},{"id":"read-the-pipeline/files-without-fear/s07-07-write-manifest-entry"},{"id":"read-the-pipeline/files-without-fear/s08-08-checkpoint"}]},{"slug":"talking-to-apis","steps":[{"id":"read-the-pipeline/talking-to-apis/s01-01-intro"},{"id":"read-the-pipeline/talking-to-apis/s02-02-which-status-family"},{"id":"read-the-pipeline/talking-to-apis/s03-03-parsing-the-json"},{"id":"read-the-pipeline/talking-to-apis/s04-04-predict-the-key"},{"id":"read-the-pipeline/talking-to-apis/s05-05-fill-the-method"},{"id":"read-the-pipeline/talking-to-apis/s06-06-fix-the-status-check"},{"id":"read-the-pipeline/talking-to-apis/s07-07-write-classify-status"},{"id":"read-the-pipeline/talking-to-apis/s08-08-checkpoint"}]},{"slug":"when-the-pipeline-breaks","steps":[{"id":"read-the-pipeline/when-the-pipeline-breaks/s01-01-intro"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s02-02-honor-the-retry-after"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s03-03-predict-the-status-handling"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s04-04-fill-the-raise-for-status"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s05-05-fix-the-missing-check"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s06-06-fix-the-broad-retry"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s07-07-checkpoint"},{"id":"read-the-pipeline/when-the-pipeline-breaks/s08-08-going-deeper"}]}]},{"slug":"mutation-and-state","lessons":[{"slug":"why-it-breaks","steps":[{"id":"mutation-and-state/why-it-breaks/s01-01-intro"},{"id":"mutation-and-state/why-it-breaks/s02-02-which-mutates"},{"id":"mutation-and-state/why-it-breaks/s03-03-shared-reference"},{"id":"mutation-and-state/why-it-breaks/s04-04-fix-the-shared-list"},{"id":"mutation-and-state/why-it-breaks/s05-05-checkpoint"}]},{"slug":"copy-vs-reference","steps":[{"id":"mutation-and-state/copy-vs-reference/s01-01-deep-copy"},{"id":"mutation-and-state/copy-vs-reference/s02-02-predict-the-shallow"},{"id":"mutation-and-state/copy-vs-reference/s03-03-fix-the-shallow-bug"},{"id":"mutation-and-state/copy-vs-reference/s04-04-checkpoint"}]}]},{"slug":"llm-apis","lessons":[{"slug":"before-the-api-what-the-model-is-doing","steps":[{"id":"llm-apis/before-the-api-what-the-model-is-doing/s01-01-intro"}]},{"slug":"the-messages-pattern","steps":[{"id":"llm-apis/the-messages-pattern/s01-01-intro"},{"id":"llm-apis/the-messages-pattern/s02-02-which-role-is-yours"},{"id":"llm-apis/the-messages-pattern/s03-03-reading-the-response"},{"id":"llm-apis/the-messages-pattern/s04-04-predict-the-text"},{"id":"llm-apis/the-messages-pattern/s05-05-fill-the-role"},{"id":"llm-apis/the-messages-pattern/s06-06-fix-the-message-shape"},{"id":"llm-apis/the-messages-pattern/s07-07-fix-the-response-access"},{"id":"llm-apis/the-messages-pattern/s08-08-write-ask-claude"},{"id":"llm-apis/the-messages-pattern/s09-09-checkpoint"}]},{"slug":"reading-the-response","steps":[{"id":"llm-apis/reading-the-response/s01-01-intro"},{"id":"llm-apis/reading-the-response/s02-02-which-block-do-you-read"},{"id":"llm-apis/reading-the-response/s03-03-the-five-stop-reasons"},{"id":"llm-apis/reading-the-response/s04-04-predict-the-text"},{"id":"llm-apis/reading-the-response/s05-05-fill-the-text-extract"},{"id":"llm-apis/reading-the-response/s06-06-fix-the-content-zero"},{"id":"llm-apis/reading-the-response/s07-07-fix-the-stop-assumption"},{"id":"llm-apis/reading-the-response/s08-08-write-the-extractor"},{"id":"llm-apis/reading-the-response/s09-09-checkpoint"}]},{"slug":"the-model-picker","steps":[{"id":"llm-apis/the-model-picker/s01-01-why-model-choice-is-a-product-decision"},{"id":"llm-apis/the-model-picker/s02-02-haiku-territory"},{"id":"llm-apis/the-model-picker/s03-03-which-model-here"},{"id":"llm-apis/the-model-picker/s04-04-sonnet-territory"},{"id":"llm-apis/the-model-picker/s05-05-opus-and-the-thinking-budget"},{"id":"llm-apis/the-model-picker/s06-06-pick-the-model"},{"id":"llm-apis/the-model-picker/s07-07-write-the-model-routing-memo"},{"id":"llm-apis/the-model-picker/s08-08-checkpoint"}]}]},{"slug":"structured-output","lessons":[{"slug":"getting-json-back","steps":[{"id":"structured-output/getting-json-back/s01-01-intro"},{"id":"structured-output/getting-json-back/s02-02-why-validate"},{"id":"structured-output/getting-json-back/s03-03-pydantic-models"},{"id":"structured-output/getting-json-back/s04-04-predict-the-validated"},{"id":"structured-output/getting-json-back/s05-05-fill-the-field-type"},{"id":"structured-output/getting-json-back/s06-06-fix-the-required-field"},{"id":"structured-output/getting-json-back/s07-07-fix-the-type-mismatch"},{"id":"structured-output/getting-json-back/s08-08-write-parse-extraction"},{"id":"structured-output/getting-json-back/s09-09-checkpoint"}]},{"slug":"schemas-eat-prompts","steps":[{"id":"structured-output/schemas-eat-prompts/s01-01-the-trust-boundary"},{"id":"structured-output/schemas-eat-prompts/s02-02-three-shipped-breakages"},{"id":"structured-output/schemas-eat-prompts/s03-03-which-boundary-broke"},{"id":"structured-output/schemas-eat-prompts/s04-04-schema-as-prompt-replacement"},{"id":"structured-output/schemas-eat-prompts/s05-05-where-validation-belongs"},{"id":"structured-output/schemas-eat-prompts/s06-06-the-pydantic-migration-as-war-story"},{"id":"structured-output/schemas-eat-prompts/s07-07-write-the-boundary-audit"},{"id":"structured-output/schemas-eat-prompts/s08-08-checkpoint"}]},{"slug":"dataset-schema-validation","steps":[{"id":"structured-output/dataset-schema-validation/s01-01-validate-at-the-boundary"},{"id":"structured-output/dataset-schema-validation/s02-02-the-silent-drop"},{"id":"structured-output/dataset-schema-validation/s03-03-predict-the-missing-fields"},{"id":"structured-output/dataset-schema-validation/s04-04-fill-the-membership-test"},{"id":"structured-output/dataset-schema-validation/s05-05-fix-the-range-check"},{"id":"structured-output/dataset-schema-validation/s06-06-write-validate-batch"},{"id":"structured-output/dataset-schema-validation/s07-07-claims-with-receipts"},{"id":"structured-output/dataset-schema-validation/s08-08-checkpoint"}]}]},{"slug":"mcp","lessons":[{"slug":"what-mcp-is","steps":[{"id":"mcp/what-mcp-is/s01-01-intro"},{"id":"mcp/what-mcp-is/s02-02-which-side-is-the-server"},{"id":"mcp/what-mcp-is/s03-03-the-tool-list"},{"id":"mcp/what-mcp-is/s04-04-predict-the-tool-result"},{"id":"mcp/what-mcp-is/s05-05-fill-the-tool-name"},{"id":"mcp/what-mcp-is/s06-06-fix-the-missing-argument"},{"id":"mcp/what-mcp-is/s07-07-fix-the-result-shape"},{"id":"mcp/what-mcp-is/s08-08-write-call-tool"},{"id":"mcp/what-mcp-is/s09-09-checkpoint"}]},{"slug":"writing-a-tiny-mcp-server","steps":[{"id":"mcp/writing-a-tiny-mcp-server/s01-01-intro"},{"id":"mcp/writing-a-tiny-mcp-server/s02-02-which-message-is-call-tool"},{"id":"mcp/writing-a-tiny-mcp-server/s03-03-the-tool-registry"},{"id":"mcp/writing-a-tiny-mcp-server/s04-04-predict-the-list-tools-output"},{"id":"mcp/writing-a-tiny-mcp-server/s05-05-fill-the-input-schema"},{"id":"mcp/writing-a-tiny-mcp-server/s06-06-fix-the-missing-error-response"},{"id":"mcp/writing-a-tiny-mcp-server/s07-07-fix-the-validation-gap"},{"id":"mcp/writing-a-tiny-mcp-server/s08-08-write-the-call-tool-dispatcher"},{"id":"mcp/writing-a-tiny-mcp-server/s09-09-checkpoint"}]},{"slug":"why-mcp-won","steps":[{"id":"mcp/why-mcp-won/s01-01-the-graveyard-of-tool-protocols"},{"id":"mcp/why-mcp-won/s02-02-what-openai-plugins-got-wrong"},{"id":"mcp/why-mcp-won/s03-03-which-failure-mode"},{"id":"mcp/why-mcp-won/s04-04-the-mcp-bet"},{"id":"mcp/why-mcp-won/s05-05-the-network-effect"},{"id":"mcp/why-mcp-won/s06-06-spot-the-mcp-wedge"},{"id":"mcp/why-mcp-won/s07-07-write-the-protocol-postmortem"},{"id":"mcp/why-mcp-won/s08-08-checkpoint"}]}]},{"slug":"agent-loops","lessons":[{"slug":"the-loop","steps":[{"id":"agent-loops/the-loop/s01-01-intro"},{"id":"agent-loops/the-loop/s02-02-which-stop-reason"},{"id":"agent-loops/the-loop/s03-03-tool-result-shape"},{"id":"agent-loops/the-loop/s04-04-predict-the-next-call"},{"id":"agent-loops/the-loop/s05-05-fill-the-stop-condition"},{"id":"agent-loops/the-loop/s06-06-fix-the-tool-id"},{"id":"agent-loops/the-loop/s07-07-fix-the-loop-exit"},{"id":"agent-loops/the-loop/s08-08-write-the-loop"},{"id":"agent-loops/the-loop/s09-09-checkpoint"}]},{"slug":"multi-step-tools","steps":[{"id":"agent-loops/multi-step-tools/s01-01-intro"},{"id":"agent-loops/multi-step-tools/s02-02-which-tool-runs"},{"id":"agent-loops/multi-step-tools/s03-03-tool-registry"},{"id":"agent-loops/multi-step-tools/s04-04-predict-the-chain"},{"id":"agent-loops/multi-step-tools/s05-05-fill-the-dispatch"},{"id":"agent-loops/multi-step-tools/s06-06-fix-the-missing-tool"},{"id":"agent-loops/multi-step-tools/s07-07-fix-the-parallel-ids"},{"id":"agent-loops/multi-step-tools/s08-08-write-multi-tool-agent"},{"id":"agent-loops/multi-step-tools/s09-09-checkpoint"}]},{"slug":"routing","steps":[{"id":"agent-loops/routing/s01-01-intro"},{"id":"agent-loops/routing/s02-02-when-to-route"},{"id":"agent-loops/routing/s03-03-classifier-output"},{"id":"agent-loops/routing/s04-04-predict-the-route"},{"id":"agent-loops/routing/s05-05-fill-the-dispatch"},{"id":"agent-loops/routing/s06-06-fix-the-fallback"},{"id":"agent-loops/routing/s07-07-fix-the-loose-classifier"},{"id":"agent-loops/routing/s08-08-write-the-router"},{"id":"agent-loops/routing/s09-09-checkpoint"}]},{"slug":"evaluator-optimizer","steps":[{"id":"agent-loops/evaluator-optimizer/s01-01-intro"},{"id":"agent-loops/evaluator-optimizer/s02-02-when-it-earns-its-keep"},{"id":"agent-loops/evaluator-optimizer/s03-03-judge-output-shape"},{"id":"agent-loops/evaluator-optimizer/s04-04-predict-the-iterations"},{"id":"agent-loops/evaluator-optimizer/s05-05-fill-the-exit"},{"id":"agent-loops/evaluator-optimizer/s06-06-fix-the-judge-parsing"},{"id":"agent-loops/evaluator-optimizer/s07-07-fix-the-missing-cap"},{"id":"agent-loops/evaluator-optimizer/s08-08-write-the-evaluator-optimizer"},{"id":"agent-loops/evaluator-optimizer/s09-09-checkpoint"}]},{"slug":"why-every-framework-is-thirty-lines","steps":[{"id":"agent-loops/why-every-framework-is-thirty-lines/s01-01-the-frameworks-converged"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s02-02-langgraph-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s03-03-the-work-is-a-graph"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s04-04-which-framework-fits"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s05-05-vercel-ai-sdk-walkthrough"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s06-06-the-five-canonical-patterns"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s07-07-when-to-stay-framework-free"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s08-08-write-the-tradeoff-memo"},{"id":"agent-loops/why-every-framework-is-thirty-lines/s09-09-checkpoint"}]},{"slug":"the-industry-map","steps":[{"id":"agent-loops/the-industry-map/s01-01-the-five-layers"},{"id":"agent-loops/the-industry-map/s02-02-which-layer-failed"},{"id":"agent-loops/the-industry-map/s03-03-the-shape-of-the-stack"}]}]},{"slug":"git-and-github","lessons":[{"slug":"the-three-states","steps":[{"id":"git-and-github/the-three-states/s01-01-intro"},{"id":"git-and-github/the-three-states/s02-02-which-command-stages"},{"id":"git-and-github/the-three-states/s03-03-status-and-diff"},{"id":"git-and-github/the-three-states/s04-04-predict-the-status"},{"id":"git-and-github/the-three-states/s05-05-fill-the-command"},{"id":"git-and-github/the-three-states/s06-06-fix-the-staged-files"},{"id":"git-and-github/the-three-states/s07-07-fix-the-pr-command"},{"id":"git-and-github/the-three-states/s08-08-write-status-summary"},{"id":"git-and-github/the-three-states/s09-09-checkpoint"}]},{"slug":"three-git-disasters-ai-shipped","steps":[{"id":"git-and-github/three-git-disasters-ai-shipped/s01-01-why-postmortems-teach-better"},{"id":"git-and-github/three-git-disasters-ai-shipped/s02-02-uber-2016-credentials-in-repo"},{"id":"git-and-github/three-git-disasters-ai-shipped/s03-03-which-control-would-have-caught-it"},{"id":"git-and-github/three-git-disasters-ai-shipped/s04-04-the-samsung-paste"},{"id":"git-and-github/three-git-disasters-ai-shipped/s05-05-the-cursor-git-add-dot-default"},{"id":"git-and-github/three-git-disasters-ai-shipped/s06-06-spot-the-failure-mode"},{"id":"git-and-github/three-git-disasters-ai-shipped/s07-07-write-your-pre-commit-checklist"},{"id":"git-and-github/three-git-disasters-ai-shipped/s08-08-checkpoint"}]},{"slug":"github-actions-for-model-checks","steps":[{"id":"git-and-github/github-actions-for-model-checks/s01-01-intro"},{"id":"git-and-github/github-actions-for-model-checks/s02-02-green-but-wrong"},{"id":"git-and-github/github-actions-for-model-checks/s03-03-predict-the-run"},{"id":"git-and-github/github-actions-for-model-checks/s04-04-fill-the-trigger"},{"id":"git-and-github/github-actions-for-model-checks/s05-05-fix-the-eval-gate"},{"id":"git-and-github/github-actions-for-model-checks/s06-06-write-run-job"},{"id":"git-and-github/github-actions-for-model-checks/s07-07-checkpoint"}]}]},{"slug":"secrets-and-env","lessons":[{"slug":"keeping-keys-safe","steps":[{"id":"secrets-and-env/keeping-keys-safe/s01-01-intro"},{"id":"secrets-and-env/keeping-keys-safe/s02-02-which-line-leaks"},{"id":"secrets-and-env/keeping-keys-safe/s03-03-loading-env-vars"},{"id":"secrets-and-env/keeping-keys-safe/s04-04-predict-the-key"},{"id":"secrets-and-env/keeping-keys-safe/s05-05-fill-the-getenv"},{"id":"secrets-and-env/keeping-keys-safe/s06-06-fix-the-hardcoded-key"},{"id":"secrets-and-env/keeping-keys-safe/s07-07-fix-the-missing-default"},{"id":"secrets-and-env/keeping-keys-safe/s08-08-write-load-config"},{"id":"secrets-and-env/keeping-keys-safe/s09-09-checkpoint"}]},{"slug":"ml-service-secrets","steps":[{"id":"secrets-and-env/ml-service-secrets/s01-01-intro"},{"id":"secrets-and-env/ml-service-secrets/s02-02-which-log-line-leaks"},{"id":"secrets-and-env/ml-service-secrets/s03-03-predict-the-scan"},{"id":"secrets-and-env/ml-service-secrets/s04-04-fill-the-lookup"},{"id":"secrets-and-env/ml-service-secrets/s05-05-fix-the-logged-key"},{"id":"secrets-and-env/ml-service-secrets/s06-06-write-scan-logs"},{"id":"secrets-and-env/ml-service-secrets/s07-07-checkpoint"}]}]},{"slug":"prompting","lessons":[{"slug":"builder-briefs-before-prompts","steps":[{"id":"prompting/builder-briefs-before-prompts/s01-01-intro"}]},{"slug":"the-prompt-craft","steps":[{"id":"prompting/the-prompt-craft/s01-01-intro"},{"id":"prompting/the-prompt-craft/s02-02-which-prompt-is-better"},{"id":"prompting/the-prompt-craft/s03-03-context-rot"},{"id":"prompting/the-prompt-craft/s04-04-predict-the-output"},{"id":"prompting/the-prompt-craft/s05-05-fill-the-constraint"},{"id":"prompting/the-prompt-craft/s06-06-fix-the-vague-prompt"},{"id":"prompting/the-prompt-craft/s07-07-fix-the-bloated-context"},{"id":"prompting/the-prompt-craft/s08-08-write-the-prompt"},{"id":"prompting/the-prompt-craft/s09-09-checkpoint"}]},{"slug":"few-shot-and-reasoning","steps":[{"id":"prompting/few-shot-and-reasoning/s01-01-intro"},{"id":"prompting/few-shot-and-reasoning/s02-02-when-few-shot-helps"},{"id":"prompting/few-shot-and-reasoning/s03-03-the-cot-trap"},{"id":"prompting/few-shot-and-reasoning/s04-04-predict-the-format-lock"},{"id":"prompting/few-shot-and-reasoning/s05-05-fill-the-example-format"},{"id":"prompting/few-shot-and-reasoning/s06-06-fix-the-cot-on-reasoning"},{"id":"prompting/few-shot-and-reasoning/s07-07-fix-the-format-mismatch"},{"id":"prompting/few-shot-and-reasoning/s08-08-write-the-prompt-builder"},{"id":"prompting/few-shot-and-reasoning/s09-09-checkpoint"}]},{"slug":"agent-config-files","steps":[{"id":"prompting/agent-config-files/s01-01-intro"},{"id":"prompting/agent-config-files/s02-02-which-belongs-in-the-file"},{"id":"prompting/agent-config-files/s03-03-anatomy-of-a-good-file"},{"id":"prompting/agent-config-files/s04-04-predict-which-rule-fires"},{"id":"prompting/agent-config-files/s05-05-fill-the-rule-block"},{"id":"prompting/agent-config-files/s06-06-fix-the-too-vague"},{"id":"prompting/agent-config-files/s07-07-fix-the-stale-pointer"},{"id":"prompting/agent-config-files/s08-08-write-a-claude-md"},{"id":"prompting/agent-config-files/s09-09-checkpoint"}]},{"slug":"what-aged-in-prompting","steps":[{"id":"prompting/what-aged-in-prompting/s01-01-prompting-as-a-moving-target"},{"id":"prompting/what-aged-in-prompting/s02-02-cot-then-vs-now"},{"id":"prompting/what-aged-in-prompting/s03-03-which-still-works"},{"id":"prompting/what-aged-in-prompting/s04-04-the-system-prompt-arms-race"},{"id":"prompting/what-aged-in-prompting/s05-05-spot-the-stale-technique"},{"id":"prompting/what-aged-in-prompting/s06-06-the-prompt-engineer-debate"},{"id":"prompting/what-aged-in-prompting/s07-07-write-the-cheatsheet-update"},{"id":"prompting/what-aged-in-prompting/s08-08-checkpoint"}]}]},{"slug":"agent-traces","lessons":[{"slug":"reading-the-trace","steps":[{"id":"agent-traces/reading-the-trace/s01-01-intro"},{"id":"agent-traces/reading-the-trace/s02-02-which-turn-failed"},{"id":"agent-traces/reading-the-trace/s03-03-the-stop-reasons"},{"id":"agent-traces/reading-the-trace/s04-04-predict-the-stop"},{"id":"agent-traces/reading-the-trace/s05-05-fill-the-tool-name"},{"id":"agent-traces/reading-the-trace/s06-06-fix-the-loop"},{"id":"agent-traces/reading-the-trace/s07-07-fix-the-arg-shape"},{"id":"agent-traces/reading-the-trace/s08-08-write-the-summarizer"},{"id":"agent-traces/reading-the-trace/s09-09-checkpoint"}]},{"slug":"trace-driven-debugging","steps":[{"id":"agent-traces/trace-driven-debugging/s01-01-intro"},{"id":"agent-traces/trace-driven-debugging/s02-02-which-pattern-is-this"},{"id":"agent-traces/trace-driven-debugging/s03-03-the-loop-signature"},{"id":"agent-traces/trace-driven-debugging/s04-04-predict-the-call-count"},{"id":"agent-traces/trace-driven-debugging/s05-05-fill-the-detector"},{"id":"agent-traces/trace-driven-debugging/s06-06-fix-the-tool-routing"},{"id":"agent-traces/trace-driven-debugging/s07-07-fix-the-prompt-bloat"},{"id":"agent-traces/trace-driven-debugging/s08-08-write-the-trace-summary"},{"id":"agent-traces/trace-driven-debugging/s09-09-checkpoint"}]},{"slug":"prediction-ids-and-inference-logs","steps":[{"id":"agent-traces/prediction-ids-and-inference-logs/s01-01-intro"},{"id":"agent-traces/prediction-ids-and-inference-logs/s02-02-pick-the-log-line"},{"id":"agent-traces/prediction-ids-and-inference-logs/s03-03-predict-the-ids"},{"id":"agent-traces/prediction-ids-and-inference-logs/s04-04-fill-the-join-key"},{"id":"agent-traces/prediction-ids-and-inference-logs/s05-05-fix-the-join"},{"id":"agent-traces/prediction-ids-and-inference-logs/s06-06-write-the-logger"},{"id":"agent-traces/prediction-ids-and-inference-logs/s07-07-checkpoint"}]}]},{"slug":"evals","lessons":[{"slug":"check-before-you-trust","steps":[{"id":"evals/check-before-you-trust/s01-01-intro"}]},{"slug":"writing-evals","steps":[{"id":"evals/writing-evals/s01-01-intro"},{"id":"evals/writing-evals/s02-02-which-eval-passes"},{"id":"evals/writing-evals/s03-03-eval-patterns"},{"id":"evals/writing-evals/s04-04-predict-the-pass-rate"},{"id":"evals/writing-evals/s05-05-fill-the-assertion"},{"id":"evals/writing-evals/s06-06-fix-the-flaky-eval"},{"id":"evals/writing-evals/s07-07-fix-the-overstrict-eval"},{"id":"evals/writing-evals/s08-08-write-the-suite"},{"id":"evals/writing-evals/s09-09-checkpoint"}]},{"slug":"llm-as-judge","steps":[{"id":"evals/llm-as-judge/s01-01-intro"},{"id":"evals/llm-as-judge/s02-02-pairwise-or-rubric"},{"id":"evals/llm-as-judge/s03-03-the-four-biases"},{"id":"evals/llm-as-judge/s04-04-predict-the-bias"},{"id":"evals/llm-as-judge/s05-05-fill-the-rubric-call"},{"id":"evals/llm-as-judge/s06-06-fix-the-likert-trap"},{"id":"evals/llm-as-judge/s07-07-fix-the-position-bias"},{"id":"evals/llm-as-judge/s08-08-write-the-rubric-judge"},{"id":"evals/llm-as-judge/s09-09-checkpoint"}]},{"slug":"the-rise-of-evals-as-a-discipline","steps":[{"id":"evals/the-rise-of-evals-as-a-discipline/s01-01-the-prompt-engineering-era"},{"id":"evals/the-rise-of-evals-as-a-discipline/s02-02-the-eval-turn"},{"id":"evals/the-rise-of-evals-as-a-discipline/s03-03-which-company-survives-a-model-swap"},{"id":"evals/the-rise-of-evals-as-a-discipline/s04-04-case-studies"},{"id":"evals/the-rise-of-evals-as-a-discipline/s05-05-eval-first-vs-prompt-first"},{"id":"evals/the-rise-of-evals-as-a-discipline/s06-06-why-the-eval-engineer-eats-the-prompt-engineer"},{"id":"evals/the-rise-of-evals-as-a-discipline/s07-07-write-the-eval-readiness-audit"},{"id":"evals/the-rise-of-evals-as-a-discipline/s08-08-checkpoint"}]},{"slug":"model-validation-gates","steps":[{"id":"evals/model-validation-gates/s01-01-intro"},{"id":"evals/model-validation-gates/s02-02-spot-the-theater"},{"id":"evals/model-validation-gates/s03-03-predict-the-decision"},{"id":"evals/model-validation-gates/s04-04-fill-the-regression-check"},{"id":"evals/model-validation-gates/s05-05-fix-the-theater-gate"},{"id":"evals/model-validation-gates/s06-06-write-the-gate-runner"},{"id":"evals/model-validation-gates/s07-07-checkpoint"}]}]},{"slug":"context-and-retrieval","lessons":[{"slug":"chunking-that-respects-structure","steps":[{"id":"context-and-retrieval/chunking-that-respects-structure/s01-01-intro"},{"id":"context-and-retrieval/chunking-that-respects-structure/s02-02-which-content-shape-breaks"},{"id":"context-and-retrieval/chunking-that-respects-structure/s03-03-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s04-04-predict-the-shred"},{"id":"context-and-retrieval/chunking-that-respects-structure/s05-05-fill-the-separator-list"},{"id":"context-and-retrieval/chunking-that-respects-structure/s06-06-fix-the-mid-sentence-cut"},{"id":"context-and-retrieval/chunking-that-respects-structure/s07-07-fix-the-missing-overlap"},{"id":"context-and-retrieval/chunking-that-respects-structure/s08-08-write-the-recursive-splitter"},{"id":"context-and-retrieval/chunking-that-respects-structure/s09-09-checkpoint"}]},{"slug":"embedding-that-fits-the-budget","steps":[{"id":"context-and-retrieval/embedding-that-fits-the-budget/s01-01-intro"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s02-02-which-model-fits"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s03-03-the-vector-space"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s04-04-predict-the-closest"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s05-05-fill-the-cosine-formula"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s06-06-fix-the-wrong-embedding-model"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s07-07-fix-the-missing-normalization"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s08-08-write-the-similarity-ranker"},{"id":"context-and-retrieval/embedding-that-fits-the-budget/s09-09-checkpoint"}]},{"slug":"retrieval-that-finds-the-right-thing","steps":[{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s01-01-intro"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s02-02-which-result-is-good"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s03-03-the-distance-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s04-04-predict-the-top-k"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s05-05-fill-the-cutoff"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s06-06-fix-the-no-threshold"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s07-07-fix-the-dup-results"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s08-08-write-the-ranker"},{"id":"context-and-retrieval/retrieval-that-finds-the-right-thing/s09-09-checkpoint"}]},{"slug":"rag-vs-long-context-vs-fine-tune","steps":[{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s01-01-the-three-way-fork"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s02-02-rag-when-you-need-freshness"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s03-03-which-fork-for-this-scenario"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s04-04-long-context-when-the-corpus-fits"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s05-05-fine-tune-when-style-or-format-is-the-product"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s06-06-which-fork-dies-first"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s07-07-graph-vector-or-both"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s08-08-the-rubric"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s09-09-write-the-fork-decision"},{"id":"context-and-retrieval/rag-vs-long-context-vs-fine-tune/s10-10-checkpoint"}]},{"slug":"retrieval-metrics-and-vector-db-shape","steps":[{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s01-01-the-eval-set"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s02-02-filter-or-search-first"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s03-03-predict-recall-precision"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s04-04-fill-the-reciprocal"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s05-05-fix-the-late-filter"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s06-06-write-recall-at-k"},{"id":"context-and-retrieval/retrieval-metrics-and-vector-db-shape/s07-07-checkpoint"}]}]},{"slug":"production-tradeoffs","lessons":[{"slug":"prompt-caching-correctly","steps":[{"id":"production-tradeoffs/prompt-caching-correctly/s01-01-intro"},{"id":"production-tradeoffs/prompt-caching-correctly/s02-02-which-piece-caches"},{"id":"production-tradeoffs/prompt-caching-correctly/s03-03-the-cache-breakpoints"},{"id":"production-tradeoffs/prompt-caching-correctly/s04-04-predict-the-savings"},{"id":"production-tradeoffs/prompt-caching-correctly/s05-05-fill-the-cache-control"},{"id":"production-tradeoffs/prompt-caching-correctly/s06-06-fix-the-cache-buster"},{"id":"production-tradeoffs/prompt-caching-correctly/s07-07-fix-the-ttl-default"},{"id":"production-tradeoffs/prompt-caching-correctly/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/prompt-caching-correctly/s09-09-checkpoint"}]},{"slug":"read-the-token-bill","steps":[{"id":"production-tradeoffs/read-the-token-bill/s01-01-intro"},{"id":"production-tradeoffs/read-the-token-bill/s02-02-which-prompt-costs-more"},{"id":"production-tradeoffs/read-the-token-bill/s03-03-the-shape-of-a-bill"},{"id":"production-tradeoffs/read-the-token-bill/s04-04-predict-the-monthly-spend"},{"id":"production-tradeoffs/read-the-token-bill/s05-05-fill-the-cost-formula"},{"id":"production-tradeoffs/read-the-token-bill/s06-06-fix-the-pricing-ratio-error"},{"id":"production-tradeoffs/read-the-token-bill/s07-07-fix-the-units-error"},{"id":"production-tradeoffs/read-the-token-bill/s08-08-write-the-cost-estimator"},{"id":"production-tradeoffs/read-the-token-bill/s09-09-checkpoint"}]},{"slug":"the-model-price-war","steps":[{"id":"production-tradeoffs/the-model-price-war/s01-01-the-price-curve"},{"id":"production-tradeoffs/the-model-price-war/s02-02-three-bets-and-how-they-aged"},{"id":"production-tradeoffs/the-model-price-war/s03-03-which-startup-survived"},{"id":"production-tradeoffs/the-model-price-war/s04-04-the-incumbent-trap"},{"id":"production-tradeoffs/the-model-price-war/s05-05-features-now-viable"},{"id":"production-tradeoffs/the-model-price-war/s06-06-pricing-for-a-war-you-cant-predict"},{"id":"production-tradeoffs/the-model-price-war/s07-07-write-the-cost-model"},{"id":"production-tradeoffs/the-model-price-war/s08-08-checkpoint"}]},{"slug":"batch-realtime-costs","steps":[{"id":"production-tradeoffs/batch-realtime-costs/s01-01-who-is-waiting"},{"id":"production-tradeoffs/batch-realtime-costs/s02-02-which-lane"},{"id":"production-tradeoffs/batch-realtime-costs/s03-03-predict-the-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s04-04-fill-the-miss-rate"},{"id":"production-tradeoffs/batch-realtime-costs/s05-05-fix-the-free-discount"},{"id":"production-tradeoffs/batch-realtime-costs/s06-06-write-monthly-bill"},{"id":"production-tradeoffs/batch-realtime-costs/s07-07-checkpoint"}]}]},{"slug":"debugging-output","lessons":[{"slug":"read-the-trace-not-the-chat","steps":[{"id":"debugging-output/read-the-trace-not-the-chat/s01-01-intro"},{"id":"debugging-output/read-the-trace-not-the-chat/s02-02-where-to-look-first"},{"id":"debugging-output/read-the-trace-not-the-chat/s03-03-the-four-failure-classes"},{"id":"debugging-output/read-the-trace-not-the-chat/s04-04-predict-the-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s05-05-fill-the-find-bad-turn"},{"id":"debugging-output/read-the-trace-not-the-chat/s06-06-fix-the-fix-the-symptom"},{"id":"debugging-output/read-the-trace-not-the-chat/s07-07-fix-the-blame-the-model"},{"id":"debugging-output/read-the-trace-not-the-chat/s08-08-write-the-trace-summary"},{"id":"debugging-output/read-the-trace-not-the-chat/s09-09-checkpoint"}]},{"slug":"the-four-breakage-classes","steps":[{"id":"debugging-output/the-four-breakage-classes/s01-01-intro"},{"id":"debugging-output/the-four-breakage-classes/s02-02-classify-the-refund-bug"},{"id":"debugging-output/the-four-breakage-classes/s03-03-the-trace-anatomy"},{"id":"debugging-output/the-four-breakage-classes/s04-04-predict-the-class"},{"id":"debugging-output/the-four-breakage-classes/s05-05-fill-the-class-name"},{"id":"debugging-output/the-four-breakage-classes/s06-06-fix-the-misclassification"},{"id":"debugging-output/the-four-breakage-classes/s07-07-fix-the-parse-mangle"},{"id":"debugging-output/the-four-breakage-classes/s08-08-write-the-classifier"},{"id":"debugging-output/the-four-breakage-classes/s09-09-checkpoint"}]},{"slug":"five-real-postmortems","steps":[{"id":"debugging-output/five-real-postmortems/s01-01-the-postmortem-template"},{"id":"debugging-output/five-real-postmortems/s02-02-air-canada-refund-lawsuit"},{"id":"debugging-output/five-real-postmortems/s03-03-dpd-and-nyc-mycity"},{"id":"debugging-output/five-real-postmortems/s04-04-classify-the-five"},{"id":"debugging-output/five-real-postmortems/s05-05-recruiter-json-mangle"},{"id":"debugging-output/five-real-postmortems/s06-06-glean-style-retrieval-miss"},{"id":"debugging-output/five-real-postmortems/s07-07-which-fix-catches-the-most"},{"id":"debugging-output/five-real-postmortems/s08-08-write-the-postmortem"},{"id":"debugging-output/five-real-postmortems/s09-09-checkpoint"}]},{"slug":"prediction-failure-triage","steps":[{"id":"debugging-output/prediction-failure-triage/s01-01-reproduce-classify-route"},{"id":"debugging-output/prediction-failure-triage/s02-02-first-move"},{"id":"debugging-output/prediction-failure-triage/s03-03-predict-the-route"},{"id":"debugging-output/prediction-failure-triage/s04-04-fill-the-owners"},{"id":"debugging-output/prediction-failure-triage/s05-05-fix-the-paraphrase-repro"},{"id":"debugging-output/prediction-failure-triage/s06-06-write-the-router"},{"id":"debugging-output/prediction-failure-triage/s07-07-checkpoint"}]}]},{"slug":"capstone","lessons":[{"slug":"pick-a-project-that-ships","steps":[{"id":"capstone/pick-a-project-that-ships/s01-01-the-builders-graveyard"},{"id":"capstone/pick-a-project-that-ships/s02-02-which-project-survives-contact-with-reality"},{"id":"capstone/pick-a-project-that-ships/s03-03-the-three-shapes-that-ship"},{"id":"capstone/pick-a-project-that-ships/s04-04-the-wedge-checklist"},{"id":"capstone/pick-a-project-that-ships/s05-05-spot-the-missing-wedge"},{"id":"capstone/pick-a-project-that-ships/s06-06-scope-down-the-capstone"},{"id":"capstone/pick-a-project-that-ships/s07-07-checkpoint"}]},{"slug":"build-a-cli-agent","steps":[{"id":"capstone/build-a-cli-agent/s01-01-intro"},{"id":"capstone/build-a-cli-agent/s02-02-which-piece-is-missing"},{"id":"capstone/build-a-cli-agent/s03-03-the-tool-loop"},{"id":"capstone/build-a-cli-agent/s04-04-predict-the-final-answer"},{"id":"capstone/build-a-cli-agent/s05-05-fill-the-stop-check"},{"id":"capstone/build-a-cli-agent/s06-06-fix-the-tool-dispatch"},{"id":"capstone/build-a-cli-agent/s07-07-fix-the-message-append"},{"id":"capstone/build-a-cli-agent/s08-08-the-logging-pattern"},{"id":"capstone/build-a-cli-agent/s09-09-write-the-tool-runner"},{"id":"capstone/build-a-cli-agent/s10-10-fix-the-loop-exit"},{"id":"capstone/build-a-cli-agent/s11-11-write-the-agent"},{"id":"capstone/build-a-cli-agent/s12-12-checkpoint"}]},{"slug":"wire-the-real-model","steps":[{"id":"capstone/wire-the-real-model/s01-01-intro"},{"id":"capstone/wire-the-real-model/s02-02-where-the-key-lives"},{"id":"capstone/wire-the-real-model/s03-03-the-sdk-response-shape"},{"id":"capstone/wire-the-real-model/s04-04-predict-the-block-type"},{"id":"capstone/wire-the-real-model/s05-05-fill-the-key-load"},{"id":"capstone/wire-the-real-model/s06-06-fix-the-hardcoded-key"},{"id":"capstone/wire-the-real-model/s07-07-fix-the-block-access"},{"id":"capstone/wire-the-real-model/s08-08-write-the-shape-adapter"},{"id":"capstone/wire-the-real-model/s09-09-checkpoint"}]},{"slug":"validate-tool-inputs","steps":[{"id":"capstone/validate-tool-inputs/s01-01-intro"},{"id":"capstone/validate-tool-inputs/s02-02-which-input-needs-validation"},{"id":"capstone/validate-tool-inputs/s03-03-the-validator-surface"},{"id":"capstone/validate-tool-inputs/s04-04-predict-the-error"},{"id":"capstone/validate-tool-inputs/s05-05-fill-the-validate-call"},{"id":"capstone/validate-tool-inputs/s06-06-fix-the-untyped-tool"},{"id":"capstone/validate-tool-inputs/s07-07-fix-the-error-leak"},{"id":"capstone/validate-tool-inputs/s08-08-write-the-validator"},{"id":"capstone/validate-tool-inputs/s09-09-checkpoint"}]},{"slug":"add-evals-and-traces","steps":[{"id":"capstone/add-evals-and-traces/s01-01-intro"},{"id":"capstone/add-evals-and-traces/s02-02-which-eval-fits"},{"id":"capstone/add-evals-and-traces/s03-03-the-trace-shape"},{"id":"capstone/add-evals-and-traces/s04-04-predict-the-pass-rate"},{"id":"capstone/add-evals-and-traces/s05-05-fill-the-trace-append"},{"id":"capstone/add-evals-and-traces/s06-06-fix-the-trace-mutation"},{"id":"capstone/add-evals-and-traces/s07-07-fix-the-tautological-eval"},{"id":"capstone/add-evals-and-traces/s08-08-write-the-eval-runner"},{"id":"capstone/add-evals-and-traces/s09-09-checkpoint"}]},{"slug":"wire-an-mcp-tool","steps":[{"id":"capstone/wire-an-mcp-tool/s01-01-intro"},{"id":"capstone/wire-an-mcp-tool/s02-02-when-to-use-mcp"},{"id":"capstone/wire-an-mcp-tool/s03-03-the-mcp-protocol"},{"id":"capstone/wire-an-mcp-tool/s04-04-predict-the-tool-list"},{"id":"capstone/wire-an-mcp-tool/s05-05-fill-the-tools-call"},{"id":"capstone/wire-an-mcp-tool/s06-06-fix-the-result-shape"},{"id":"capstone/wire-an-mcp-tool/s07-07-fix-the-iserror-flag"},{"id":"capstone/wire-an-mcp-tool/s08-08-write-the-mcp-bridge"},{"id":"capstone/wire-an-mcp-tool/s09-09-checkpoint"}]},{"slug":"portfolio-ml-system-option","steps":[{"id":"capstone/portfolio-ml-system-option/s01-01-the-other-capstone"},{"id":"capstone/portfolio-ml-system-option/s02-02-what-reviewers-open"},{"id":"capstone/portfolio-ml-system-option/s03-03-predict-the-gate"},{"id":"capstone/portfolio-ml-system-option/s04-04-fill-the-gate"},{"id":"capstone/portfolio-ml-system-option/s05-05-fix-the-train-accuracy-gate"},{"id":"capstone/portfolio-ml-system-option/s06-06-write-the-scope-check"},{"id":"capstone/portfolio-ml-system-option/s07-07-checkpoint"}]}]},{"slug":"agent-harnesses","lessons":[{"slug":"project-workspaces-not-product-names","steps":[{"id":"agent-harnesses/project-workspaces-not-product-names/s01-01-intro"}]},{"slug":"what-a-harness-is","steps":[{"id":"agent-harnesses/what-a-harness-is/s01-01-intro"},{"id":"agent-harnesses/what-a-harness-is/s02-02-which-is-a-harness"},{"id":"agent-harnesses/what-a-harness-is/s03-03-the-four-layers"},{"id":"agent-harnesses/what-a-harness-is/s04-04-predict-the-layer"},{"id":"agent-harnesses/what-a-harness-is/s05-05-fill-the-dispatch"},{"id":"agent-harnesses/what-a-harness-is/s06-06-fix-the-no-retry"},{"id":"agent-harnesses/what-a-harness-is/s07-07-fix-the-hardcoded-registry"},{"id":"agent-harnesses/what-a-harness-is/s08-08-write-mini-harness"},{"id":"agent-harnesses/what-a-harness-is/s09-09-checkpoint"}]},{"slug":"architecting-an-ai-native-workflow","steps":[{"id":"agent-harnesses/architecting-an-ai-native-workflow/s01-01-intro"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s02-02-which-workflow-fits"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s03-03-mapping-the-workflow"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s04-04-predict-the-readiness-score"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s05-05-fill-the-required-fields"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s06-06-fix-the-implicit-policy"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s07-07-fix-the-wrong-metric"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s08-08-write-the-workflow-validator"},{"id":"agent-harnesses/architecting-an-ai-native-workflow/s09-09-checkpoint"}]},{"slug":"five-industries-walked-through","steps":[{"id":"agent-harnesses/five-industries-walked-through/s01-01-the-case-study-method"},{"id":"agent-harnesses/five-industries-walked-through/s02-02-home-services"},{"id":"agent-harnesses/five-industries-walked-through/s03-03-which-task-belongs-to-the-agent"},{"id":"agent-harnesses/five-industries-walked-through/s04-04-insurance-brokerage"},{"id":"agent-harnesses/five-industries-walked-through/s05-05-recruiting-firm"},{"id":"agent-harnesses/five-industries-walked-through/s06-06-which-industry-wedges-fastest"},{"id":"agent-harnesses/five-industries-walked-through/s07-07-the-incumbent-trap"},{"id":"agent-harnesses/five-industries-walked-through/s08-08-write-the-process-debt-audit"},{"id":"agent-harnesses/five-industries-walked-through/s09-09-checkpoint"}]}]},{"slug":"ai-image-generation","lessons":[{"slug":"the-image-model-landscape","steps":[{"id":"ai-image-generation/the-image-model-landscape/s01-01-the-six-families"},{"id":"ai-image-generation/the-image-model-landscape/s02-02-flux-vs-midjourney"},{"id":"ai-image-generation/the-image-model-landscape/s03-03-pick-the-model"},{"id":"ai-image-generation/the-image-model-landscape/s04-04-nano-banana-and-the-batch-economics"},{"id":"ai-image-generation/the-image-model-landscape/s05-05-text-in-image"},{"id":"ai-image-generation/the-image-model-landscape/s06-06-which-model-for-text"},{"id":"ai-image-generation/the-image-model-landscape/s07-07-the-decision-tree"},{"id":"ai-image-generation/the-image-model-landscape/s08-08-write-pick-image-model"},{"id":"ai-image-generation/the-image-model-landscape/s09-09-checkpoint"}]},{"slug":"prompting-for-real-output","steps":[{"id":"ai-image-generation/prompting-for-real-output/s01-01-why-the-girl-with-red-hair-fails"},{"id":"ai-image-generation/prompting-for-real-output/s02-02-the-eight-knobs"},{"id":"ai-image-generation/prompting-for-real-output/s03-03-fix-the-vague-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s04-04-controlnet-and-reference"},{"id":"ai-image-generation/prompting-for-real-output/s05-05-which-knob-is-missing"},{"id":"ai-image-generation/prompting-for-real-output/s06-06-failure-modes"},{"id":"ai-image-generation/prompting-for-real-output/s07-07-write-score-image-prompt"},{"id":"ai-image-generation/prompting-for-real-output/s08-08-checkpoint"}]},{"slug":"the-image-pipeline","steps":[{"id":"ai-image-generation/the-image-pipeline/s01-01-the-six-stages"},{"id":"ai-image-generation/the-image-pipeline/s02-02-batch-and-filter"},{"id":"ai-image-generation/the-image-pipeline/s03-03-which-stage-is-broken"},{"id":"ai-image-generation/the-image-pipeline/s04-04-format-and-platform"},{"id":"ai-image-generation/the-image-pipeline/s05-05-fill-the-pipeline"},{"id":"ai-image-generation/the-image-pipeline/s06-06-write-cost-for-batch"},{"id":"ai-image-generation/the-image-pipeline/s07-07-checkpoint"}]}]},{"slug":"ai-video-generation","lessons":[{"slug":"the-video-model-lineup","steps":[{"id":"ai-video-generation/the-video-model-lineup/s01-01-the-landscape"},{"id":"ai-video-generation/the-video-model-lineup/s02-02-sora-and-veo"},{"id":"ai-video-generation/the-video-model-lineup/s03-03-higgsfield-and-camera-control"},{"id":"ai-video-generation/the-video-model-lineup/s04-04-the-second-wave"},{"id":"ai-video-generation/the-video-model-lineup/s05-05-text-vs-image-vs-video"},{"id":"ai-video-generation/the-video-model-lineup/s06-06-which-lane"},{"id":"ai-video-generation/the-video-model-lineup/s07-07-pick-the-model"},{"id":"ai-video-generation/the-video-model-lineup/s08-08-write-pick-video-model"},{"id":"ai-video-generation/the-video-model-lineup/s09-09-checkpoint"}]},{"slug":"camera-control-and-motion","steps":[{"id":"ai-video-generation/camera-control-and-motion/s01-01-the-static-camera-problem"},{"id":"ai-video-generation/camera-control-and-motion/s02-02-shot-vocabulary"},{"id":"ai-video-generation/camera-control-and-motion/s03-03-which-shot-is-which"},{"id":"ai-video-generation/camera-control-and-motion/s04-04-camera-moves"},{"id":"ai-video-generation/camera-control-and-motion/s05-05-the-i2v-keyframe-pattern"},{"id":"ai-video-generation/camera-control-and-motion/s06-06-spot-the-generic-prompt"},{"id":"ai-video-generation/camera-control-and-motion/s07-07-write-audit-shot-list"},{"id":"ai-video-generation/camera-control-and-motion/s08-08-checkpoint"}]},{"slug":"the-cost-math","steps":[{"id":"ai-video-generation/the-cost-math/s01-01-the-unit-economics"},{"id":"ai-video-generation/the-cost-math/s02-02-the-retake-rate-reality"},{"id":"ai-video-generation/the-cost-math/s03-03-which-tier-for-this-budget"},{"id":"ai-video-generation/the-cost-math/s04-04-the-60-second-promo"},{"id":"ai-video-generation/the-cost-math/s05-05-write-cost-per-minute"},{"id":"ai-video-generation/the-cost-math/s06-06-checkpoint"}]}]},{"slug":"programmatic-design","lessons":[{"slug":"why-programmatic","steps":[{"id":"programmatic-design/why-programmatic/s01-01-intro"},{"id":"programmatic-design/why-programmatic/s02-02-the-three-jobs"},{"id":"programmatic-design/why-programmatic/s03-03-which-task-is-code"},{"id":"programmatic-design/why-programmatic/s04-04-parametric-vs-one-shot"},{"id":"programmatic-design/why-programmatic/s05-05-the-classifier"},{"id":"programmatic-design/why-programmatic/s06-06-write-classify-video-task"},{"id":"programmatic-design/why-programmatic/s07-07-checkpoint"}]},{"slug":"hyperframes-and-remotion","steps":[{"id":"programmatic-design/hyperframes-and-remotion/s01-01-the-two-tools"},{"id":"programmatic-design/hyperframes-and-remotion/s02-02-hyperframes-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s03-03-hyperframes-cli"},{"id":"programmatic-design/hyperframes-and-remotion/s04-04-remotion-tour"},{"id":"programmatic-design/hyperframes-and-remotion/s05-05-the-decision-tree"},{"id":"programmatic-design/hyperframes-and-remotion/s06-06-pick-the-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s07-07-claude-design-html"},{"id":"programmatic-design/hyperframes-and-remotion/s08-08-write-pick-tool"},{"id":"programmatic-design/hyperframes-and-remotion/s09-09-checkpoint"}]},{"slug":"the-ai-native-design-pipeline","steps":[{"id":"programmatic-design/the-ai-native-design-pipeline/s01-01-the-seven-steps"},{"id":"programmatic-design/the-ai-native-design-pipeline/s02-02-where-ai-fits-each-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s03-03-which-model-which-step"},{"id":"programmatic-design/the-ai-native-design-pipeline/s04-04-website-to-hyperframes"},{"id":"programmatic-design/the-ai-native-design-pipeline/s05-05-the-cost-model"},{"id":"programmatic-design/the-ai-native-design-pipeline/s06-06-score-the-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s07-07-write-score-pipeline"},{"id":"programmatic-design/the-ai-native-design-pipeline/s08-08-checkpoint"}]}]},{"slug":"harness-engineering","lessons":[{"slug":"the-harness-engineering-mindset","steps":[{"id":"harness-engineering/the-harness-engineering-mindset/s01-01-the-model-is-only-one-input"},{"id":"harness-engineering/the-harness-engineering-mindset/s02-02-the-six-pieces-of-a-harness"},{"id":"harness-engineering/the-harness-engineering-mindset/s03-03-which-piece-is-missing"},{"id":"harness-engineering/the-harness-engineering-mindset/s04-04-the-skill-issue-reframe"},{"id":"harness-engineering/the-harness-engineering-mindset/s05-05-pick-the-fix"},{"id":"harness-engineering/the-harness-engineering-mindset/s06-06-the-harness-gap"},{"id":"harness-engineering/the-harness-engineering-mindset/s07-07-write-the-harness-inventory"},{"id":"harness-engineering/the-harness-engineering-mindset/s08-08-checkpoint"}]},{"slug":"the-ratchet","steps":[{"id":"harness-engineering/the-ratchet/s01-01-failures-are-signals-not-flukes"},{"id":"harness-engineering/the-ratchet/s02-02-every-line-traces-to-a-failure"},{"id":"harness-engineering/the-ratchet/s03-03-which-rule-earned-its-line"},{"id":"harness-engineering/the-ratchet/s04-04-working-backwards-from-behavior"},{"id":"harness-engineering/the-ratchet/s05-05-the-ratchet-log"},{"id":"harness-engineering/the-ratchet/s06-06-spot-the-cruft"},{"id":"harness-engineering/the-ratchet/s07-07-write-the-ratchet-update"},{"id":"harness-engineering/the-ratchet/s08-08-audit-the-ratchet"},{"id":"harness-engineering/the-ratchet/s09-09-checkpoint"}]},{"slug":"context-engineering","steps":[{"id":"harness-engineering/context-engineering/s01-01-the-context-rot"},{"id":"harness-engineering/context-engineering/s02-02-compaction"},{"id":"harness-engineering/context-engineering/s03-03-tool-call-offloading"},{"id":"harness-engineering/context-engineering/s04-04-progressive-disclosure"},{"id":"harness-engineering/context-engineering/s05-05-which-technique-fits"},{"id":"harness-engineering/context-engineering/s06-06-the-memory-hierarchy"},{"id":"harness-engineering/context-engineering/s07-07-classify-the-data"},{"id":"harness-engineering/context-engineering/s08-08-write-the-context-budget"},{"id":"harness-engineering/context-engineering/s09-09-checkpoint"}]},{"slug":"long-horizon-execution","steps":[{"id":"harness-engineering/long-horizon-execution/s01-01-the-40-step-problem"},{"id":"harness-engineering/long-horizon-execution/s02-02-loops-intercepting-early-stops"},{"id":"harness-engineering/long-horizon-execution/s03-03-planning-as-a-separate-step"},{"id":"harness-engineering/long-horizon-execution/s04-04-splits-generation-vs-evaluation"},{"id":"harness-engineering/long-horizon-execution/s05-05-which-pattern-fixes-the-failure"},{"id":"harness-engineering/long-horizon-execution/s06-06-hooks-as-the-enforcement-layer"},{"id":"harness-engineering/long-horizon-execution/s07-07-the-type-check-backpressure-pattern"},{"id":"harness-engineering/long-horizon-execution/s08-08-write-the-hook-router"},{"id":"harness-engineering/long-horizon-execution/s09-09-write-the-completion-guard"},{"id":"harness-engineering/long-horizon-execution/s10-10-checkpoint"}]},{"slug":"the-haas-shift","steps":[{"id":"harness-engineering/the-haas-shift/s01-01-llm-apis-to-harness-apis"},{"id":"harness-engineering/the-haas-shift/s02-02-the-convergence"},{"id":"harness-engineering/the-haas-shift/s03-03-which-haas-fits"},{"id":"harness-engineering/the-haas-shift/s04-04-harnesses-dont-shrink-they-move"},{"id":"harness-engineering/the-haas-shift/s05-05-the-overfitting-feedback-loop"},{"id":"harness-engineering/the-haas-shift/s06-06-mcp-security-as-prompt-injection"},{"id":"harness-engineering/the-haas-shift/s07-07-which-component-is-debt"},{"id":"harness-engineering/the-haas-shift/s08-08-write-the-build-vs-buy"},{"id":"harness-engineering/the-haas-shift/s09-09-checkpoint"}]},{"slug":"pipeline-boundaries","steps":[{"id":"harness-engineering/pipeline-boundaries/s01-01-the-boundary-is-an-artifact"},{"id":"harness-engineering/pipeline-boundaries/s02-02-spot-the-risky-handoff"},{"id":"harness-engineering/pipeline-boundaries/s03-03-predict-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s04-04-fill-the-gate"},{"id":"harness-engineering/pipeline-boundaries/s05-05-fix-the-overtrust"},{"id":"harness-engineering/pipeline-boundaries/s06-06-write-the-advance-check"},{"id":"harness-engineering/pipeline-boundaries/s07-07-checkpoint"}]}]},{"slug":"intro-to-terminal","lessons":[{"slug":"what-the-terminal-is","steps":[{"id":"intro-to-terminal/what-the-terminal-is/s01-01-not-a-cockpit"},{"id":"intro-to-terminal/what-the-terminal-is/s02-02-what-it-actually-is"},{"id":"intro-to-terminal/what-the-terminal-is/s03-03-open-it"},{"id":"intro-to-terminal/what-the-terminal-is/s04-04-which-app"},{"id":"intro-to-terminal/what-the-terminal-is/s05-05-the-prompt"},{"id":"intro-to-terminal/what-the-terminal-is/s06-06-what-a-command-is"},{"id":"intro-to-terminal/what-the-terminal-is/s07-07-checkpoint"}]},{"slug":"moving-around","steps":[{"id":"intro-to-terminal/moving-around/s01-01-the-filesystem-is-a-tree"},{"id":"intro-to-terminal/moving-around/s02-02-pwd-and-ls"},{"id":"intro-to-terminal/moving-around/s03-03-where-am-i"},{"id":"intro-to-terminal/moving-around/s04-04-read-this-listing"},{"id":"intro-to-terminal/moving-around/s05-05-cd"},{"id":"intro-to-terminal/moving-around/s06-06-follow-the-moves"},{"id":"intro-to-terminal/moving-around/s07-07-pick-the-command"},{"id":"intro-to-terminal/moving-around/s08-08-checkpoint"}]},{"slug":"making-things","steps":[{"id":"intro-to-terminal/making-things/s01-01-mkdir-and-touch"},{"id":"intro-to-terminal/making-things/s02-02-folder-or-file"},{"id":"intro-to-terminal/making-things/s03-03-cat-and-echo"},{"id":"intro-to-terminal/making-things/s04-04-reading-a-file"},{"id":"intro-to-terminal/making-things/s05-05-rm-carefully"},{"id":"intro-to-terminal/making-things/s06-06-the-no-undo-rule"},{"id":"intro-to-terminal/making-things/s07-07-string-it-together"},{"id":"intro-to-terminal/making-things/s08-08-checkpoint"}]}]},{"slug":"intro-to-claude-cli","lessons":[{"slug":"what-the-claude-cli-is","steps":[{"id":"intro-to-claude-cli/what-the-claude-cli-is/s01-01-the-glass-is-gone"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s02-02-chat-vs-cli"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s03-03-which-tool"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s04-04-the-agent-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s05-05-read-the-loop"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s06-06-permission-and-limits"},{"id":"intro-to-claude-cli/what-the-claude-cli-is/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-claude-cli/install-and-sign-in/s01-01-two-ways-to-install"},{"id":"intro-to-claude-cli/install-and-sign-in/s02-02-run-the-installer"},{"id":"intro-to-claude-cli/install-and-sign-in/s03-03-install-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s04-04-sign-in"},{"id":"intro-to-claude-cli/install-and-sign-in/s05-05-what-sign-in-did"},{"id":"intro-to-claude-cli/install-and-sign-in/s06-06-auth-check"},{"id":"intro-to-claude-cli/install-and-sign-in/s07-07-account-not-api-key"},{"id":"intro-to-claude-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-claude-cli/your-first-session/s01-01-start-in-a-folder"},{"id":"intro-to-claude-cli/your-first-session/s02-02-your-first-instruction"},{"id":"intro-to-claude-cli/your-first-session/s03-03-watch-the-loop"},{"id":"intro-to-claude-cli/your-first-session/s04-04-read-what-it-did"},{"id":"intro-to-claude-cli/your-first-session/s05-05-one-shot-or-interactive"},{"id":"intro-to-claude-cli/your-first-session/s06-06-chat-or-cli-decision"},{"id":"intro-to-claude-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-claude-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"intro-to-codex-cli","lessons":[{"slug":"another-tool-same-idea","steps":[{"id":"intro-to-codex-cli/another-tool-same-idea/s01-01-why-a-second-tool"},{"id":"intro-to-codex-cli/another-tool-same-idea/s02-02-what-carries-over"},{"id":"intro-to-codex-cli/another-tool-same-idea/s03-03-what-carries-over-check"},{"id":"intro-to-codex-cli/another-tool-same-idea/s04-04-what-is-different"},{"id":"intro-to-codex-cli/another-tool-same-idea/s05-05-spot-the-difference"},{"id":"intro-to-codex-cli/another-tool-same-idea/s06-06-when-to-reach-for-which"},{"id":"intro-to-codex-cli/another-tool-same-idea/s07-07-checkpoint"}]},{"slug":"install-and-sign-in","steps":[{"id":"intro-to-codex-cli/install-and-sign-in/s01-01-the-install-paths"},{"id":"intro-to-codex-cli/install-and-sign-in/s02-02-installing-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s03-03-verify-it"},{"id":"intro-to-codex-cli/install-and-sign-in/s04-04-install-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s05-05-sign-in"},{"id":"intro-to-codex-cli/install-and-sign-in/s06-06-account-vs-api-key"},{"id":"intro-to-codex-cli/install-and-sign-in/s07-07-auth-check"},{"id":"intro-to-codex-cli/install-and-sign-in/s08-08-checkpoint"}]},{"slug":"your-first-session","steps":[{"id":"intro-to-codex-cli/your-first-session/s01-01-start-and-instruct"},{"id":"intro-to-codex-cli/your-first-session/s02-02-the-loop-again"},{"id":"intro-to-codex-cli/your-first-session/s03-03-read-what-it-did"},{"id":"intro-to-codex-cli/your-first-session/s04-04-usage-patterns"},{"id":"intro-to-codex-cli/your-first-session/s05-05-usage-scenario"},{"id":"intro-to-codex-cli/your-first-session/s06-06-running-both"},{"id":"intro-to-codex-cli/your-first-session/s07-07-where-this-goes"},{"id":"intro-to-codex-cli/your-first-session/s08-08-checkpoint"}]}]},{"slug":"team-skills","lessons":[{"slug":"what-a-skill-is","steps":[{"id":"team-skills/what-a-skill-is/s01-01-the-re-explaining-problem"},{"id":"team-skills/what-a-skill-is/s02-02-a-skill-defined"},{"id":"team-skills/what-a-skill-is/s03-03-a-real-skill"},{"id":"team-skills/what-a-skill-is/s04-04-what-a-skill-is-check"},{"id":"team-skills/what-a-skill-is/s05-05-skill-vs-prompt-vs-agent"},{"id":"team-skills/what-a-skill-is/s06-06-skill-or-agent"},{"id":"team-skills/what-a-skill-is/s07-07-checkpoint"}]},{"slug":"skills-as-a-team-thing","steps":[{"id":"team-skills/skills-as-a-team-thing/s01-01-the-team-problem"},{"id":"team-skills/skills-as-a-team-thing/s02-02-org-wide-skills"},{"id":"team-skills/skills-as-a-team-thing/s03-03-sharing-and-the-directory"},{"id":"team-skills/skills-as-a-team-thing/s04-04-deployment-check"},{"id":"team-skills/skills-as-a-team-thing/s05-05-collaborating-through-skills"},{"id":"team-skills/skills-as-a-team-thing/s06-06-sharing-scenario"},{"id":"team-skills/skills-as-a-team-thing/s07-07-checkpoint"}]},{"slug":"role-playbooks","steps":[{"id":"team-skills/role-playbooks/s01-01-hr-skills"},{"id":"team-skills/role-playbooks/s02-02-legal-skills"},{"id":"team-skills/role-playbooks/s03-03-operations-skills"},{"id":"team-skills/role-playbooks/s04-04-match-the-skill"},{"id":"team-skills/role-playbooks/s05-05-skill-or-one-off"},{"id":"team-skills/role-playbooks/s06-06-skill-or-one-off-check"},{"id":"team-skills/role-playbooks/s07-07-checkpoint"}]},{"slug":"governance-and-trust","steps":[{"id":"team-skills/governance-and-trust/s01-01-a-skill-can-do-real-things"},{"id":"team-skills/governance-and-trust/s02-02-treat-it-like-software"},{"id":"team-skills/governance-and-trust/s03-03-evaluate-before-you-deploy"},{"id":"team-skills/governance-and-trust/s04-04-access-control"},{"id":"team-skills/governance-and-trust/s05-05-governance-scenario"},{"id":"team-skills/governance-and-trust/s06-06-a-rollout-checklist"},{"id":"team-skills/governance-and-trust/s07-07-checkpoint"}]}]},{"slug":"dataframes-numpy-pandas","lessons":[{"slug":"ml-package-map","steps":[{"id":"dataframes-numpy-pandas/ml-package-map/s01-01-intro"},{"id":"dataframes-numpy-pandas/ml-package-map/s02-02-match-the-tool"},{"id":"dataframes-numpy-pandas/ml-package-map/s03-03-predict-sys-modules"},{"id":"dataframes-numpy-pandas/ml-package-map/s04-04-fill-the-registry-key"},{"id":"dataframes-numpy-pandas/ml-package-map/s05-05-fix-the-swapped-map"},{"id":"dataframes-numpy-pandas/ml-package-map/s06-06-write-find-unused"},{"id":"dataframes-numpy-pandas/ml-package-map/s07-07-checkpoint"}]},{"slug":"array-and-series-shapes","steps":[{"id":"dataframes-numpy-pandas/array-and-series-shapes/s01-01-intro"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/array-and-series-shapes/s07-07-checkpoint"}]},{"slug":"dataframe-selection-and-cleaning","steps":[{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s01-01-intro"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/dataframe-selection-and-cleaning/s07-07-checkpoint"}]},{"slug":"missing-values-and-types","steps":[{"id":"dataframes-numpy-pandas/missing-values-and-types/s01-01-intro"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/missing-values-and-types/s07-07-checkpoint"}]},{"slug":"groupby-joins-and-features","steps":[{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s01-01-intro"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/groupby-joins-and-features/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataframe","steps":[{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s01-01-intro"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s02-02-choose-the-risk"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s03-03-predict-the-checks"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s04-04-fill-the-return"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s05-05-fix-the-overtrust"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s06-06-write-count-ready"},{"id":"dataframes-numpy-pandas/mission-api-to-dataframe/s07-07-checkpoint"}]}]},{"slug":"sql-for-ml-datasets","lessons":[{"slug":"select-filter-aggregate","steps":[{"id":"sql-for-ml-datasets/select-filter-aggregate/s01-01-intro"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/select-filter-aggregate/s07-07-checkpoint"}]},{"slug":"joins-and-label-windows","steps":[{"id":"sql-for-ml-datasets/joins-and-label-windows/s01-01-intro"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/joins-and-label-windows/s07-07-checkpoint"}]},{"slug":"sql-quality-checks","steps":[{"id":"sql-for-ml-datasets/sql-quality-checks/s01-01-intro"},{"id":"sql-for-ml-datasets/sql-quality-checks/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/sql-quality-checks/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/sql-quality-checks/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/sql-quality-checks/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/sql-quality-checks/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/sql-quality-checks/s07-07-checkpoint"}]},{"slug":"mission-sql-feature-query-lab","steps":[{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s01-01-intro"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s02-02-choose-the-risk"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s03-03-predict-the-checks"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s04-04-fill-the-return"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s05-05-fix-the-overtrust"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s06-06-write-count-ready"},{"id":"sql-for-ml-datasets/mission-sql-feature-query-lab/s07-07-checkpoint"}]}]},{"slug":"dataset-pipelines","lessons":[{"slug":"csv-jsonl-parquet-tradeoffs","steps":[{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s01-01-intro"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s02-02-choose-the-risk"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s03-03-predict-the-checks"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s04-04-fill-the-return"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s06-06-write-count-ready"},{"id":"dataset-pipelines/csv-jsonl-parquet-tradeoffs/s07-07-checkpoint"}]},{"slug":"pagination-retries-checkpoints","steps":[{"id":"dataset-pipelines/pagination-retries-checkpoints/s01-01-intro"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s04-04-fill-the-return"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s06-06-write-count-ready"},{"id":"dataset-pipelines/pagination-retries-checkpoints/s07-07-checkpoint"}]},{"slug":"schema-drift-and-contracts","steps":[{"id":"dataset-pipelines/schema-drift-and-contracts/s01-01-intro"},{"id":"dataset-pipelines/schema-drift-and-contracts/s02-02-choose-the-risk"},{"id":"dataset-pipelines/schema-drift-and-contracts/s03-03-predict-the-checks"},{"id":"dataset-pipelines/schema-drift-and-contracts/s04-04-fill-the-return"},{"id":"dataset-pipelines/schema-drift-and-contracts/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/schema-drift-and-contracts/s06-06-write-count-ready"},{"id":"dataset-pipelines/schema-drift-and-contracts/s07-07-checkpoint"}]},{"slug":"pipeline-orchestration-shape","steps":[{"id":"dataset-pipelines/pipeline-orchestration-shape/s01-01-intro"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s02-02-choose-the-risk"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s03-03-predict-the-checks"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s04-04-fill-the-return"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s06-06-write-count-ready"},{"id":"dataset-pipelines/pipeline-orchestration-shape/s07-07-checkpoint"}]},{"slug":"mission-api-to-dataset-pipeline","steps":[{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s01-01-intro"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s02-02-choose-the-risk"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s03-03-predict-the-checks"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s04-04-fill-the-return"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s05-05-fix-the-overtrust"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s06-06-write-count-ready"},{"id":"dataset-pipelines/mission-api-to-dataset-pipeline/s07-07-checkpoint"}]}]},{"slug":"ml-math-and-stats","lessons":[{"slug":"vectors-matrices-dot-products","steps":[{"id":"ml-math-and-stats/vectors-matrices-dot-products/s01-01-intro"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s04-04-fill-the-return"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s06-06-write-count-ready"},{"id":"ml-math-and-stats/vectors-matrices-dot-products/s07-07-checkpoint"}]},{"slug":"probability-and-baselines","steps":[{"id":"ml-math-and-stats/probability-and-baselines/s01-01-intro"},{"id":"ml-math-and-stats/probability-and-baselines/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/probability-and-baselines/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/probability-and-baselines/s04-04-fill-the-return"},{"id":"ml-math-and-stats/probability-and-baselines/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/probability-and-baselines/s06-06-write-count-ready"},{"id":"ml-math-and-stats/probability-and-baselines/s07-07-checkpoint"}]},{"slug":"distributions-sampling-variance","steps":[{"id":"ml-math-and-stats/distributions-sampling-variance/s01-01-intro"},{"id":"ml-math-and-stats/distributions-sampling-variance/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/distributions-sampling-variance/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/distributions-sampling-variance/s04-04-fill-the-return"},{"id":"ml-math-and-stats/distributions-sampling-variance/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/distributions-sampling-variance/s06-06-write-variance"},{"id":"ml-math-and-stats/distributions-sampling-variance/s07-07-checkpoint"}]},{"slug":"bias-variance-and-leakage","steps":[{"id":"ml-math-and-stats/bias-variance-and-leakage/s01-01-intro"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s04-04-fill-the-return"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s06-06-write-diagnose"},{"id":"ml-math-and-stats/bias-variance-and-leakage/s07-07-checkpoint"}]},{"slug":"mission-statistical-sanity-check","steps":[{"id":"ml-math-and-stats/mission-statistical-sanity-check/s01-01-intro"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s02-02-choose-the-risk"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s03-03-predict-the-checks"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s04-04-fill-the-return"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s05-05-fix-the-overtrust"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s06-06-write-count-ready"},{"id":"ml-math-and-stats/mission-statistical-sanity-check/s07-07-checkpoint"}]}]},{"slug":"supervised-learning-workflows","lessons":[{"slug":"labels-features-splits","steps":[{"id":"supervised-learning-workflows/labels-features-splits/s01-01-intro"},{"id":"supervised-learning-workflows/labels-features-splits/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/labels-features-splits/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/labels-features-splits/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/labels-features-splits/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/labels-features-splits/s06-06-write-split-audit"},{"id":"supervised-learning-workflows/labels-features-splits/s07-07-checkpoint"}]},{"slug":"baselines-before-fancy-models","steps":[{"id":"supervised-learning-workflows/baselines-before-fancy-models/s01-01-intro"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s06-06-write-baseline-comparison"},{"id":"supervised-learning-workflows/baselines-before-fancy-models/s07-07-checkpoint"}]},{"slug":"training-and-prediction-loop","steps":[{"id":"supervised-learning-workflows/training-and-prediction-loop/s01-01-intro"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s06-06-write-train"},{"id":"supervised-learning-workflows/training-and-prediction-loop/s07-07-checkpoint"}]},{"slug":"overfitting-and-regularization","steps":[{"id":"supervised-learning-workflows/overfitting-and-regularization/s01-01-intro"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s06-06-write-overfit-audit"},{"id":"supervised-learning-workflows/overfitting-and-regularization/s07-07-checkpoint"}]},{"slug":"mission-baseline-model-showdown","steps":[{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s01-01-intro"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s02-02-choose-the-risk"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s03-03-predict-the-checks"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s04-04-fill-the-return"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s05-05-fix-the-overtrust"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s06-06-write-classifier-brief"},{"id":"supervised-learning-workflows/mission-baseline-model-showdown/s07-07-checkpoint"}]}]},{"slug":"unsupervised-learning-and-embeddings","lessons":[{"slug":"clustering-without-labels","steps":[{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s06-06-write-cluster-summary"},{"id":"unsupervised-learning-and-embeddings/clustering-without-labels/s07-07-checkpoint"}]},{"slug":"dimensionality-and-distance","steps":[{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s06-06-write-euclidean"},{"id":"unsupervised-learning-and-embeddings/dimensionality-and-distance/s07-07-checkpoint"}]},{"slug":"embeddings-as-features","steps":[{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s06-06-write-retrieval-receipt"},{"id":"unsupervised-learning-and-embeddings/embeddings-as-features/s07-07-checkpoint"}]},{"slug":"mission-tiny-recommender","steps":[{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s01-01-intro"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s02-02-choose-the-risk"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s03-03-predict-the-checks"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s04-04-fill-the-return"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s05-05-fix-the-overtrust"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s06-06-write-recommendation-card"},{"id":"unsupervised-learning-and-embeddings/mission-tiny-recommender/s07-07-checkpoint"}]}]},{"slug":"metrics-and-error-analysis","lessons":[{"slug":"confusion-matrix","steps":[{"id":"metrics-and-error-analysis/confusion-matrix/s01-01-intro"},{"id":"metrics-and-error-analysis/confusion-matrix/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/confusion-matrix/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/confusion-matrix/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/confusion-matrix/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/confusion-matrix/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/confusion-matrix/s07-07-checkpoint"}]},{"slug":"precision-recall-thresholds","steps":[{"id":"metrics-and-error-analysis/precision-recall-thresholds/s01-01-intro"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/precision-recall-thresholds/s07-07-checkpoint"}]},{"slug":"slice-based-error-analysis","steps":[{"id":"metrics-and-error-analysis/slice-based-error-analysis/s01-01-intro"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/slice-based-error-analysis/s07-07-checkpoint"}]},{"slug":"regression-metrics-and-residuals","steps":[{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s01-01-intro"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/regression-metrics-and-residuals/s07-07-checkpoint"}]},{"slug":"mission-confusion-matrix-triage","steps":[{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s01-01-intro"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s02-02-choose-the-risk"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s03-03-predict-the-checks"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s04-04-fill-the-return"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s05-05-fix-the-overtrust"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s06-06-write-count-ready"},{"id":"metrics-and-error-analysis/mission-confusion-matrix-triage/s07-07-checkpoint"}]}]},{"slug":"pytorch-tensors-and-autograd","lessons":[{"slug":"tensors-and-shapes","steps":[{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/tensors-and-shapes/s07-07-checkpoint"}]},{"slug":"broadcasting-and-vectorization","steps":[{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/broadcasting-and-vectorization/s07-07-checkpoint"}]},{"slug":"autograd-mental-model","steps":[{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/autograd-mental-model/s07-07-checkpoint"}]},{"slug":"mission-tensor-playground","steps":[{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s01-01-intro"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s02-02-choose-the-risk"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s03-03-predict-the-checks"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s04-04-fill-the-return"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s05-05-fix-the-overtrust"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s06-06-write-count-ready"},{"id":"pytorch-tensors-and-autograd/mission-tensor-playground/s07-07-checkpoint"}]}]},{"slug":"training-loops-and-optimizers","lessons":[{"slug":"loss-forward-backward-step","steps":[{"id":"training-loops-and-optimizers/loss-forward-backward-step/s01-01-intro"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/loss-forward-backward-step/s07-07-checkpoint"}]},{"slug":"gradient-descent-by-hand","steps":[{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s01-01-intro"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/gradient-descent-by-hand/s07-07-checkpoint"}]},{"slug":"optimizers-and-schedulers","steps":[{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s01-01-intro"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/optimizers-and-schedulers/s07-07-checkpoint"}]},{"slug":"checkpoints-and-reproducibility","steps":[{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s01-01-intro"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/checkpoints-and-reproducibility/s07-07-checkpoint"}]},{"slug":"mission-overfit-then-recover","steps":[{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s01-01-intro"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s02-02-choose-the-risk"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s03-03-predict-the-checks"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s04-04-fill-the-return"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s05-05-fix-the-overtrust"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s06-06-write-count-ready"},{"id":"training-loops-and-optimizers/mission-overfit-then-recover/s07-07-checkpoint"}]}]},{"slug":"deep-learning-architectures","lessons":[{"slug":"cnns-and-local-patterns","steps":[{"id":"deep-learning-architectures/cnns-and-local-patterns/s01-01-intro"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s04-04-fill-the-return"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s06-06-write-count-ready"},{"id":"deep-learning-architectures/cnns-and-local-patterns/s07-07-checkpoint"}]},{"slug":"tokenizers-and-context-budget","steps":[{"id":"deep-learning-architectures/tokenizers-and-context-budget/s01-01-intro"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s04-04-fill-the-return"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s06-06-write-count-ready"},{"id":"deep-learning-architectures/tokenizers-and-context-budget/s07-07-checkpoint"}]},{"slug":"attention-and-transformer-blocks","steps":[{"id":"deep-learning-architectures/attention-and-transformer-blocks/s01-01-intro"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s04-04-fill-the-return"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s06-06-write-count-ready"},{"id":"deep-learning-architectures/attention-and-transformer-blocks/s07-07-checkpoint"}]},{"slug":"decoding-kv-cache-quantization","steps":[{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s01-01-intro"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s04-04-fill-the-return"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s06-06-write-count-ready"},{"id":"deep-learning-architectures/decoding-kv-cache-quantization/s07-07-checkpoint"}]},{"slug":"mission-architecture-tradeoff-note","steps":[{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s01-01-intro"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s02-02-choose-the-risk"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s03-03-predict-the-checks"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s04-04-fill-the-return"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s05-05-fix-the-overtrust"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s06-06-write-count-ready"},{"id":"deep-learning-architectures/mission-architecture-tradeoff-note/s07-07-checkpoint"}]}]},{"slug":"feature-experiments-registries","lessons":[{"slug":"feature-pipeline-contracts","steps":[{"id":"feature-experiments-registries/feature-pipeline-contracts/s01-01-intro"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s04-04-fill-the-return"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s06-06-write-count-ready"},{"id":"feature-experiments-registries/feature-pipeline-contracts/s07-07-checkpoint"}]},{"slug":"train-inference-skew","steps":[{"id":"feature-experiments-registries/train-inference-skew/s01-01-intro"},{"id":"feature-experiments-registries/train-inference-skew/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/train-inference-skew/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/train-inference-skew/s04-04-fill-the-return"},{"id":"feature-experiments-registries/train-inference-skew/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/train-inference-skew/s06-06-write-count-ready"},{"id":"feature-experiments-registries/train-inference-skew/s07-07-checkpoint"}]},{"slug":"experiment-tracker-lite","steps":[{"id":"feature-experiments-registries/experiment-tracker-lite/s01-01-intro"},{"id":"feature-experiments-registries/experiment-tracker-lite/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/experiment-tracker-lite/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/experiment-tracker-lite/s04-04-fill-the-return"},{"id":"feature-experiments-registries/experiment-tracker-lite/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/experiment-tracker-lite/s06-06-write-count-ready"},{"id":"feature-experiments-registries/experiment-tracker-lite/s07-07-checkpoint"}]},{"slug":"model-registry-shape","steps":[{"id":"feature-experiments-registries/model-registry-shape/s01-01-intro"},{"id":"feature-experiments-registries/model-registry-shape/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/model-registry-shape/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/model-registry-shape/s04-04-fill-the-return"},{"id":"feature-experiments-registries/model-registry-shape/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/model-registry-shape/s06-06-write-count-ready"},{"id":"feature-experiments-registries/model-registry-shape/s07-07-checkpoint"}]},{"slug":"mission-feature-pipeline-and-tracker","steps":[{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s01-01-intro"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s02-02-choose-the-risk"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s03-03-predict-the-checks"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s04-04-fill-the-return"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s05-05-fix-the-overtrust"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s06-06-write-count-ready"},{"id":"feature-experiments-registries/mission-feature-pipeline-and-tracker/s07-07-checkpoint"}]}]},{"slug":"model-serving-and-mlops","lessons":[{"slug":"fastapi-inference-shape","steps":[{"id":"model-serving-and-mlops/fastapi-inference-shape/s01-01-intro"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/fastapi-inference-shape/s07-07-checkpoint"}]},{"slug":"batch-vs-realtime-serving","steps":[{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s01-01-intro"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/batch-vs-realtime-serving/s07-07-checkpoint"}]},{"slug":"docker-runtime-and-config","steps":[{"id":"model-serving-and-mlops/docker-runtime-and-config/s01-01-intro"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/docker-runtime-and-config/s07-07-checkpoint"}]},{"slug":"github-actions-validation-gates","steps":[{"id":"model-serving-and-mlops/github-actions-validation-gates/s01-01-intro"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s06-06-write-failing-checks"},{"id":"model-serving-and-mlops/github-actions-validation-gates/s07-07-checkpoint"}]},{"slug":"continuous-training-and-rollout","steps":[{"id":"model-serving-and-mlops/continuous-training-and-rollout/s01-01-intro"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/continuous-training-and-rollout/s07-07-checkpoint"}]},{"slug":"mission-fastapi-model-server","steps":[{"id":"model-serving-and-mlops/mission-fastapi-model-server/s01-01-intro"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s02-02-choose-the-risk"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s03-03-predict-the-checks"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s04-04-fill-the-return"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s05-05-fix-the-overtrust"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s06-06-write-count-ready"},{"id":"model-serving-and-mlops/mission-fastapi-model-server/s07-07-checkpoint"}]}]},{"slug":"monitoring-cloud-portfolio","lessons":[{"slug":"structured-logs-and-alerts","steps":[{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s01-01-intro"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/structured-logs-and-alerts/s07-07-checkpoint"}]},{"slug":"data-and-concept-drift","steps":[{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s01-01-intro"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/data-and-concept-drift/s07-07-checkpoint"}]},{"slug":"retraining-triggers","steps":[{"id":"monitoring-cloud-portfolio/retraining-triggers/s01-01-intro"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/retraining-triggers/s07-07-checkpoint"}]},{"slug":"cloud-gpu-kubernetes-cost","steps":[{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s01-01-intro"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/cloud-gpu-kubernetes-cost/s07-07-checkpoint"}]},{"slug":"portfolio-architecture-docs","steps":[{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s01-01-intro"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s06-06-write-count-ready"},{"id":"monitoring-cloud-portfolio/portfolio-architecture-docs/s07-07-checkpoint"}]},{"slug":"mission-final-portfolio-ml-system","steps":[{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s01-01-intro"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s02-02-choose-the-risk"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s03-03-predict-the-checks"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s04-04-fill-the-return"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s05-05-fix-the-overtrust"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s06-06-write-decide"},{"id":"monitoring-cloud-portfolio/mission-final-portfolio-ml-system/s07-07-checkpoint"}]}]},{"slug":"graphic-design-studio","lessons":[{"slug":"prompting-like-an-art-director","steps":[{"id":"graphic-design-studio/prompting-like-an-art-director/s01-01-the-prompt-is-a-creative-brief"},{"id":"graphic-design-studio/prompting-like-an-art-director/s02-02-style-language"},{"id":"graphic-design-studio/prompting-like-an-art-director/s03-03-which-tool-for-the-headline"},{"id":"graphic-design-studio/prompting-like-an-art-director/s04-04-predict-the-scaffold"},{"id":"graphic-design-studio/prompting-like-an-art-director/s05-05-negative-space-and-the-iteration-loop"},{"id":"graphic-design-studio/prompting-like-an-art-director/s06-06-fix-the-mood-word-prompt"},{"id":"graphic-design-studio/prompting-like-an-art-director/s07-07-write-prompt-linter"},{"id":"graphic-design-studio/prompting-like-an-art-director/s08-08-checkpoint"}]},{"slug":"the-design-kit","steps":[{"id":"graphic-design-studio/the-design-kit/s01-01-the-brand-bible-as-data"},{"id":"graphic-design-studio/the-design-kit/s02-02-fill-the-palette-line"},{"id":"graphic-design-studio/the-design-kit/s03-03-banned-looks"},{"id":"graphic-design-studio/the-design-kit/s04-04-predict-the-negatives"},{"id":"graphic-design-studio/the-design-kit/s05-05-which-field-failed"},{"id":"graphic-design-studio/the-design-kit/s06-06-fix-the-crossed-kit"},{"id":"graphic-design-studio/the-design-kit/s07-07-write-contrast-check"},{"id":"graphic-design-studio/the-design-kit/s08-08-tokens-for-the-interface"},{"id":"graphic-design-studio/the-design-kit/s09-09-write-token-audit"},{"id":"graphic-design-studio/the-design-kit/s10-10-checkpoint"}]},{"slug":"fifty-assets-one-brand","steps":[{"id":"graphic-design-studio/fifty-assets-one-brand/s01-01-the-consistency-problem"},{"id":"graphic-design-studio/fifty-assets-one-brand/s02-02-seeds-the-same-dice-twice"},{"id":"graphic-design-studio/fifty-assets-one-brand/s03-03-why-does-the-mascot-drift"},{"id":"graphic-design-studio/fifty-assets-one-brand/s04-04-predict-the-batch-plan"},{"id":"graphic-design-studio/fifty-assets-one-brand/s05-05-the-qa-filter"},{"id":"graphic-design-studio/fifty-assets-one-brand/s06-06-defend-the-pick"},{"id":"graphic-design-studio/fifty-assets-one-brand/s07-07-fix-the-qa-threshold"},{"id":"graphic-design-studio/fifty-assets-one-brand/s08-08-write-palette-distance"},{"id":"graphic-design-studio/fifty-assets-one-brand/s09-09-checkpoint"}]}]},{"slug":"campaign-studio","lessons":[{"slug":"voice-pack-as-data","steps":[{"id":"campaign-studio/voice-pack-as-data/s01-01-the-campaign-and-the-gate"},{"id":"campaign-studio/voice-pack-as-data/s02-02-lint-the-draft"},{"id":"campaign-studio/voice-pack-as-data/s03-03-predict-the-violations"},{"id":"campaign-studio/voice-pack-as-data/s04-04-testable-or-mood"},{"id":"campaign-studio/voice-pack-as-data/s05-05-fill-the-linter"},{"id":"campaign-studio/voice-pack-as-data/s06-06-variants-as-records"},{"id":"campaign-studio/voice-pack-as-data/s07-07-write-lint-copy"},{"id":"campaign-studio/voice-pack-as-data/s08-08-checkpoint"}]},{"slug":"batch-plan-and-calendar","steps":[{"id":"campaign-studio/batch-plan-and-calendar/s01-01-slots-before-prompts"},{"id":"campaign-studio/batch-plan-and-calendar/s02-02-which-field-is-missing"},{"id":"campaign-studio/batch-plan-and-calendar/s03-03-write-batch-plan"},{"id":"campaign-studio/batch-plan-and-calendar/s04-04-the-calendar-is-data"},{"id":"campaign-studio/batch-plan-and-calendar/s05-05-predict-the-queue"},{"id":"campaign-studio/batch-plan-and-calendar/s06-06-fix-the-gate"},{"id":"campaign-studio/batch-plan-and-calendar/s07-07-checkpoint"}]},{"slug":"the-numbers-pass","steps":[{"id":"campaign-studio/the-numbers-pass/s01-01-a-report-is-a-draft-over-data"},{"id":"campaign-studio/the-numbers-pass/s02-02-predict-the-verdict"},{"id":"campaign-studio/the-numbers-pass/s03-03-write-verify-numbers"},{"id":"campaign-studio/the-numbers-pass/s04-04-denominators-and-the-ftc"},{"id":"campaign-studio/the-numbers-pass/s05-05-what-may-not-ship"},{"id":"campaign-studio/the-numbers-pass/s06-06-fill-the-policy-router"},{"id":"campaign-studio/the-numbers-pass/s07-07-checkpoint"}]}]},{"slug":"support-studio","lessons":[{"slug":"the-policy-pack","steps":[{"id":"support-studio/the-policy-pack/s01-01-the-reply-that-binds-the-company"},{"id":"support-studio/the-policy-pack/s02-02-the-pack-as-data"},{"id":"support-studio/the-policy-pack/s03-03-spot-the-banned-promise"},{"id":"support-studio/the-policy-pack/s04-04-predict-the-lint"},{"id":"support-studio/the-policy-pack/s05-05-fix-the-stale-citation"},{"id":"support-studio/the-policy-pack/s06-06-the-review-loop-evidence"},{"id":"support-studio/the-policy-pack/s07-07-write-lint-draft"},{"id":"support-studio/the-policy-pack/s08-08-checkpoint"}]},{"slug":"golden-evals-earn-autonomy","steps":[{"id":"support-studio/golden-evals-earn-autonomy/s01-01-the-klarna-arc"},{"id":"support-studio/golden-evals-earn-autonomy/s02-02-a-macro-is-three-things"},{"id":"support-studio/golden-evals-earn-autonomy/s03-03-inside-the-lane"},{"id":"support-studio/golden-evals-earn-autonomy/s04-04-predict-the-eval-run"},{"id":"support-studio/golden-evals-earn-autonomy/s05-05-fill-the-promotion-rule"},{"id":"support-studio/golden-evals-earn-autonomy/s06-06-write-run-golden-evals"},{"id":"support-studio/golden-evals-earn-autonomy/s07-07-checkpoint"}]},{"slug":"the-escalation-boundary","steps":[{"id":"support-studio/the-escalation-boundary/s01-01-drafting-is-not-spending"},{"id":"support-studio/the-escalation-boundary/s02-02-triggers-as-conditions"},{"id":"support-studio/the-escalation-boundary/s03-03-route-this-ticket"},{"id":"support-studio/the-escalation-boundary/s04-04-predict-the-routing"},{"id":"support-studio/the-escalation-boundary/s05-05-fix-the-refund-hole"},{"id":"support-studio/the-escalation-boundary/s06-06-disclosure-and-the-kill-switch"},{"id":"support-studio/the-escalation-boundary/s07-07-write-route-ticket"},{"id":"support-studio/the-escalation-boundary/s08-08-checkpoint"}]}]},{"slug":"copy-studio","lessons":[{"slug":"the-voice-fingerprint","steps":[{"id":"copy-studio/the-voice-fingerprint/s01-01-the-client-you-inherit"},{"id":"copy-studio/the-voice-fingerprint/s02-02-the-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s03-03-predict-the-ceiling"},{"id":"copy-studio/the-voice-fingerprint/s04-04-the-samples-say-otherwise"},{"id":"copy-studio/the-voice-fingerprint/s05-05-fill-derive-the-rules"},{"id":"copy-studio/the-voice-fingerprint/s06-06-the-pack-with-provenance"},{"id":"copy-studio/the-voice-fingerprint/s07-07-write-fingerprint"},{"id":"copy-studio/the-voice-fingerprint/s08-08-checkpoint"}]},{"slug":"variants-that-hold-the-line","steps":[{"id":"copy-studio/variants-that-hold-the-line/s01-01-twelve-by-thursday"},{"id":"copy-studio/variants-that-hold-the-line/s02-02-the-drift-problem"},{"id":"copy-studio/variants-that-hold-the-line/s03-03-predict-the-drift-report"},{"id":"copy-studio/variants-that-hold-the-line/s04-04-which-subject-line-ships"},{"id":"copy-studio/variants-that-hold-the-line/s05-05-fix-the-channel-check"},{"id":"copy-studio/variants-that-hold-the-line/s06-06-the-kill-log"},{"id":"copy-studio/variants-that-hold-the-line/s07-07-write-voice-gate"},{"id":"copy-studio/variants-that-hold-the-line/s08-08-checkpoint"}]},{"slug":"the-claim-check","steps":[{"id":"copy-studio/the-claim-check/s01-01-forty-one-of-seventy-seven"},{"id":"copy-studio/the-claim-check/s02-02-what-counts-as-checkable"},{"id":"copy-studio/the-claim-check/s03-03-predict-the-claim-list"},{"id":"copy-studio/the-claim-check/s04-04-what-closes-a-row"},{"id":"copy-studio/the-claim-check/s05-05-fill-the-ship-decision"},{"id":"copy-studio/the-claim-check/s06-06-the-corrections-log"},{"id":"copy-studio/the-claim-check/s07-07-write-ship-gate"},{"id":"copy-studio/the-claim-check/s08-08-checkpoint"}]}]},{"slug":"analytics-studio","lessons":[{"slug":"the-cleaning-log","steps":[{"id":"analytics-studio/the-cleaning-log/s01-01-where-did-those-rows-go"},{"id":"analytics-studio/the-cleaning-log/s02-02-the-log-as-data"},{"id":"analytics-studio/the-cleaning-log/s03-03-refuse-the-silent-step"},{"id":"analytics-studio/the-cleaning-log/s04-04-predict-the-receipts"},{"id":"analytics-studio/the-cleaning-log/s05-05-fill-the-null-decision"},{"id":"analytics-studio/the-cleaning-log/s06-06-fix-the-vanishing-rows"},{"id":"analytics-studio/the-cleaning-log/s07-07-write-the-null-ledger"},{"id":"analytics-studio/the-cleaning-log/s08-08-checkpoint"}]},{"slug":"every-number-has-an-address","steps":[{"id":"analytics-studio/every-number-has-an-address/s01-01-the-plausible-wrong-number"},{"id":"analytics-studio/every-number-has-an-address/s02-02-sample-first-then-full"},{"id":"analytics-studio/every-number-has-an-address/s03-03-which-check-catches-it"},{"id":"analytics-studio/every-number-has-an-address/s04-04-predict-the-fan-out"},{"id":"analytics-studio/every-number-has-an-address/s05-05-fill-the-trace"},{"id":"analytics-studio/every-number-has-an-address/s06-06-write-audit-the-memo"},{"id":"analytics-studio/every-number-has-an-address/s07-07-checkpoint"}]},{"slug":"challenge-then-rerun","steps":[{"id":"analytics-studio/challenge-then-rerun/s01-01-the-confident-recommendation"},{"id":"analytics-studio/challenge-then-rerun/s02-02-the-triad-as-a-checklist"},{"id":"analytics-studio/challenge-then-rerun/s03-03-name-the-leg"},{"id":"analytics-studio/challenge-then-rerun/s04-04-predict-the-rerun"},{"id":"analytics-studio/challenge-then-rerun/s05-05-fix-the-drifting-memo"},{"id":"analytics-studio/challenge-then-rerun/s06-06-write-score-the-challenge"},{"id":"analytics-studio/challenge-then-rerun/s07-07-checkpoint"}]}]},{"slug":"delivery-studio","lessons":[{"slug":"the-spec-is-the-build","steps":[{"id":"delivery-studio/the-spec-is-the-build/s01-01-the-clarifying-question-is-gone"},{"id":"delivery-studio/the-spec-is-the-build/s02-02-the-ticket-as-data"},{"id":"delivery-studio/the-spec-is-the-build/s03-03-spot-the-untestable-check"},{"id":"delivery-studio/the-spec-is-the-build/s04-04-predict-the-spec-lint"},{"id":"delivery-studio/the-spec-is-the-build/s05-05-fill-the-scope-fence"},{"id":"delivery-studio/the-spec-is-the-build/s06-06-inside-or-outside-the-frontier"},{"id":"delivery-studio/the-spec-is-the-build/s07-07-write-lint-spec"},{"id":"delivery-studio/the-spec-is-the-build/s08-08-checkpoint"}]},{"slug":"acceptance-before-the-build","steps":[{"id":"delivery-studio/acceptance-before-the-build/s01-01-the-ninety-seven-thousand-dollar-skip"},{"id":"delivery-studio/acceptance-before-the-build/s02-02-evidence-against-the-checks"},{"id":"delivery-studio/acceptance-before-the-build/s03-03-which-check-catches-it"},{"id":"delivery-studio/acceptance-before-the-build/s04-04-fix-the-vibes-review"},{"id":"delivery-studio/acceptance-before-the-build/s05-05-predict-the-review-run"},{"id":"delivery-studio/acceptance-before-the-build/s06-06-rework-is-a-list-not-a-feeling"},{"id":"delivery-studio/acceptance-before-the-build/s07-07-write-review-deliverable"},{"id":"delivery-studio/acceptance-before-the-build/s08-08-checkpoint"}]},{"slug":"status-from-artifacts","steps":[{"id":"delivery-studio/status-from-artifacts/s01-01-watermelon-status"},{"id":"delivery-studio/status-from-artifacts/s02-02-derive-the-rag"},{"id":"delivery-studio/status-from-artifacts/s03-03-spot-the-artifact"},{"id":"delivery-studio/status-from-artifacts/s04-04-fill-the-worst-of"},{"id":"delivery-studio/status-from-artifacts/s05-05-predict-the-rollup"},{"id":"delivery-studio/status-from-artifacts/s06-06-the-not-ready-call"},{"id":"delivery-studio/status-from-artifacts/s07-07-write-readiness-gate"},{"id":"delivery-studio/status-from-artifacts/s08-08-checkpoint"}]}]},{"slug":"people-studio","lessons":[{"slug":"the-data-firewall","steps":[{"id":"people-studio/the-data-firewall/s01-01-the-rule-that-cannot-live-in-memory"},{"id":"people-studio/the-data-firewall/s02-02-the-firewall-as-data"},{"id":"people-studio/the-data-firewall/s03-03-which-payload-passes"},{"id":"people-studio/the-data-firewall/s04-04-predict-the-scan"},{"id":"people-studio/the-data-firewall/s05-05-fix-the-leaky-gate"},{"id":"people-studio/the-data-firewall/s06-06-employee-relations-takes-nothing"},{"id":"people-studio/the-data-firewall/s07-07-write-prompt-gate"},{"id":"people-studio/the-data-firewall/s08-08-checkpoint"}]},{"slug":"briefs-with-receipts","steps":[{"id":"people-studio/briefs-with-receipts/s01-01-the-most-automated-desk"},{"id":"people-studio/briefs-with-receipts/s02-02-a-claim-with-a-receipt"},{"id":"people-studio/briefs-with-receipts/s03-03-sourced-or-inference"},{"id":"people-studio/briefs-with-receipts/s04-04-predict-the-receipt-audit"},{"id":"people-studio/briefs-with-receipts/s05-05-fill-the-lookup"},{"id":"people-studio/briefs-with-receipts/s06-06-the-screening-line"},{"id":"people-studio/briefs-with-receipts/s07-07-write-audit-brief"},{"id":"people-studio/briefs-with-receipts/s08-08-checkpoint"}]},{"slug":"the-paper-trail","steps":[{"id":"people-studio/the-paper-trail/s01-01-policy-is-legal-surface"},{"id":"people-studio/the-paper-trail/s02-02-the-boundary-as-a-state-machine"},{"id":"people-studio/the-paper-trail/s03-03-who-signs-off"},{"id":"people-studio/the-paper-trail/s04-04-predict-the-version-log"},{"id":"people-studio/the-paper-trail/s05-05-fix-the-shortcut"},{"id":"people-studio/the-paper-trail/s06-06-the-evidence-trail"},{"id":"people-studio/the-paper-trail/s07-07-write-decision-record"},{"id":"people-studio/the-paper-trail/s08-08-checkpoint"}]}]},{"slug":"ops-studio","lessons":[{"slug":"the-sop-is-data","steps":[{"id":"ops-studio/the-sop-is-data/s01-01-the-two-weeks-notice-problem"},{"id":"ops-studio/the-sop-is-data/s02-02-the-sop-as-data"},{"id":"ops-studio/the-sop-is-data/s03-03-spot-the-invented-step"},{"id":"ops-studio/the-sop-is-data/s04-04-fill-the-sop-lint"},{"id":"ops-studio/the-sop-is-data/s05-05-predict-the-diff"},{"id":"ops-studio/the-sop-is-data/s06-06-the-diff-is-the-change-control"},{"id":"ops-studio/the-sop-is-data/s07-07-write-lint-sop"},{"id":"ops-studio/the-sop-is-data/s08-08-checkpoint"}]},{"slug":"auto-run-or-named-approval","steps":[{"id":"ops-studio/auto-run-or-named-approval/s01-01-the-checklist-that-lied"},{"id":"ops-studio/auto-run-or-named-approval/s02-02-the-schedule-is-data"},{"id":"ops-studio/auto-run-or-named-approval/s03-03-which-item-may-auto-run"},{"id":"ops-studio/auto-run-or-named-approval/s04-04-fix-the-silent-auto-complete"},{"id":"ops-studio/auto-run-or-named-approval/s05-05-predict-the-exception-queue"},{"id":"ops-studio/auto-run-or-named-approval/s06-06-exceptions-are-a-queue-not-a-shrug"},{"id":"ops-studio/auto-run-or-named-approval/s07-07-write-run-checklist"},{"id":"ops-studio/auto-run-or-named-approval/s08-08-checkpoint"}]},{"slug":"numbers-with-receipts","steps":[{"id":"ops-studio/numbers-with-receipts/s01-01-the-number-that-moves-money"},{"id":"ops-studio/numbers-with-receipts/s02-02-recompute-dont-trust"},{"id":"ops-studio/numbers-with-receipts/s03-03-which-claim-has-no-receipt"},{"id":"ops-studio/numbers-with-receipts/s04-04-predict-the-verification"},{"id":"ops-studio/numbers-with-receipts/s05-05-fill-the-receipt-check"},{"id":"ops-studio/numbers-with-receipts/s06-06-the-anomaly-is-a-question"},{"id":"ops-studio/numbers-with-receipts/s07-07-write-verify-report"},{"id":"ops-studio/numbers-with-receipts/s08-08-checkpoint"}]}]},{"slug":"legal-studio","lessons":[{"slug":"the-cite-check-gate","steps":[{"id":"legal-studio/the-cite-check-gate/s01-01-the-brief-that-reads-perfectly"},{"id":"legal-studio/the-cite-check-gate/s02-02-three-verdicts-as-data"},{"id":"legal-studio/the-cite-check-gate/s03-03-which-cite-needs-no-check"},{"id":"legal-studio/the-cite-check-gate/s04-04-predict-the-verification-run"},{"id":"legal-studio/the-cite-check-gate/s05-05-fix-the-verifier-that-asks-again"},{"id":"legal-studio/the-cite-check-gate/s06-06-write-verify-authorities"},{"id":"legal-studio/the-cite-check-gate/s07-07-checkpoint"}]},{"slug":"clause-extraction-with-pinpoints","steps":[{"id":"legal-studio/clause-extraction-with-pinpoints/s01-01-a-claim-without-a-pinpoint-is-a-rumor"},{"id":"legal-studio/clause-extraction-with-pinpoints/s02-02-the-playbook-as-data"},{"id":"legal-studio/clause-extraction-with-pinpoints/s03-03-which-row-is-unusable"},{"id":"legal-studio/clause-extraction-with-pinpoints/s04-04-predict-the-triage"},{"id":"legal-studio/clause-extraction-with-pinpoints/s05-05-fill-the-walk-away-route"},{"id":"legal-studio/clause-extraction-with-pinpoints/s06-06-write-triage-extraction"},{"id":"legal-studio/clause-extraction-with-pinpoints/s07-07-checkpoint"}]},{"slug":"privilege-rules-and-the-review-log","steps":[{"id":"legal-studio/privilege-rules-and-the-review-log/s01-01-privilege-is-a-routing-question"},{"id":"legal-studio/privilege-rules-and-the-review-log/s02-02-the-routing-table"},{"id":"legal-studio/privilege-rules-and-the-review-log/s03-03-which-request-is-the-waiver-risk"},{"id":"legal-studio/privilege-rules-and-the-review-log/s04-04-predict-the-router"},{"id":"legal-studio/privilege-rules-and-the-review-log/s05-05-fix-the-privilege-hole"},{"id":"legal-studio/privilege-rules-and-the-review-log/s06-06-the-log-that-survives-a-motion"},{"id":"legal-studio/privilege-rules-and-the-review-log/s07-07-write-log-ready"},{"id":"legal-studio/privilege-rules-and-the-review-log/s08-08-checkpoint"}]}]},{"slug":"agent-studio","lessons":[{"slug":"the-tool-contract","steps":[{"id":"agent-studio/the-tool-contract/s01-01-the-feature-you-ship-today"},{"id":"agent-studio/the-tool-contract/s02-02-the-schema-is-the-blast-radius"},{"id":"agent-studio/the-tool-contract/s03-03-spot-the-loose-tool"},{"id":"agent-studio/the-tool-contract/s04-04-predict-the-validation"},{"id":"agent-studio/the-tool-contract/s05-05-fix-the-yolo-rerun"},{"id":"agent-studio/the-tool-contract/s06-06-write-validate-call"},{"id":"agent-studio/the-tool-contract/s07-07-checkpoint"}]},{"slug":"the-loop-with-a-meter","steps":[{"id":"agent-studio/the-loop-with-a-meter/s01-01-a-loop-that-cannot-stop-is-an-invoice"},{"id":"agent-studio/the-loop-with-a-meter/s02-02-caps-are-part-of-the-loop"},{"id":"agent-studio/the-loop-with-a-meter/s03-03-why-did-it-stop"},{"id":"agent-studio/the-loop-with-a-meter/s04-04-predict-the-stop-reason"},{"id":"agent-studio/the-loop-with-a-meter/s05-05-fill-the-budget-check"},{"id":"agent-studio/the-loop-with-a-meter/s06-06-the-trace-is-the-receipt"},{"id":"agent-studio/the-loop-with-a-meter/s07-07-write-run-loop"},{"id":"agent-studio/the-loop-with-a-meter/s08-08-checkpoint"}]},{"slug":"evals-before-it-works","steps":[{"id":"agent-studio/evals-before-it-works/s01-01-green-before-it-runs"},{"id":"agent-studio/evals-before-it-works/s02-02-a-golden-case-is-a-past-incident"},{"id":"agent-studio/evals-before-it-works/s03-03-which-case-catches-it"},{"id":"agent-studio/evals-before-it-works/s04-04-predict-the-eval-run"},{"id":"agent-studio/evals-before-it-works/s05-05-fix-the-router-to-green"},{"id":"agent-studio/evals-before-it-works/s06-06-the-ratchet-entry"},{"id":"agent-studio/evals-before-it-works/s07-07-write-lint-the-ratchet"},{"id":"agent-studio/evals-before-it-works/s08-08-checkpoint"}]}]}]},"detail":{"number":25,"slug":"capstone","title":"capstone — ship the system","blurb":"wire it all together. the prompt, the call, the validation, the trace, the eval, the MCP tool. less a tutorial demo, more the smallest end-to-end llm feature you could ship to a real user. (retrieval and prompt-cache cost work live in chapters 22-23 — extend the capstone with them when you scale past the demo input set.)","overview":"$20","status":"live","lessons":[{"slug":"pick-a-project-that-ships","title":"Why most beginner agents die in production — and how to pick one that ships","estMinutes":18,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"capstone/pick-a-project-that-ships/s01-01-the-builders-graveyard","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":220,"concept":"capstone-project-failure-modes","body":"$21","cta":"Got it","runnable":true},{"type":"mc","id":"capstone/pick-a-project-that-ships/s02-02-which-project-survives-contact-with-reality","xp":2,"hint":[{"level":1,"body":"Look for the four wedge signals from step 01: volume (>10/day), structured exit condition (approve/comment/block, not vibes), explicit rules (a style guide is rules an agent can read), single user/team (4-person repo, not 'everyone').","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"project-with-a-real-wedge","prompt":"Four capstone project pitches. Only ONE has a real wedge —\nhigh volume, structured outcome, explicit rules, observable\nsuccess. Which one ships?\n","options":[{"id":"a","label":"A — Twitter summarizer","explain":"No wedge. Volume is too low to matter (10 min/day of reading anyway), success is 'feels good' (no eval possible), rules are pure taste. Classic graveyard project — fine on day one, dead on day five."},{"id":"b","label":"B — PR review agent against a fixed style guide"},{"id":"c","label":"C — AI life coach","explain":"Worst possible profile. Subjective success criterion ('helpful reflection'), no structured outcome, no rules an agent can ground in. Will stall at 'kind of works' forever."},{"id":"d","label":"D — deep-research assistant","explain":"The trap project. Feels like it works on three test queries, falls apart on the long tail (hallucinated citations, ambiguous queries, paywalled sources). Eval question 'is this research good?' is unanswerable. Ships a demo, never a product."}],"answerIds":["b"],"shuffle":false,"code":"# four capstone pitches. one has a real wedge. three do not.\npitches = {\n \"A\": \"An agent that summarizes my Twitter feed every morning into a 5-bullet briefing.\",\n \"B\": \"An agent that reads every PR opened in our 4-person team's repo and posts a 1-paragraph review against a fixed style guide — runs ~30 times/day, exits with approve/comment/block.\",\n \"C\": \"An AI life coach that helps me reflect on my day and suggests improvements.\",\n \"D\": \"A research assistant that does deep dives on any topic I ask about and writes a report with citations.\",\n}\nfor k, v in pitches.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"capstone/pick-a-project-that-ships/s03-03-the-three-shapes-that-ship","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":220,"concept":"project-shapes-that-ship","body":"$22","cta":"Got it","runnable":true},{"type":"read","id":"capstone/pick-a-project-that-ships/s04-04-the-wedge-checklist","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":200,"concept":"wedge-checklist","body":"$23","cta":"Got it","runnable":true},{"type":"mc","id":"capstone/pick-a-project-that-ships/s05-05-spot-the-missing-wedge","xp":2,"hint":[{"level":1,"body":"C is missing every signal: volume undefined (how often do you write a draft? weekly?), output is 'a helpful response' (no schema), no rubric (what makes feedback good?), user is one person with no accountability, success is 'I grow as a person' (unevaluable). It's not a project — it's a wish.","cost":0}],"personalize":false,"phase":"build","estSeconds":60,"concept":"spot-the-missing-wedge","prompt":"Run the wedge checklist on each brief. Three pass. One fails on\nalmost every signal. Which brief has NO wedge?\n","options":[{"id":"a","label":"A — support email triage","explain":"Passes all five. Volume (80/day), structured output ({category, priority}), explicit rules (priority.md), single team (6-person CS), observable success (≥85% match on held-out set). Textbook capstone."},{"id":"b","label":"B — resume scoring against rubric","explain":"Passes all five. Volume (40/day), structured output ({score, top_match}), explicit rules (rubric.md), single team (3-person recruiting), observable success (within 2 on held-out set). Ships."},{"id":"c","label":"C — writing-feedback life coach"},{"id":"d","label":"D — auto-tag bug tracker tickets","explain":"Passes all five. Volume (50/day), structured output (a tag from 12 options), explicit rules (taxonomy.md), single team (whoever owns the bug tracker), observable success (≤10% disagreement). Ships in a weekend."}],"answerIds":["c"],"shuffle":false,"code":"# four project briefs. three pass the wedge checklist. one does not.\nbriefs = {\n \"A\": \"Triage every inbound support email for a 6-person CS team. Output {category, priority} from a fixed list. Rules: priority.md. Volume: ~80/day. Eval: matches human ≥85% on a held-out 100-email set.\",\n \"B\": \"Score every applicant resume for a 3-person recruiting team against a 10-point rubric in rubric.md. Output {score: int 0-10, top_match: str}. Volume: ~40/day. Eval: matches recruiter score within 2 on held-out set.\",\n \"C\": \"Help me become a better writer by analyzing my drafts and giving me thoughtful feedback that makes me grow as a person. Output: a helpful response.\",\n \"D\": \"Auto-tag every new ticket in our internal bug tracker against a fixed taxonomy of 12 tags. Volume: ~50/day. Rules: taxonomy.md with examples. Eval: human disagrees ≤10% on a held-out set.\",\n}\nfor k, v in briefs.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"write","id":"capstone/pick-a-project-that-ships/s06-06-scope-down-the-capstone","xp":5,"hint":[{"level":1,"body":"Five conditionals, each adds 20 points. Use `.get()` with defaults so missing fields don't crash. Then four-way threshold for the verdict. Watch the users_count check — it has to be in the range [1, 10], not just >= 1.","cost":0},{"level":2,"body":"PR review agent passes all five signals: volume_per_day=30 (>=10), schema=True, rubric=True, users_count=4 (between 1 and 10), eval=True. Total 100. Life coach fails all five: volume=1, schema=False, rubric=False, users_count=100000 (way over 10 — 'everyone' isn't a user), eval=False. Total 0.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"score-a-capstone-pitch","prompt":"Write `score_project(spec)` that takes a project spec (dict) and\nreturns a dict with two fields:\n\n- `score`: integer 0-100, higher is better\n- `verdict`: string, one of:\n - `\"ship it\"` if score >= 80\n - `\"narrow the scope\"` if score >= 60\n - `\"pick something else\"` if score >= 40\n - `\"don't build this\"` if score < 40\n\nScore the spec on the five wedge signals from step 04. Each\nsignal that PASSES adds 20 points:\n\n- `volume_per_day >= 10`: +20 (volume signal)\n- `output_schema_defined`: +20 (structured outcome signal)\n- `has_rubric`: +20 (explicit rules signal)\n- `users_count >= 1 and users_count <= 10`: +20 (single user/team signal — exactly one team, not \"everyone\")\n- `eval_method_defined`: +20 (observable success signal)\n\nTwo projects run. Expected output:\n```\nPR review agent: {'score': 100, 'verdict': 'ship it'}\nAI life coach: {'score': 0, 'verdict': \"don't build this\"}\n```\n","starter":"pr_agent = {\n \"name\": \"PR review agent\",\n \"volume_per_day\": 30,\n \"output_schema_defined\": True,\n \"has_rubric\": True,\n \"users_count\": 4,\n \"eval_method_defined\": True,\n}\n\nlife_coach = {\n \"name\": \"AI life coach\",\n \"volume_per_day\": 1,\n \"output_schema_defined\": False,\n \"has_rubric\": False,\n \"users_count\": 100000,\n \"eval_method_defined\": False,\n}\n\n# define score_project(spec) below\n\nprint(f\"PR review agent: {score_project(pr_agent)}\")\nprint(f\"AI life coach: {score_project(life_coach)}\")\n","grader":{"kind":"stdout-equality","expected":"PR review agent: {'score': 100, 'verdict': 'ship it'}\nAI life coach: {'score': 0, 'verdict': \"don't build this\"}\n","normalize":"collapse-trailing-newline"},"solution":"$24","hiddenTests":[]},{"type":"checkpoint","id":"capstone/pick-a-project-that-ships/s07-07-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":240,"concept":"pick-the-shipping-capstone","prompt":"Final drill. Synthesize the wedge checklist into one function:\n`pick_capstone(candidates)` that takes a list of project pitches\nand returns the NAME of the highest-scoring candidate.\n\nEach candidate is a dict with:\n- `name`: string\n- `volume_per_day`: int\n- `output_schema_defined`: bool\n- `has_rubric`: bool\n- `users_count`: int\n- `eval_method_defined`: bool\n\nScore each candidate using the same five rules from step 06\n(each passing signal adds 20):\n\n- `volume_per_day >= 10`: +20\n- `output_schema_defined`: +20\n- `has_rubric`: +20\n- `1 <= users_count <= 10`: +20\n- `eval_method_defined`: +20\n\nReturn the name of the highest-scoring candidate. On a tie,\nreturn the FIRST one in the list (Python's `max` with `key=`\npreserves stable order).\n\nFive candidates run. Expected output:\n```\nship: Support email triage\n```\n","starter":"candidates = [\n {\"name\": \"Twitter summarizer\", \"volume_per_day\": 1, \"output_schema_defined\": False, \"has_rubric\": False, \"users_count\": 1, \"eval_method_defined\": False},\n {\"name\": \"Support email triage\", \"volume_per_day\": 80, \"output_schema_defined\": True, \"has_rubric\": True, \"users_count\": 6, \"eval_method_defined\": True},\n {\"name\": \"Personal finance agent\", \"volume_per_day\": 5, \"output_schema_defined\": True, \"has_rubric\": False, \"users_count\": 1, \"eval_method_defined\": False},\n {\"name\": \"Research assistant\", \"volume_per_day\": 3, \"output_schema_defined\": False, \"has_rubric\": False, \"users_count\": 100000, \"eval_method_defined\": False},\n {\"name\": \"Bug-tracker auto-tagger\", \"volume_per_day\": 50, \"output_schema_defined\": True, \"has_rubric\": True, \"users_count\": 5, \"eval_method_defined\": False},\n]\n\n# define pick_capstone(candidates) below\n\nprint(f\"ship: {pick_capstone(candidates)}\")\n","grader":{"kind":"stdout-equality","expected":"ship: Support email triage","normalize":"collapse-trailing-newline"},"solution":"$25","passThreshold":0.66}],"xpTotal":20},{"slug":"build-a-cli-agent","title":"Wire it all together — a CLI agent in 12 steps","estMinutes":14,"prerequisites":[],"status":"live","steps":[{"type":"read","id":"capstone/build-a-cli-agent/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":120,"concept":"agent-architecture","body":"$26","cta":"Got it","code":"# the agent we're building, in pseudocode-as-Python comments.\nAGENT = \"\"\"\n1. read the user's prompt\n2. send it to the LLM with the tool definitions\n3. if the LLM returns text → print and exit\n4. if the LLM returns a tool_use → run the tool, append result, goto 2\n5. log every turn to agent.jsonl\n\"\"\"\nprint(AGENT)\n","runnable":true},{"type":"mc","id":"capstone/build-a-cli-agent/s02-02-which-piece-is-missing","xp":2,"hint":[{"level":1,"body":"Scan the dict for the only `False`.","cost":0},{"level":2,"body":"It's `stop_condition`. An agent without one will spin or terminate prematurely. Usually checks `stop_reason == \"end_turn\"` or a max-turns guard.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"agent-architecture-pieces","prompt":"Cursor scaffolded an agent and asked you to review it. Three of the\nfour core pieces are in place; one is missing. Without that piece,\nthe agent runs forever or stops at the wrong moment. Which piece is\nmissing?\n","options":[{"id":"a","label":"The LLM call","explain":"It's listed as `True` in the dict — already in place."},{"id":"b","label":"The tool registry","explain":"Also `True`. The agent knows which tools exist."},{"id":"c","label":"The loop","explain":"Also `True`. The skeleton has a `while`."},{"id":"d","label":"The stop condition"}],"answerIds":["d"],"shuffle":false,"code":"# the four core ingredients of a working agent.\ningredients = {\n \"llm_call\": True,\n \"tool_registry\": True,\n \"loop\": True,\n \"stop_condition\": False, # missing\n}\n\nfor k, v in ingredients.items():\n print(f\"{k}: {'ok' if v else 'missing'}\")\n","runnable":true},{"type":"read","id":"capstone/build-a-cli-agent/s03-03-the-tool-loop","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":130,"concept":"agent-loop-mechanics","body":"$27","cta":"Got it","code":"$28","runnable":true},{"type":"predict","id":"capstone/build-a-cli-agent/s04-04-predict-the-final-answer","xp":3,"hint":[{"level":1,"body":"Trace the two turns. Turn 1: tool_use, run tool, append. Turn 2: end_turn, print the text.","cost":0},{"level":2,"body":"On turn 2 the loop hits the `end_turn` branch and prints `r[\"content\"][0][\"text\"]`, which is `boston: 52F and clear`.","cost":0}],"personalize":false,"phase":"build","estSeconds":80,"concept":"agent-loop-output","code":"def fake_llm(messages):\n if len(messages) == 1:\n return {\n \"stop_reason\": \"tool_use\",\n \"content\": [{\n \"type\": \"tool_use\", \"id\": \"toolu_1\",\n \"name\": \"get_weather\",\n \"input\": {\"city\": \"boston\"},\n }],\n }\n return {\n \"stop_reason\": \"end_turn\",\n \"content\": [{\"type\": \"text\", \"text\": \"boston: 52F and clear\"}],\n }\n\ndef get_weather(city):\n return f\"{city}: 52F and clear\"\n\nmessages = [{\"role\": \"user\", \"content\": \"weather in boston?\"}]\n\nfor _ in range(3):\n r = fake_llm(messages)\n if r[\"stop_reason\"] == \"end_turn\":\n print(r[\"content\"][0][\"text\"])\n break\n blk = r[\"content\"][0]\n result = get_weather(blk[\"input\"][\"city\"])\n messages.append({\"role\": \"assistant\", \"content\": r[\"content\"]})\n messages.append({\"role\": \"user\", \"content\": [\n {\"type\": \"tool_result\", \"tool_use_id\": blk[\"id\"], \"content\": result},\n ]})\n","prompt":"Read the agent loop on the right. The fake LLM returns a `tool_use` on\nthe first turn and an `end_turn` on the second. What gets printed?\n","grader":{"kind":"stdout-equality","expected":"boston: 52F and clear","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/build-a-cli-agent/s05-05-fill-the-stop-check","xp":3,"hint":[{"level":1,"body":"Two stop reasons matter inside the loop: one for tool calls, one for the final answer. You want the one that means 'finished.'","cost":0},{"level":2,"body":"It's `end_turn`. The check becomes `response[\"stop_reason\"] == \"end_turn\"`.","cost":0}],"personalize":false,"phase":"build","estSeconds":60,"concept":"end-turn-stop-reason","prompt":"Cursor wrote a loop body but the stop-condition check is missing the\nstring it should compare against. Fill in the value — the stop\nreason that means \"the LLM is done; show the answer.\"\n\nStop reason value (string, no quotes — just the word): ___\n","code":"def fake_llm():\n return {\"stop_reason\": \"end_turn\",\n \"content\": [{\"type\": \"text\", \"text\": \"done\"}]}\n\ndef main():\n response = fake_llm()\n if response[\"stop_reason\"] == \"___\":\n print(response[\"content\"][0][\"text\"])\n\nmain()\n","blanks":[{"id":"stop","accept":["end_turn"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"capstone/build-a-cli-agent/s06-06-fix-the-tool-dispatch","xp":4,"hint":[{"level":1,"body":"Check whether `name` is in the registry before calling it. Return `\"unknown tool\"` when it isn't.","cost":0},{"level":2,"body":"Replace the body with `if name not in registry: return \"unknown tool\"` and then `return registry[name](args)`.","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"tool-registry-dispatch","brokenCode":"registry = {\n \"read_file\": lambda args: f\"contents of {args['path']}\",\n \"write_file\": lambda args: \"ok\",\n}\n\ndef dispatch(name, args):\n return registry[name](args)\n\nprint(dispatch(\"send_email\", {\"to\": \"x\"}))\n","prompt":"Cursor wrote a tool dispatcher that looks up a tool by name and calls\nit. The bug is on line 6 — when the tool isn't in the registry, the\ndispatcher crashes with a `KeyError`. Fix it so an unknown tool name\nreturns the string `\"unknown tool\"` instead.\n\nExpected output:\n```\nunknown tool\n```\n","grader":{"kind":"stdout-equality","expected":"unknown tool","normalize":"collapse-trailing-newline"},"bugLines":[7],"revealAfter":4},{"type":"fix","id":"capstone/build-a-cli-agent/s07-07-fix-the-message-append","xp":4,"hint":[{"level":1,"body":"On a tool round-trip, you append two messages: the assistant's tool_use AND the user's tool_result. The first one is missing.","cost":0},{"level":2,"body":"Before the existing append, add: `messages.append({\"role\": \"assistant\", \"content\": response[\"content\"]})`.","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"message-append-pair","brokenCode":"response = {\n \"stop_reason\": \"tool_use\",\n \"content\": [{\n \"type\": \"tool_use\", \"id\": \"toolu_1\",\n \"name\": \"ping\", \"input\": {},\n }],\n}\nblock = response[\"content\"][0]\ntool_result = \"pong\"\n\nmessages = [{\"role\": \"user\", \"content\": \"ping\"}]\n\n# missing: append the assistant's tool_use message here\nmessages.append({\"role\": \"user\", \"content\": [\n {\"type\": \"tool_result\", \"tool_use_id\": block[\"id\"], \"content\": tool_result},\n]})\n\nprint(len(messages))\n","prompt":"Claude wrote a tool round-trip that drops the assistant's tool_use\nmessage before sending the next request. The script counts how many\nmessages should be in the history after one tool call (it should be 3:\nuser prompt, assistant tool_use, user tool_result). Right now it's 2.\nAdd the missing append on line 14 so the count is exactly 3.\n\nExpected output:\n```\n3\n```\n","grader":{"kind":"stdout-equality","expected":"3","normalize":"collapse-trailing-newline"},"bugLines":[13],"revealAfter":5},{"type":"read","id":"capstone/build-a-cli-agent/s08-08-the-logging-pattern","xp":1,"hint":[],"personalize":false,"phase":"build","estSeconds":110,"concept":"jsonl-logging","body":"$29","cta":"Got it","code":"import json\n\n# collected log lines (in real life, written to agent.jsonl)\nlog_lines = []\n\ndef log_turn(turn_num, kind, payload):\n entry = {\"turn\": turn_num, \"kind\": kind, \"payload\": payload}\n log_lines.append(json.dumps(entry))\n\nlog_turn(1, \"user_prompt\", {\"text\": \"what's on my todo list?\"})\nlog_turn(1, \"tool_use\", {\"name\": \"read_file\", \"input\": {\"path\": \"todo.txt\"}})\nlog_turn(1, \"tool_result\", {\"text\": \"write tests\"})\nlog_turn(2, \"final_text\", {\"text\": \"your todos: write tests\"})\n\nfor line in log_lines:\n print(line)\n","runnable":true},{"type":"write","id":"capstone/build-a-cli-agent/s09-09-write-the-tool-runner","xp":5,"hint":[{"level":1,"body":"Two-line function. Guard for unknown name first, then call.","cost":0},{"level":2,"body":"`if name not in tools: return \"unknown tool\"` then `return tools[name](args)`.","cost":0}],"personalize":false,"phase":"build","estSeconds":150,"concept":"build-tool-registry","prompt":"Build the tool runner. The starter has a `tools` dict mapping tool\nnames to functions. Write a function `run_tool(name, args)` that:\n\n- Returns `\"unknown tool\"` if `name` is not in `tools`.\n- Otherwise calls `tools[name](args)` and returns the result.\n\nThen call `run_tool(\"read_file\", {\"path\": \"todo.txt\"})` and print\nthe result.\n\nExpected output:\n```\ncontents of todo.txt\n```\n","starter":"tools = {\n \"read_file\": lambda args: f\"contents of {args['path']}\",\n \"write_file\": lambda args: \"wrote file\",\n}\n\n# define run_tool below\n\nprint(run_tool(\"read_file\", {\"path\": \"todo.txt\"}))\n","grader":{"kind":"stdout-equality","expected":"contents of todo.txt","normalize":"collapse-trailing-newline"},"solution":"tools = {\n \"read_file\": lambda args: f\"contents of {args['path']}\",\n \"write_file\": lambda args: \"wrote file\",\n}\n\ndef run_tool(name, args):\n if name not in tools:\n return \"unknown tool\"\n return tools[name](args)\n\nprint(run_tool(\"read_file\", {\"path\": \"todo.txt\"}))\n","hiddenTests":[]},{"type":"fix","id":"capstone/build-a-cli-agent/s10-10-fix-the-loop-exit","xp":4,"hint":[{"level":1,"body":"Replace `while True` with a `while turn < 3` so the loop bails after three turns regardless of stop_reason.","cost":0},{"level":2,"body":"Change line 14 to `while turn < 3:`. The print at the bottom will report `bailed after 3 turns`.","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"max-turns-guard","brokenCode":"def fake_llm():\n # always asks for another tool — simulates a runaway model\n return {\n \"stop_reason\": \"tool_use\",\n \"content\": [{\"type\": \"tool_use\", \"id\": \"tu\",\n \"name\": \"noop\", \"input\": {}}],\n }\n\ndef run_tool(name, args):\n return \"ok\"\n\nturn = 0\nmessages = []\nwhile True:\n response = fake_llm()\n if response[\"stop_reason\"] == \"end_turn\":\n print(response[\"content\"][0][\"text\"])\n break\n block = response[\"content\"][0]\n run_tool(block[\"name\"], block[\"input\"])\n turn += 1\n\nprint(f\"bailed after {turn} turns\")\n","prompt":"Cursor wrote an agent loop without a max-turns guard. The fake LLM in\nthis snippet always returns `tool_use` (it never produces an\n`end_turn`), so the loop would run forever in real life. Convert the\n`while True` on line 14 into a bounded loop that stops after exactly\n3 turns and prints the turn count.\n\nExpected output:\n```\nbailed after 3 turns\n```\n","grader":{"kind":"stdout-equality","expected":"bailed after 3 turns","normalize":"collapse-trailing-newline"},"bugLines":[14],"revealAfter":5},{"type":"write","id":"capstone/build-a-cli-agent/s11-11-write-the-agent","xp":7,"hint":[{"level":1,"body":"The loop is `for _ in range(MAX_TURNS):`. Inside: call `fake_llm`, branch on stop_reason. On `end_turn`, print and break. On `tool_use`, run the tool and append both messages.","cost":0},{"level":2,"body":"Don't forget BOTH appends on the tool path. Assistant message is `{\"role\": \"assistant\", \"content\": response[\"content\"]}`. Tool result is wrapped in a user-role message: `{\"role\": \"user\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": block[\"id\"], \"content\": result}]}`.","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"full-agent-loop","prompt":"Build the full agent loop. The starter has:\n\n- `fake_llm(messages)` — returns hardcoded responses based on history\n length (1 message → tool_use, 3 messages → end_turn).\n- `run_tool(name, args)` — handles tool dispatch.\n- `MAX_TURNS = 5` — the loop guardrail.\n- `messages = [...]` — initial user prompt.\n\nWrite the agent loop that:\n\n1. Runs at most `MAX_TURNS` iterations.\n2. Calls `fake_llm(messages)`.\n3. If `stop_reason == \"end_turn\"`, prints the text and breaks.\n4. If `stop_reason == \"tool_use\"`, runs the tool, then appends\n **two** messages — the assistant's tool_use, and a user message\n wrapping the tool_result.\n\nExpected output:\n```\ndone. todos: write tests\n```\n","starter":"def fake_llm(messages):\n if len(messages) == 1:\n return {\n \"stop_reason\": \"tool_use\",\n \"content\": [{\n \"type\": \"tool_use\", \"id\": \"toolu_1\",\n \"name\": \"read_file\",\n \"input\": {\"path\": \"todo.txt\"},\n }],\n }\n return {\n \"stop_reason\": \"end_turn\",\n \"content\": [{\"type\": \"text\", \"text\": \"done. todos: write tests\"}],\n }\n\ndef run_tool(name, args):\n if name == \"read_file\":\n return \"write tests\"\n return \"unknown tool\"\n\nMAX_TURNS = 5\nmessages = [{\"role\": \"user\", \"content\": \"what's on my todo?\"}]\n\n# write the agent loop below\n","grader":{"kind":"stdout-equality","expected":"done. todos: write tests","normalize":"collapse-trailing-newline"},"solution":"$2a","hiddenTests":[]},{"type":"checkpoint","id":"capstone/build-a-cli-agent/s12-12-checkpoint","xp":10,"hint":[],"personalize":false,"phase":"check","estSeconds":180,"concept":"capstone-recap","prompt":"Last one. The starter has a complete agent — fake LLM, tool registry,\nthe loop, and a `log_lines` list for the trace. The loop is wired to\nexit after the LLM returns `end_turn`. Right now the script prints\nthe final answer but doesn't log a single thing — `log_lines` ends\nempty.\n\nAdd **two** `log_lines.append(...)` calls inside the loop so that:\n\n1. After every LLM response, you append a JSON string of\n `{\"turn\": turn, \"stop_reason\": response[\"stop_reason\"]}`.\n2. After every successful tool call, you append a JSON string of\n `{\"turn\": turn, \"tool\": block[\"name\"], \"result\": result}`.\n\nThen the script prints the count of log entries.\n\nExpected output:\n```\ndone. todos: write tests\nlog entries: 3\n```\n","starter":"$2b","grader":{"kind":"stdout-equality","expected":"done. todos: write tests\nlog entries: 3\n","normalize":"collapse-trailing-newline"},"solution":"$2c","passThreshold":0.66}],"xpTotal":45},{"slug":"wire-the-real-model","title":"Wire the real model — swap fake_llm for the Anthropic SDK shape","estMinutes":14,"prerequisites":["build-a-cli-agent"],"status":"live","steps":[{"type":"read","id":"capstone/wire-the-real-model/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":130,"concept":"real-sdk-swap","body":"$2d","cta":"Got it","code":"$2e","runnable":true},{"type":"mc","id":"capstone/wire-the-real-model/s02-02-where-the-key-lives","xp":2,"hint":[{"level":1,"body":"Two questions: where does the key come from (env var, not source) and what happens if it's missing (clear error, not a 5-frames-deep crash)?","cost":0}],"personalize":false,"phase":"warmup","estSeconds":50,"concept":"env-var-loading","prompt":"Five ways to load the API key. Which TWO are production-grade\npatterns you'd ship?\n","options":[{"id":"c_and_e","label":"C (os.getenv) and E (getenv with explicit error fallback)"},{"id":"b_and_c","label":"B (os.environ bracket) and C (os.getenv)","explain":"B raises a raw `KeyError` with no context when the env var is missing. Always prefer `getenv` so you can return your own error message."},{"id":"a_and_d","label":"A (hardcoded) and D (manual file parse)","explain":"A puts the key in git history forever. D reinvents `python-dotenv` with all the bugs and none of the standardization. Both are how you leak keys to PRs."},{"id":"c_only","label":"C only — E is overengineering","explain":"Falling back with a clear error message *is* production-grade. Returning `None` and crashing five frames deeper makes debugging painful."}],"answerIds":["c_and_e"],"shuffle":false,"code":"import os\n\n# five places people put the API key. only some are correct.\noptions = {\n \"A\": 'api_key = \"sk-ant-api03-xxxxx\" # right at top of agent.py',\n \"B\": 'api_key = os.environ[\"ANTHROPIC_API_KEY\"]',\n \"C\": 'api_key = os.getenv(\"ANTHROPIC_API_KEY\")',\n \"D\": 'api_key = open(\".env\").read().strip().split(\"=\")[1]',\n \"E\": 'api_key = os.getenv(\"ANTHROPIC_API_KEY\") or raise_clear_error()',\n}\nfor k, v in options.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"capstone/wire-the-real-model/s03-03-the-sdk-response-shape","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":100,"concept":"anthropic-sdk-response-shape","body":"$2f","cta":"Got it","code":"$30","runnable":true},{"type":"predict","id":"capstone/wire-the-real-model/s04-04-predict-the-block-type","xp":3,"hint":[{"level":1,"body":"Three blocks, three iterations. Each iteration the if/elif picks one branch and prints. Trace each block by .type.","cost":0},{"level":2,"body":"Block 1 is text → `TEXT: Let me look that up.`\nBlock 2 is tool_use → `TOOL: search`\nBlock 3 is text → `TEXT: Here's what I found.`\n","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"block-type-discrimination","code":"# an assistant turn with a text block AND a tool_use block.\n# the loop has to print one line per block, branching on .type.\n\nclass MockTextBlock:\n def __init__(self, text):\n self.type = \"text\"\n self.text = text\n\nclass MockToolUseBlock:\n def __init__(self, name):\n self.type = \"tool_use\"\n self.name = name\n\nblocks = [\n MockTextBlock(\"Let me look that up.\"),\n MockToolUseBlock(\"search\"),\n MockTextBlock(\"Here's what I found.\"),\n]\n\nfor block in blocks:\n if block.type == \"text\":\n print(f\"TEXT: {block.text}\")\n elif block.type == \"tool_use\":\n print(f\"TOOL: {block.name}\")\n","prompt":"Read the editor. Three blocks in order: text, tool_use, text. The\nloop branches on `.type` and prints one line per block. What does\nthe script print?\n","grader":{"kind":"stdout-equality","expected":"TEXT: Let me look that up.\nTOOL: search\nTEXT: Here's what I found.\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/wire-the-real-model/s05-05-fill-the-key-load","xp":3,"hint":[{"level":1,"body":"Use `os.getenv` (returns None on miss), not `os.environ[...]` (raises KeyError on miss). The argument is the env var name as a string.","cost":0},{"level":2,"body":"`os.getenv(\"ANTHROPIC_API_KEY\")` — getenv with the var name. The `if api_key is None` branch handles the miss case cleanly.","cost":0}],"personalize":false,"phase":"build","estSeconds":60,"concept":"getenv-with-default-error","prompt":"The agent expects `ANTHROPIC_API_KEY` from the environment. Fill in\nthe load using `os.getenv` so a missing key returns `None`, then\nthe code immediately raises a clear error instead of crashing five\nframes deep when the SDK rejects the call.\n\nIn this demo the env var IS set (mocked via `os.environ`), so the\nload succeeds and the script prints `loaded`.\n\nExpected output:\n```\nloaded\n```\n","code":"import os\nos.environ[\"ANTHROPIC_API_KEY\"] = \"sk-ant-test-key-deterministic-for-demo\"\n\napi_key = ___\nif api_key is None:\n raise RuntimeError(\"ANTHROPIC_API_KEY is not set; export it or load .env first\")\n\nprint(\"loaded\")\n","blanks":[{"id":"getenv_call","accept":["os.getenv(\"ANTHROPIC_API_KEY\")","os.getenv('ANTHROPIC_API_KEY')"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"capstone/wire-the-real-model/s06-06-fix-the-hardcoded-key","xp":4,"hint":[{"level":1,"body":"Replace the hardcoded string with `os.getenv(\"ANTHROPIC_API_KEY\")`. Same first 6 characters land in stdout, but the key never appears in source.","cost":0},{"level":2,"body":"Change line 5 to:\n\n```python\napi_key = os.getenv(\"ANTHROPIC_API_KEY\")\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"hardcoded-key-removal","brokenCode":"import os\nos.environ[\"ANTHROPIC_API_KEY\"] = \"sk-ant-prod-real-key-do-not-commit\"\n\n# bug: hardcoded key in source. ends up in git history forever.\napi_key = \"sk-ant-prod-real-key-do-not-commit\"\n\nprint(f\"key starts with: {api_key[:6]}\")\n","prompt":"Cursor wrote this for \"just one quick test.\" The key is hardcoded\nas a Python literal. If you commit and push, the key is in your\ngit history forever — even after you delete the line, even after\nyou `git push --force` (the rotation API logs prove it).\n\nFix the code to load the key from the environment via `os.getenv`.\nIn this demo the env var is set for you.\n\nExpected output:\n```\nkey starts with: sk-ant\n```\n","grader":{"kind":"stdout-equality","expected":"key starts with: sk-ant","normalize":"collapse-trailing-newline"},"bugLines":[5],"revealAfter":4},{"type":"fix","id":"capstone/wire-the-real-model/s07-07-fix-the-block-access","xp":4,"hint":[{"level":1,"body":"Branch on `block.type` before reading type-specific attributes. Text blocks have `.text`; tool_use blocks have `.name`, `.input`, `.id`.","cost":0},{"level":2,"body":"Replace the `print` line with:\n\n```python\nif block.type == \"text\":\n print(f\"TEXT: {block.text}\")\nelif block.type == \"tool_use\":\n print(f\"TOOL: {block.name}\")\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"block-type-attribute-access","brokenCode":"class MockTextBlock:\n def __init__(self, text):\n self.type = \"text\"\n self.text = text\n\nclass MockToolUseBlock:\n def __init__(self, name):\n self.type = \"tool_use\"\n self.name = name\n\nblocks = [\n MockTextBlock(\"Let me search.\"),\n MockToolUseBlock(\"search\"),\n]\n\nfor block in blocks:\n # bug: assumes every block has .text — AttributeError on tool_use.\n print(f\"TEXT: {block.text}\")\n","prompt":"The loop is reading `.text` on every block in `response.content`,\nbut one of the blocks is a `tool_use` block that has no `.text`\nattribute — it has `.name`, `.input`, `.id`. The current code\nraises `AttributeError` on that block.\n\nFix the loop to branch on `block.type` first, so text blocks emit\ntheir text and tool_use blocks emit their name.\n\nExpected output:\n```\nTEXT: Let me search.\nTOOL: search\n```\n","grader":{"kind":"stdout-equality","expected":"TEXT: Let me search.\nTOOL: search\n","normalize":"collapse-trailing-newline"},"bugLines":[18],"revealAfter":5},{"type":"write","id":"capstone/wire-the-real-model/s08-08-write-the-shape-adapter","xp":5,"hint":[{"level":1,"body":"Initialize two empty lists. Iterate `response.content`. Branch on `block.type`: append `.text` to one list for text blocks; append a dict for tool_use blocks. Return both lists as a tuple.","cost":0},{"level":2,"body":"```python\ndef extract_text_and_tools(response):\n text_parts = []\n tool_calls = []\n for block in response.content:\n if block.type == \"text\":\n text_parts.append(block.text)\n elif block.type == \"tool_use\":\n tool_calls.append({\"id\": block.id, \"name\": block.name, \"input\": block.input})\n return text_parts, tool_calls\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"sdk-shape-adapter","prompt":"Write `extract_text_and_tools(response)` that takes a response\nobject (real `anthropic.types.Message` OR a `MockMessage` with\nmatching attribute access) and returns a tuple `(text_parts, tool_calls)`:\n\n- `text_parts` is a list of strings, one per text block in\n `response.content`, in order.\n- `tool_calls` is a list of dicts, one per tool_use block, each\n shaped `{\"id\": ..., \"name\": ..., \"input\": ...}`.\n\nIterate `response.content`, branch on `block.type`, build both\nlists. Skip blocks whose type is neither `\"text\"` nor `\"tool_use\"`\n(forward-compat for `thinking` blocks — they exist but you don't\nneed them here).\n\nThe script will run two cases. Expected output:\n```\ntext=['Let me look that up.'] tools=[{'id': 'toolu_01', 'name': 'search', 'input': {'q': 'ramen'}}]\ntext=['Found a 2026 guide.', 'Hope this helps!'] tools=[]\n```\n","starter":"class MockTextBlock:\n def __init__(self, text):\n self.type = \"text\"\n self.text = text\n\nclass MockToolUseBlock:\n def __init__(self, id, name, input):\n self.type = \"tool_use\"\n self.id = id\n self.name = name\n self.input = input\n\nclass MockMessage:\n def __init__(self, stop_reason, content):\n self.stop_reason = stop_reason\n self.content = content\n\n# define extract_text_and_tools(response) below\n\ncase_1 = MockMessage(\n stop_reason=\"tool_use\",\n content=[\n MockTextBlock(\"Let me look that up.\"),\n MockToolUseBlock(id=\"toolu_01\", name=\"search\", input={\"q\": \"ramen\"}),\n ],\n)\ncase_2 = MockMessage(\n stop_reason=\"end_turn\",\n content=[\n MockTextBlock(\"Found a 2026 guide.\"),\n MockTextBlock(\"Hope this helps!\"),\n ],\n)\n\ntext, tools = extract_text_and_tools(case_1)\nprint(f\"text={text} tools={tools}\")\ntext, tools = extract_text_and_tools(case_2)\nprint(f\"text={text} tools={tools}\")\n","grader":{"kind":"stdout-equality","expected":"text=['Let me look that up.'] tools=[{'id': 'toolu_01', 'name': 'search', 'input': {'q': 'ramen'}}]\ntext=['Found a 2026 guide.', 'Hope this helps!'] tools=[]\n","normalize":"collapse-trailing-newline"},"solution":"$31","hiddenTests":[]},{"type":"checkpoint","id":"capstone/wire-the-real-model/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"real-shape-agent-loop","prompt":"$32","starter":"$33","grader":{"kind":"stdout-equality","expected":"ok=True iters=2 tokens=120 answer=Found Tokyo's best ramen, 2026 guide.","normalize":"collapse-trailing-newline"},"solution":"$34","passThreshold":0.66}],"xpTotal":32},{"slug":"validate-tool-inputs","title":"Validate tool inputs — when the model invents arguments","estMinutes":12,"prerequisites":["build-a-cli-agent","wire-the-real-model"],"status":"live","steps":[{"type":"read","id":"capstone/validate-tool-inputs/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":110,"concept":"schema-validate-before-dispatch","body":"$35","cta":"Got it","code":"$36","runnable":true},{"type":"mc","id":"capstone/validate-tool-inputs/s02-02-which-input-needs-validation","xp":2,"hint":[{"level":1,"body":"Validation catches the model inventing or mis-typing arguments. Which tools (a) take inputs from the model and (b) have constraints those inputs must satisfy?","cost":0}],"personalize":false,"phase":"warmup","estSeconds":50,"concept":"validation-fit","prompt":"Four tools. Validation costs ~1ms per call but catches the model\ninventing arguments. Which TWO benefit MOST from input validation\nbefore dispatch?\n","options":[{"id":"a_and_c","label":"A (URL must start with https://) and C (page_size 1-100)"},{"id":"b_and_d","label":"B (hardcoded config) and D (printf debugger)","explain":"B has no inputs — nothing to validate. D's input is from the user, not the model — model didn't invent it."},{"id":"a_and_b","label":"A and B","explain":"B has no inputs. Validation here is a no-op."},{"id":"c_and_d","label":"C and D","explain":"D's input came from the user, not the model. The bug class validation prevents (model inventing args) doesn't apply."}],"answerIds":["a_and_c"],"shuffle":false,"code":"scenarios = {\n \"A\": \"Tool reads a URL the model picked. URL must be a string starting with https://.\",\n \"B\": \"Tool that hardcodes a fixed config — no inputs.\",\n \"C\": \"Tool that takes a 'page_size' integer, must be 1-100.\",\n \"D\": \"Tool that takes free-text the user typed, passed straight to a printf-debugger.\",\n}\nfor k, v in scenarios.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"capstone/validate-tool-inputs/s03-03-the-validator-surface","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":90,"concept":"validator-three-checks","body":"$37","cta":"Got it","code":"$38","runnable":true},{"type":"predict","id":"capstone/validate-tool-inputs/s04-04-predict-the-error","xp":3,"hint":[{"level":1,"body":"The first check loops over `required` fields. The required field `q` is not in the args dict (only `query` is). The function returns the dict with `ok=False` and the missing-field error.","cost":0},{"level":2,"body":"`{\"ok\": False, \"error\": \"missing required field: q\"}` — Python dict repr uses single quotes by default.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"validator-error-trace","code":"def validate(args, schema):\n for field in schema.get(\"required\", []):\n if field not in args:\n return {\"ok\": False, \"error\": f\"missing required field: {field}\"}\n for field, expected_type in schema.get(\"types\", {}).items():\n if field in args and not isinstance(args[field], expected_type):\n actual = type(args[field]).__name__\n return {\"ok\": False, \"error\": f\"{field} should be {expected_type.__name__}, got {actual}\"}\n return {\"ok\": True, \"args\": args}\n\nsearch_schema = {\"required\": [\"q\"], \"types\": {\"q\": str}}\n\n# the model invented a field name. classic.\nbad_args = {\"query\": \"ramen tokyo\"}\nresult = validate(bad_args, search_schema)\nprint(result)\n","prompt":"Read the editor. The schema says `q` is required. The model passed\n`{\"query\": \"ramen tokyo\"}` instead — wrong field name. What does\nthe validator return?\n","grader":{"kind":"stdout-equality","expected":"{'ok': False, 'error': 'missing required field: q'}","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/validate-tool-inputs/s05-05-fill-the-validate-call","xp":3,"hint":[{"level":1,"body":"Call `validate` with two arguments: `args` (the dict the model sent) and `SCHEMAS[tool_name]` (the schema for that tool).","cost":0},{"level":2,"body":"`validate(args, SCHEMAS[tool_name])` — pass the args dict and the per-tool schema lookup.","cost":0}],"personalize":false,"phase":"build","estSeconds":65,"concept":"validate-before-dispatch","prompt":"The dispatcher is about to call the tool. Fill in the call to\n`validate(args, schema)` and check the `ok` field before running.\nOn failure, the dispatcher returns the error string so it can\nflow back to the model as a `tool_result`.\n\nExpected output:\n```\nERROR: missing required field: q\n```\n","code":"def validate(args, schema):\n for field in schema.get(\"required\", []):\n if field not in args:\n return {\"ok\": False, \"error\": f\"missing required field: {field}\"}\n return {\"ok\": True, \"args\": args}\n\nTOOLS = {\"search\": lambda q: f\"results for {q}\"}\nSCHEMAS = {\"search\": {\"required\": [\"q\"]}}\n\ndef dispatch(tool_name, args):\n verdict = ___\n if verdict[\"ok\"]:\n return TOOLS[tool_name](**verdict[\"args\"])\n return f\"ERROR: {verdict['error']}\"\n\nprint(dispatch(\"search\", {\"query\": \"ramen\"}))\n","blanks":[{"id":"validate_call","accept":["validate(args, SCHEMAS[tool_name])"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"capstone/validate-tool-inputs/s06-06-fix-the-untyped-tool","xp":4,"hint":[{"level":1,"body":"Move the `validate(...)` call BEFORE the tool runs. If the verdict is bad, return the error and never call the tool.","cost":0},{"level":2,"body":"Reorder the body of `dispatch` so validation comes first:\n\n```python\nverdict = validate(args, SCHEMAS[tool_name])\nif not verdict[\"ok\"]:\n return f\"ERROR: {verdict['error']}\"\nreturn TOOLS[tool_name](**args)\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"validate-before-tool-runs","brokenCode":"def validate(args, schema):\n for field, expected in schema.get(\"types\", {}).items():\n if field in args and not isinstance(args[field], expected):\n actual = type(args[field]).__name__\n return {\"ok\": False, \"error\": f\"{field} should be {expected.__name__}, got {actual}\"}\n return {\"ok\": True, \"args\": args}\n\nTOOLS = {\"search\": lambda q: f\"results for {q}\"}\nSCHEMAS = {\"search\": {\"types\": {\"q\": str}}}\n\ndef dispatch(tool_name, args):\n # bug: tool runs before validation. bad args reach the tool.\n result = TOOLS[tool_name](**args)\n verdict = validate(args, SCHEMAS[tool_name])\n if not verdict[\"ok\"]:\n return f\"ERROR: {verdict['error']}\"\n return result\n\nprint(dispatch(\"search\", {\"q\": 42}))\n","prompt":"The dispatcher runs the tool BEFORE validating the args. The model\npassed `{\"q\": 42}` instead of `{\"q\": \"ramen\"}` (number instead of\nstring). The tool's f-string still works, but downstream the\nsearch API rejects it with a confusing 400 error.\n\nFix the dispatcher to validate FIRST, then only run the tool on\nvalid args. On failure return the error message; never let the\ntool see bad input.\n\nExpected output:\n```\nERROR: q should be str, got int\n```\n","grader":{"kind":"stdout-equality","expected":"ERROR: q should be str, got int","normalize":"collapse-trailing-newline"},"bugLines":[14],"revealAfter":5},{"type":"fix","id":"capstone/validate-tool-inputs/s07-07-fix-the-error-leak","xp":4,"hint":[{"level":1,"body":"Replace the `raise ValueError(...)` with a `return f\"VALIDATION_ERROR: {verdict[\\\"error\\\"]}\"`. The string flows back as the tool_result content; the loop survives.","cost":0},{"level":2,"body":"Change line 12 to:\n\n```python\nreturn f\"VALIDATION_ERROR: {verdict['error']}\"\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"error-as-tool-result","brokenCode":"def validate(args, schema):\n for field in schema.get(\"required\", []):\n if field not in args:\n return {\"ok\": False, \"error\": f\"missing required field: {field}\"}\n return {\"ok\": True, \"args\": args}\n\nTOOLS = {\"search\": lambda q: f\"results for {q}\"}\nSCHEMAS = {\"search\": {\"required\": [\"q\"]}}\n\ndef dispatch(tool_name, args):\n verdict = validate(args, SCHEMAS[tool_name])\n if not verdict[\"ok\"]:\n # bug: raising kills the loop instead of feeding back to the model.\n raise ValueError(verdict[\"error\"])\n return TOOLS[tool_name](**args)\n\ntry:\n result = dispatch(\"search\", {\"query\": \"ramen\"})\nexcept ValueError as e:\n result = \"the loop crashed\"\nprint(result)\n","prompt":"The validator now runs first — but on failure the dispatcher\nraises `ValueError` instead of returning a string. That bubbles\nup through the agent loop and crashes it. The model never sees\nthe error and can't self-correct.\n\nFix the dispatcher to RETURN a string on validation failure\n(formatted as `\"VALIDATION_ERROR: \"`) so the agent loop\ncan package it as a `tool_result` block and feed it back to the\nmodel.\n\nExpected output:\n```\nVALIDATION_ERROR: missing required field: q\n```\n","grader":{"kind":"stdout-equality","expected":"VALIDATION_ERROR: missing required field: q","normalize":"collapse-trailing-newline"},"bugLines":[12],"revealAfter":5},{"type":"write","id":"capstone/validate-tool-inputs/s08-08-write-the-validator","xp":5,"hint":[{"level":1,"body":"Three branches: tool not in SCHEMAS → UNKNOWN_TOOL; validate fails → VALIDATION_ERROR; both pass → dispatch and return result.","cost":0},{"level":2,"body":"```python\ndef dispatch_with_validation(tool_name, args):\n schema = SCHEMAS.get(tool_name)\n if schema is None:\n return f\"UNKNOWN_TOOL: {tool_name}\"\n verdict = validate(args, schema)\n if not verdict[\"ok\"]:\n return f\"VALIDATION_ERROR: {verdict['error']}\"\n return TOOLS[tool_name](**verdict[\"args\"])\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"dispatch-with-validation","prompt":"Write `dispatch_with_validation(tool_name, args)` that:\n\n- Looks up `SCHEMAS[tool_name]`. If missing, return\n `\"UNKNOWN_TOOL: \"` (don't crash).\n- Runs `validate(args, schema)`. If invalid, return\n `\"VALIDATION_ERROR: \"`.\n- On valid: looks up `TOOLS[tool_name]`, calls it with\n `**verdict[\"args\"]`, returns the result.\n\nThree cases run for you. Expected output:\n```\nresults for ramen\nVALIDATION_ERROR: missing required field: q\nUNKNOWN_TOOL: frobnicate\n```\n","starter":"def validate(args, schema):\n for field in schema.get(\"required\", []):\n if field not in args:\n return {\"ok\": False, \"error\": f\"missing required field: {field}\"}\n for field, expected_type in schema.get(\"types\", {}).items():\n if field in args and not isinstance(args[field], expected_type):\n actual = type(args[field]).__name__\n return {\"ok\": False, \"error\": f\"{field} should be {expected_type.__name__}, got {actual}\"}\n return {\"ok\": True, \"args\": args}\n\nTOOLS = {\"search\": lambda q: f\"results for {q}\"}\nSCHEMAS = {\"search\": {\"required\": [\"q\"], \"types\": {\"q\": str}}}\n\n# define dispatch_with_validation(tool_name, args) below\n\nprint(dispatch_with_validation(\"search\", {\"q\": \"ramen\"}))\nprint(dispatch_with_validation(\"search\", {\"query\": \"ramen\"}))\nprint(dispatch_with_validation(\"frobnicate\", {\"x\": 1}))\n","grader":{"kind":"stdout-equality","expected":"results for ramen\nVALIDATION_ERROR: missing required field: q\nUNKNOWN_TOOL: frobnicate\n","normalize":"collapse-trailing-newline"},"solution":"$39","hiddenTests":[]},{"type":"checkpoint","id":"capstone/validate-tool-inputs/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"validated-agent-loop","prompt":"$3a","starter":"$3b","grader":{"kind":"stdout-equality","expected":"ok=True iters=3 validation_errors=1 answer=Found a 2026 ramen guide.","normalize":"collapse-trailing-newline"},"solution":"$3c","passThreshold":0.66}],"xpTotal":32},{"slug":"add-evals-and-traces","title":"Add evals and traces — measure the agent, don't trust it","estMinutes":13,"prerequisites":["build-a-cli-agent","wire-the-real-model","validate-tool-inputs"],"status":"live","steps":[{"type":"read","id":"capstone/add-evals-and-traces/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":110,"concept":"instrument-then-eval","body":"$3d","cta":"Got it","code":"$3e","runnable":true},{"type":"mc","id":"capstone/add-evals-and-traces/s02-02-which-eval-fits","xp":2,"hint":[{"level":1,"body":"Exact = fixed string. Contains = substring with known answer. Schema = structured output validation. Judge = \"did it sound right\" / semantic. Pick the cheapest pattern that catches the bug for each case.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"eval-pattern-selection","prompt":"Four eval patterns: exact, contains, schema, judge. Match each case\nto its strongest fit. Which combination is correct?\n","options":[{"id":"correct_pairing","label":"A=contains, B=judge, C=schema, D=contains (literal 'I don't know' substring)"},{"id":"all_judge","label":"All four → LLM-as-judge","explain":"A and C have ground truth (a known capital, a known schema). Burning judge calls on those is wasteful and adds bias risk for no gain."},{"id":"all_exact","label":"All four → exact match","explain":"B's output varies by tone — exact match would fail valid replies. C's JSON has key ordering issues. Exact only works for fixed strings."},{"id":"a_judge_b_contains","label":"A=judge, B=contains, C=judge, D=schema","explain":"A is fact recall — `'Paris' in answer` is the cheap, deterministic check. C produces structured JSON — that's a schema check, not a judge."}],"answerIds":["correct_pairing"],"shuffle":false,"code":"# four agent capstone cases. each needs the right eval pattern.\ncases = {\n \"A\": \"Agent returns the capital of a country. Question: 'capital of France?'\",\n \"B\": \"Agent writes a friendly support reply. Tone matters more than exact words.\",\n \"C\": \"Agent extracts a structured invoice (number, date, total) as JSON.\",\n \"D\": \"Agent should answer 'I don't know' when the question is ambiguous.\",\n}\nfor k, v in cases.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"capstone/add-evals-and-traces/s03-03-the-trace-shape","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":90,"concept":"minimal-trace-record","body":"$3f","cta":"Got it","code":"# the minimum viable trace record per turn.\n# six fields. anything else is nice-to-have.\n\ntrace = []\n\nturn_record = {\n \"turn\": 1,\n \"stop_reason\": \"tool_use\",\n \"tools_called\": [\"search\"],\n \"input_tokens\": 412,\n \"output_tokens\": 38,\n \"validation_errors\": 0,\n}\ntrace.append(turn_record)\n\nfor k, v in turn_record.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"predict","id":"capstone/add-evals-and-traces/s04-04-predict-the-pass-rate","xp":3,"hint":[{"level":1,"body":"Case 1: \"Paris\" in \"Paris is...\" → True. Case 2: \"Berlin\" in \"Berlin is...\" → True. Case 3: \"Tokyo\" in \"I think it might be Kyoto.\" → False. Two pass, one fails.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"eval-suite-output","code":"# tiny eval suite. three cases, each a (question, expected_substring).\n# the agent returns a fixed answer per question.\n\nAGENT_ANSWERS = {\n \"capital of France?\": \"Paris is the capital of France.\",\n \"capital of Germany?\": \"Berlin is the capital, sort of probably.\",\n \"capital of Japan?\": \"I think it might be Kyoto.\",\n}\n\ndef run_agent(question):\n return {\"answer\": AGENT_ANSWERS[question]}\n\ncases = [\n (\"capital of France?\", \"Paris\"),\n (\"capital of Germany?\", \"Berlin\"),\n (\"capital of Japan?\", \"Tokyo\"),\n]\n\npassed = 0\nfor question, expected in cases:\n result = run_agent(question)\n if expected in result[\"answer\"]:\n passed += 1\nprint(f\"{passed} / {len(cases)} passed\")\n","prompt":"The agent's canned answers are above. Each case checks whether the\nexpected substring is in the agent's answer. The third case will\nfail (agent says \"Kyoto\" instead of \"Tokyo\"). What does the suite\nprint?\n","grader":{"kind":"stdout-equality","expected":"2 / 3 passed","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/add-evals-and-traces/s05-05-fill-the-trace-append","xp":3,"hint":[{"level":1,"body":"Build the dict literal with three keys: `turn` (i + 1, 1-based), `stop_reason` (read from response), `tokens` (sum of input_tokens + output_tokens). Append to `trace`.","cost":0},{"level":2,"body":"`trace.append({\"turn\": i + 1, \"stop_reason\": response[\"stop_reason\"], \"tokens\": response[\"usage\"][\"input_tokens\"] + response[\"usage\"][\"output_tokens\"]})`","cost":0}],"personalize":false,"phase":"build","estSeconds":60,"concept":"per-turn-trace-record","prompt":"The agent loop tracks a `trace` list and is about to append the\nper-turn record. Fill in the `.append(...)` call so each entry is\na dict with `turn`, `stop_reason`, and `tokens` fields.\n\nExpected output:\n```\n[{'turn': 1, 'stop_reason': 'end_turn', 'tokens': 35}]\n```\n","code":"trace = []\ni = 0\nresponse = {\"stop_reason\": \"end_turn\", \"usage\": {\"input_tokens\": 30, \"output_tokens\": 5}}\n\n___\n\nprint(trace)\n","blanks":[{"id":"append_call","accept":["trace.append({\"turn\": i + 1, \"stop_reason\": response[\"stop_reason\"], \"tokens\": response[\"usage\"][\"input_tokens\"] + response[\"usage\"][\"output_tokens\"]})"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"capstone/add-evals-and-traces/s06-06-fix-the-trace-mutation","xp":4,"hint":[{"level":1,"body":"Move the `record = {}` line INSIDE the loop, so each turn allocates a fresh dict. Then append the fresh dict.","cost":0},{"level":2,"body":"Restructure the loop body:\n\n```python\nfor i, response in enumerate(responses):\n record = {\"turn\": i + 1, \"stop_reason\": response[\"stop_reason\"]}\n trace.append(record)\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"append-only-trace-deepcopy","brokenCode":"trace = []\nrecord = {} # bug: one record, reused across turns\n\nresponses = [\n {\"stop_reason\": \"tool_use\"},\n {\"stop_reason\": \"end_turn\"},\n]\n\nfor i, response in enumerate(responses):\n record[\"turn\"] = i + 1\n record[\"stop_reason\"] = response[\"stop_reason\"]\n trace.append(record)\n\nprint(trace)\n","prompt":"The loop is reusing one `record` dict across all turns and mutating\nit in place. Each iteration overwrites the previous record's\nfields, then appends the SAME object reference to the trace. By\nthe end, every entry in `trace` is the same dict — the last turn's\nvalues, repeated.\n\nFix the loop so each turn gets its OWN dict (build a fresh one\ninside the loop), so the trace is a faithful timeline.\n\nExpected output:\n```\n[{'turn': 1, 'stop_reason': 'tool_use'}, {'turn': 2, 'stop_reason': 'end_turn'}]\n```\n","grader":{"kind":"stdout-equality","expected":"[{'turn': 1, 'stop_reason': 'tool_use'}, {'turn': 2, 'stop_reason': 'end_turn'}]","normalize":"collapse-trailing-newline"},"bugLines":[2],"revealAfter":5},{"type":"fix","id":"capstone/add-evals-and-traces/s07-07-fix-the-tautological-eval","xp":4,"hint":[{"level":1,"body":"Replace `agent_said` (which is the agent's output) with an INDEPENDENT ground-truth string like `\"Paris\"`. The check `\"Paris\" in result[\"answer\"]` actually verifies correctness.","cost":0},{"level":2,"body":"Change line 9 to:\n\n```python\npassed = \"Paris\" in result[\"answer\"]\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"independent-eval-assertion","brokenCode":"def run_agent(question):\n return {\"answer\": \"Paris is the capital.\"}\n\nquestion = \"capital of France?\"\nresult = run_agent(question)\nagent_said = result[\"answer\"]\n\n# bug: assertion compares the agent's answer to itself.\npassed = agent_said in result[\"answer\"]\nprint(f\"pass: {passed}\")\n","prompt":"Cursor wrote this eval: it calls the agent, gets the answer, then\nasserts the answer equals what the agent returned. The eval ALWAYS\npasses — it's checking the agent against itself. Tautological.\n\nFix the eval to assert against an INDEPENDENT ground truth string\n(\"Paris\") that came from a fact-check, not from the agent.\n\nExpected output:\n```\npass: True\n```\n","grader":{"kind":"stdout-equality","expected":"pass: True","normalize":"collapse-trailing-newline"},"bugLines":[9],"revealAfter":4},{"type":"write","id":"capstone/add-evals-and-traces/s08-08-write-the-eval-runner","xp":5,"hint":[{"level":1,"body":"Loop over cases. For each: run agent, compute three booleans (answer substring, turn cap, end_turn exit), AND them, increment counter on True. Return totals + rate.","cost":0},{"level":2,"body":"Three asserts AND-ed together: `c[\"expected\"] in result[\"answer\"]`, `len(result[\"trace\"]) <= c[\"max_turns\"]`, `result[\"trace\"][-1][\"stop_reason\"] == \"end_turn\"`. Increment if all three are True.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"agent-eval-runner","prompt":"Write `run_eval_suite(cases)` that:\n\n- Takes a list of dicts, each shaped\n `{\"question\": str, \"expected\": str, \"max_turns\": int}`.\n- For each case:\n - Calls `run_agent(question)` (provided) which returns\n `{\"answer\": str, \"trace\": list}`.\n - Asserts THREE things, AND combines into a single boolean:\n a) `expected` substring is in `answer`,\n b) `len(trace)` is `<= max_turns`,\n c) `trace[-1][\"stop_reason\"] == \"end_turn\"`.\n- Counts passes (all three asserts true) and fails.\n- Returns `{\"total\": , \"passed\": , \"failed\": , \"pass_rate\": <0.0-1.0 rounded to 2 places>}`.\n\nThree cases provided. Expected output:\n```\ntotal=3 passed=2 failed=1 pass_rate=0.67\n```\n","starter":"$40","grader":{"kind":"stdout-equality","expected":"total=3 passed=2 failed=1 pass_rate=0.67","normalize":"collapse-trailing-newline"},"solution":"$41","hiddenTests":[]},{"type":"checkpoint","id":"capstone/add-evals-and-traces/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":220,"concept":"instrumented-agent-eval-suite","prompt":"Final drill. Build the fully instrumented agent + eval suite.\nWrite `run_agent(question)` that:\n\n- Loops up to 5 times calling `fake_create(question, turn_index)`\n which returns a dict with `stop_reason`, `text`, and a nested\n `usage` dict containing `input_tokens` and `output_tokens`\n (matching the SDK shape established in lesson 02).\n- Builds a `trace` list — one fresh dict per turn, with keys\n `turn` (1-based), `stop_reason`, `tokens` (input + output).\n- On `end_turn`: returns `{\"answer\": , \"trace\": }`.\n- On cap: returns `{\"answer\": \"capped\", \"trace\": }`.\n\nThen write `run_eval_suite(cases)` that:\n\n- For each case `{\"question\", \"expected\", \"max_turns\"}`, runs the\n agent.\n- All-three-true: `expected` in `answer`, `len(trace) <= max_turns`,\n `trace[-1][\"stop_reason\"] == \"end_turn\"`.\n- Returns\n `{\"total\", \"passed\", \"failed\", \"pass_rate\" (rounded 2)}`.\n\nTwo cases run for you. Expected output:\n```\ntotal=2 passed=1 failed=1 pass_rate=0.5\n```\n","starter":"$42","grader":{"kind":"stdout-equality","expected":"total=2 passed=1 failed=1 pass_rate=0.5","normalize":"collapse-trailing-newline"},"solution":"$43","passThreshold":0.66}],"xpTotal":32},{"slug":"wire-an-mcp-tool","title":"Wire an MCP tool — load tools from a server, not a registry","estMinutes":13,"prerequisites":["build-a-cli-agent","wire-the-real-model","validate-tool-inputs","add-evals-and-traces"],"status":"live","steps":[{"type":"read","id":"capstone/wire-an-mcp-tool/s01-01-intro","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":110,"concept":"mcp-as-tool-source","body":"$44","cta":"Got it","code":"# MCP turns \"tools my code defines\" into \"tools the agent discovers\n# at runtime by asking a server.\" JSON-RPC 2.0 over stdio or HTTP.\n\n# tools/list response (server says: here are my tools)\ntools_list_response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {\n \"tools\": [\n {\n \"name\": \"search\",\n \"description\": \"Web search.\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\"q\": {\"type\": \"string\"}},\n \"required\": [\"q\"],\n },\n },\n ],\n },\n}\n\n# adapt the server's tool definitions into our local registry shape.\nfor tool in tools_list_response[\"result\"][\"tools\"]:\n print(f\"name: {tool['name']}\")\n print(f\"desc: {tool['description']}\")\n print(f\"required: {tool['inputSchema']['required']}\")\n","runnable":true},{"type":"mc","id":"capstone/wire-an-mcp-tool/s02-02-when-to-use-mcp","xp":2,"hint":[{"level":1,"body":"MCP earns its keep when (a) the tool is *user-installed* / *third-party*, (b) the tool already lives in another process anyway (Slack, Notion, a DB), or (c) sandboxing matters. Hot-path / local-fast-path tools stay in the registry.","cost":0}],"personalize":false,"phase":"warmup","estSeconds":55,"concept":"mcp-vs-local","prompt":"Four tools. MCP adds latency (network/IPC hop) but enables plugins,\ncross-language, and sandboxing. Which TWO are the strongest fit\nfor MCP (rather than a local registry function)?\n","options":[{"id":"b_and_c","label":"B (Slack OAuth) and C (third-party plugin)"},{"id":"a_and_d","label":"A (5ms hot path) and D (50ms local SQL)","explain":"Both are high-frequency local calls — adding a network hop per call multiplies latency for no plugin benefit."},{"id":"a_and_b","label":"A and B","explain":"A is a hot path. The 5ms work doesn't justify a 10ms IPC hop."},{"id":"c_and_d","label":"C and D","explain":"D's a local SQLite call you already wrote. MCP-ifying it adds latency without adding capability."}],"answerIds":["b_and_c"],"shuffle":false,"code":"scenarios = {\n \"A\": \"Tool that runs a 5ms in-memory dict lookup, called 1000x per agent run.\",\n \"B\": \"Tool that integrates with the user's Slack workspace via OAuth.\",\n \"C\": \"Tool published by a third party (e.g., a Notion-MCP, a Postgres-MCP).\",\n \"D\": \"Tool that wraps a 50ms local SQLite query, used in 90% of agent calls.\",\n}\nfor k, v in scenarios.items():\n print(f\"{k}: {v}\")\n","runnable":true},{"type":"read","id":"capstone/wire-an-mcp-tool/s03-03-the-mcp-protocol","xp":2,"hint":[],"personalize":false,"phase":"build","estSeconds":100,"concept":"mcp-tools-call-shape","body":"$45","cta":"Got it","code":"# the shape of a tools/call request and response.\n# JSON-RPC 2.0 — id correlates request to response.\n\nrequest = {\n \"jsonrpc\": \"2.0\",\n \"id\": 42,\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"search\",\n \"arguments\": {\"q\": \"ramen tokyo\"},\n },\n}\n\n# success response:\nok_response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 42,\n \"result\": {\n \"content\": [\n {\"type\": \"text\", \"text\": \"3 results for ramen tokyo\"},\n ],\n \"isError\": False,\n },\n}\n\n# error response (server's tool failed, but the protocol succeeded):\nerr_response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 42,\n \"result\": {\n \"content\": [\n {\"type\": \"text\", \"text\": \"search service unavailable\"},\n ],\n \"isError\": True,\n },\n}\n\nfor label, r in [(\"ok\", ok_response), (\"err\", err_response)]:\n content = r[\"result\"][\"content\"][0][\"text\"]\n err_flag = r[\"result\"][\"isError\"]\n print(f\"{label}: text={content!r} isError={err_flag}\")\n","runnable":true},{"type":"predict","id":"capstone/wire-an-mcp-tool/s04-04-predict-the-tool-list","xp":3,"hint":[{"level":1,"body":"List comprehension over `response[\"result\"][\"tools\"]`, picking each tool's `name` field. The result is a Python list of strings.","cost":0},{"level":2,"body":"`[\"search\", \"read_url\", \"summarize\"]` — three strings, in the order they appeared in the response.","cost":0}],"personalize":false,"phase":"build","estSeconds":75,"concept":"tools-list-parsing","code":"# a real tools/list response from an MCP server.\nresponse = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {\n \"tools\": [\n {\"name\": \"search\", \"description\": \"Web search.\", \"inputSchema\": {\"required\": [\"q\"]}},\n {\"name\": \"read_url\", \"description\": \"Fetch a page.\", \"inputSchema\": {\"required\": [\"url\"]}},\n {\"name\": \"summarize\",\"description\": \"Compress text.\",\"inputSchema\": {\"required\": [\"text\"]}},\n ],\n },\n}\n\n# extract just the tool names.\ntool_names = [t[\"name\"] for t in response[\"result\"][\"tools\"]]\nprint(tool_names)\n","prompt":"Read the editor. The MCP server's `tools/list` response is parsed\nto extract just the tool names. What does the script print?\n","grader":{"kind":"stdout-equality","expected":"['search', 'read_url', 'summarize']","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/wire-an-mcp-tool/s05-05-fill-the-tools-call","xp":3,"hint":[{"level":1,"body":"Two keys: `name` (the tool to call as a string) and `arguments` (the dict of inputs). Build a dict literal.","cost":0},{"level":2,"body":"`{\"name\": \"search\", \"arguments\": {\"q\": \"ramen\"}}` — JSON-RPC params with the tool name and its argument dict.","cost":0}],"personalize":false,"phase":"build","estSeconds":65,"concept":"jsonrpc-tools-call-request","prompt":"You're constructing a JSON-RPC `tools/call` request. The method\nstring is fixed (`\"tools/call\"`), the params carry `name` and\n`arguments`. Fill in the params dict so the server runs the\n`search` tool with `{\"q\": \"ramen\"}`.\n\nExpected output:\n```\n{'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call', 'params': {'name': 'search', 'arguments': {'q': 'ramen'}}}\n```\n","code":"request = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"method\": \"tools/call\",\n \"params\": ___,\n}\nprint(request)\n","blanks":[{"id":"params_dict","accept":["{\"name\": \"search\", \"arguments\": {\"q\": \"ramen\"}}","{'name': 'search', 'arguments': {'q': 'ramen'}}"],"caseSensitive":false,"normalize":"trim"}]},{"type":"fix","id":"capstone/wire-an-mcp-tool/s06-06-fix-the-result-shape","xp":4,"hint":[{"level":1,"body":"The content list lives inside `response[\"result\"]`, not at the top level. Add `\"result\"` to the access chain.","cost":0},{"level":2,"body":"Change line 12 to:\n\n```python\ntext = response[\"result\"][\"content\"][0][\"text\"]\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":95,"concept":"result-content-unwrap","brokenCode":"response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 42,\n \"result\": {\n \"content\": [\n {\"type\": \"text\", \"text\": \"3 results for ramen tokyo\"},\n ],\n \"isError\": False,\n },\n}\n\n# bug: missing the result wrap. raises KeyError.\ntext = response[\"content\"][0][\"text\"]\nprint(text)\n","prompt":"The code is reading `response[\"content\"]` directly, expecting the\ncontent list. But MCP wraps that list inside `response[\"result\"]`.\nAs written, this raises `KeyError: 'content'`.\n\nFix the access path to read from `response[\"result\"][\"content\"][0][\"text\"]`.\n\nExpected output:\n```\n3 results for ramen tokyo\n```\n","grader":{"kind":"stdout-equality","expected":"3 results for ramen tokyo","normalize":"collapse-trailing-newline"},"bugLines":[12],"revealAfter":4},{"type":"fix","id":"capstone/wire-an-mcp-tool/s07-07-fix-the-iserror-flag","xp":4,"hint":[{"level":1,"body":"After reading the text, branch on `response[\"result\"][\"isError\"]`. If True, prefix the text with `\"TOOL_ERROR: \"` before printing.","cost":0},{"level":2,"body":"Replace line 13 with:\n\n```python\ntext = response[\"result\"][\"content\"][0][\"text\"]\nif response[\"result\"][\"isError\"]:\n text = f\"TOOL_ERROR: {text}\"\n```\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"iserror-tool-failure-detection","brokenCode":"response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 42,\n \"result\": {\n \"content\": [\n {\"type\": \"text\", \"text\": \"search service unavailable\"},\n ],\n \"isError\": True,\n },\n}\n\n# bug: ignores isError. error message looks like a successful result.\ntext = response[\"result\"][\"content\"][0][\"text\"]\nprint(text)\n","prompt":"The MCP server returned a successful JSON-RPC response — but\n`result.isError` is `True`, meaning the tool itself reported\nfailure. The current code ignores `isError` and treats the error\nmessage as a successful tool result, which the agent will then\nconfidently relay to the user.\n\nFix the code to check `isError` and prefix the text with\n`\"TOOL_ERROR: \"` when the flag is `True`, so the agent sees the\nfailure and can recover.\n\nExpected output:\n```\nTOOL_ERROR: search service unavailable\n```\n","grader":{"kind":"stdout-equality","expected":"TOOL_ERROR: search service unavailable","normalize":"collapse-trailing-newline"},"bugLines":[13],"revealAfter":5},{"type":"write","id":"capstone/wire-an-mcp-tool/s08-08-write-the-mcp-bridge","xp":5,"hint":[{"level":1,"body":"Iterate `tools_list_response[\"result\"][\"tools\"]`. For each tool, store its `inputSchema` in `schemas[name]` and a closure in `tools[name]`. The closure calls `mcp_call(name, args)`, unwraps the result text, and prefixes `TOOL_ERROR:` on isError.","cost":0},{"level":2,"body":"Use a `make_closure(tool_name)` helper to capture the name correctly — Python closures-over-loop-variable bug otherwise (every tool ends up calling with the LAST loop name).","cost":0}],"personalize":false,"phase":"build","estSeconds":200,"concept":"mcp-to-registry-bridge","prompt":"Write `bridge_mcp_tools(tools_list_response, mcp_call)` that:\n\n- Takes a `tools/list` response (a dict with\n `response[\"result\"][\"tools\"]`) and a `mcp_call(tool_name, args)`\n function (which simulates calling the server's `tools/call` and\n returns a `tools/call` response dict).\n- Returns a tuple `(tools, schemas)`:\n - `tools` is a dict mapping each tool's `name` to a closure that\n calls `mcp_call(name, args)`, unwraps `response[\"result\"][\"content\"][0][\"text\"]`,\n and returns it (prefixed with `\"TOOL_ERROR: \"` if `isError` is True).\n - `schemas` is a dict mapping each tool's `name` to its\n `inputSchema`.\n\nThree tools come back. After bridging, dispatch one of them to\nprove it works. Expected output:\n```\ntool_names: ['search', 'read_url', 'summarize']\nsearch_result: 3 results for ramen\n```\n","starter":"tools_list_response = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {\n \"tools\": [\n {\"name\": \"search\", \"description\": \"Web search.\", \"inputSchema\": {\"required\": [\"q\"]}},\n {\"name\": \"read_url\", \"description\": \"Fetch a page.\", \"inputSchema\": {\"required\": [\"url\"]}},\n {\"name\": \"summarize\",\"description\": \"Compress text.\",\"inputSchema\": {\"required\": [\"text\"]}},\n ],\n },\n}\n\ndef mcp_call(tool_name, args):\n # simulate the server's tools/call response\n return {\n \"jsonrpc\": \"2.0\",\n \"id\": 99,\n \"result\": {\n \"content\": [{\"type\": \"text\", \"text\": f\"3 results for {args.get('q', '?')}\"}],\n \"isError\": False,\n },\n }\n\n# define bridge_mcp_tools(tools_list_response, mcp_call) below\n\ntools, schemas = bridge_mcp_tools(tools_list_response, mcp_call)\nprint(f\"tool_names: {list(tools.keys())}\")\nprint(f\"search_result: {tools['search'](q='ramen')}\")\n","grader":{"kind":"stdout-equality","expected":"tool_names: ['search', 'read_url', 'summarize']\nsearch_result: 3 results for ramen\n","normalize":"collapse-trailing-newline"},"solution":"$46","hiddenTests":[]},{"type":"checkpoint","id":"capstone/wire-an-mcp-tool/s09-09-checkpoint","xp":8,"hint":[],"personalize":false,"phase":"check","estSeconds":230,"concept":"agent-with-mcp-tools","prompt":"$47","starter":"$48","grader":{"kind":"stdout-equality","expected":"ok=True iters=2 tool_calls=1 tool_errors=0 answer=Found Tokyo's best ramen.\nok=True iters=2 tool_calls=1 tool_errors=1 answer=I couldn't search; tool was down.\n","normalize":"collapse-trailing-newline"},"solution":"$49","passThreshold":0.66}],"xpTotal":32},{"slug":"portfolio-ml-system-option","title":"The ML-system option — six boring stages that get a repo taken seriously","estMinutes":16,"prerequisites":["pick-a-project-that-ships"],"status":"live","steps":[{"type":"read","id":"capstone/portfolio-ml-system-option/s01-01-the-other-capstone","xp":1,"hint":[],"personalize":false,"phase":"warmup","estSeconds":240,"concept":"six-stage-ml-system","body":"$4a","cta":"Got it","code":"# the whole system, as a table of contents. six stages, no exceptions.\nSTAGES = [\n \"data pull\", # fetch the raw data, snapshot it to disk\n \"validation\", # refuse rows that don't match the schema\n \"train + eval\", # fit a boring model, score it on held-out data\n \"gate\", # ship only if the eval clears the bar\n \"serving stub\", # one function: features in, prediction out\n \"monitoring hook\", # log every prediction for tomorrow's triage\n]\n\nfor i, stage in enumerate(STAGES, 1):\n print(f\"{i}. {stage}\")\n","runnable":true},{"type":"mc","id":"capstone/portfolio-ml-system-option/s02-02-what-reviewers-open","xp":2,"hint":[{"level":1,"body":"Three options make the model look smarter. One shows the system\nknows when it's wrong — and that's the ninety-second signal a\nreviewer is actually scanning for.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":70,"concept":"what-reviewers-look-at","prompt":"A senior engineer opens your capstone repo with ninety seconds\nto form an opinion. You have one evening left before you send\nthe link. Which change buys the most trust?\n","options":[{"id":"deeper-model","label":"Swap the logistic regression for a small neural network.","explain":"The reviewer has seen a thousand model definitions and reads\nexactly none of them in a skim. A fancier model with the same\npipeline moves nothing — and if it nudges the eval below the\ngate, it moves you backwards.\n"},{"id":"gate-eval","label":"Make sure the gate and the held-out eval are visible and wired together."},{"id":"accuracy","label":"Push the reported accuracy from 0.83 to 0.9 by tuning on the test set.","explain":"Tuning on the test set is the one move that can end the\ninterview. A held-out number is only evidence while it's held\nout — reviewers probe for exactly this, because a 0.9 earned\nthis way is worth less than an honest 0.83.\n"},{"id":"more-features","label":"Add six more feature columns to the data pull.","explain":"More features widen the surface without proving judgment. The\nskim doesn't count columns; it checks whether the system can\ntell when it isn't good enough to ship.\n"}],"answerIds":["gate-eval"],"shuffle":false,"code":"# your repo, as the reviewer sees it in a 90-second skim.\nrepo = {\n \"README\": \"one command: python pipeline.py\",\n \"model\": \"logistic regression, 40 lines\",\n \"eval\": \"held-out accuracy 0.83 in eval.py\",\n \"gate\": \"deploy blocked below 0.80\",\n \"monitoring\": \"every prediction logged to preds.jsonl\",\n}\nfor part, note in repo.items():\n print(f\"{part}: {note}\")\n","runnable":true},{"type":"predict","id":"capstone/portfolio-ml-system-option/s03-03-predict-the-gate","xp":3,"hint":[{"level":1,"body":"1200 minus 37 is 1163. The gate compares `eval_accuracy` (0.78)\nto 0.80 — the 0.98 training score is never consulted, which is\nthe entire point of a gate. 0.78 >= 0.80 is False, so the\nverdict is the sad one.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":90,"concept":"gate-reads-eval-not-train","code":"# one pipeline run, summarized. note which number the gate reads.\nrun = {\n \"rows_pulled\": 1200,\n \"rows_dropped_by_validation\": 37,\n \"train_accuracy\": 0.98,\n \"eval_accuracy\": 0.78,\n}\nGATE = 0.80\n\nprint(f\"validated rows: {run['rows_pulled'] - run['rows_dropped_by_validation']}\")\nprint(f\"eval accuracy: {run['eval_accuracy']}\")\nverdict = \"ship\" if run[\"eval_accuracy\"] >= GATE else \"hold\"\nprint(f\"gate: {verdict}\")\n","prompt":"A pipeline run just finished. Training accuracy is a glowing\n0.98. Held-out accuracy is 0.78. The bar is 0.80.\n\nPredict all three output lines — the row count after\nvalidation, the accuracy line, and the gate's verdict. One of\nthe four numbers in `run` never gets read at all. Which one?\n","grader":{"kind":"stdout-equality","expected":"validated rows: 1163\neval accuracy: 0.78\ngate: hold\n","normalize":"collapse-trailing-newline"}},{"type":"fill","id":"capstone/portfolio-ml-system-option/s04-04-fill-the-gate","xp":3,"hint":[{"level":1,"body":"\"Clears the bar\" means the eval score is at least as high as the\nthreshold. You want the comparison that makes 0.83 against 0.80\ncome out True — and would make 0.78 come out False.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":60,"concept":"fill-the-gate-comparison","prompt":"Stage 4, the gate, is one comparison operator. Fill it in so\nthe model ships only when held-out accuracy clears the bar —\nand holds otherwise.\n\nExpected output:\n```\ngate: ship\n```\n","code":"eval_accuracy = 0.83\nGATE = 0.80\n\n# ship only when held-out performance clears the bar.\nverdict = \"ship\" if eval_accuracy ___ GATE else \"hold\"\nprint(f\"gate: {verdict}\")\n","blanks":[{"id":"op","accept":[">=",">"],"caseSensitive":true,"normalize":"trim"}]},{"type":"fix","id":"capstone/portfolio-ml-system-option/s05-05-fix-the-train-accuracy-gate","xp":4,"hint":[{"level":1,"body":"The dict holds two accuracies. One was measured on rows the model\ntrained on; the other on rows it never saw. A gate is only worth\nhaving if it reads the second one.\n","cost":0},{"level":2,"body":"Change line 8 to:\n\n```python\nscore = run[\"eval_accuracy\"]\n```\n\nNow the gate compares 0.71 to 0.80, and the overfit model is\ncorrectly held.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":100,"concept":"overfit-model-slips-the-gate","brokenCode":"run = {\n \"train_accuracy\": 0.99,\n \"eval_accuracy\": 0.71,\n}\nGATE = 0.80\n\n# the gate decides whether this model ships.\nscore = run[\"train_accuracy\"]\n\nverdict = \"ship\" if score >= GATE else \"hold\"\nprint(f\"gated on: {score}\")\nprint(f\"gate: {verdict}\")\n","prompt":"This gate is a rubber stamp. It reads *training* accuracy — and\nan overfit model always aces the data it memorized. The 0.99\nsails through while the held-out score, the only number that\npredicts real-world behavior, says 0.71.\n\nFix the gate so it reads the held-out score. The overfit model\nshould be held, not shipped.\n\nExpected output:\n```\ngated on: 0.71\ngate: hold\n```\n","grader":{"kind":"stdout-equality","expected":"gated on: 0.71\ngate: hold\n","normalize":"collapse-trailing-newline"},"bugLines":[8],"revealAfter":3},{"type":"write","id":"capstone/portfolio-ml-system-option/s06-06-write-the-scope-check","xp":5,"hint":[{"level":1,"body":"Iterate over `REQUIRED`, not over `plan` — that's what keeps the\noutput in canonical stage order. Keep each stage that fails the\n`in plan` test. A list comprehension does it in one line.\n","cost":0}],"personalize":false,"phase":"build","estSeconds":150,"concept":"scope-check-the-weekend","prompt":"Before Saturday morning, audit the plan. Write\n`missing_stages(plan)` — return a list of every stage from\n`REQUIRED` that is absent from `plan`, in `REQUIRED` order.\n\nThe weekend plan below has the classic shape: the fun stages\nare in, the unglamorous ones are missing. Expected output:\n```\n3/6 stages present\nmissing: validation\nmissing: gate\nmissing: monitoring hook\n```\n","starter":"REQUIRED = [\n \"data pull\",\n \"validation\",\n \"train + eval\",\n \"gate\",\n \"serving stub\",\n \"monitoring hook\",\n]\n\nweekend_plan = [\"data pull\", \"train + eval\", \"serving stub\"]\n\n# define missing_stages(plan) below\n\ngaps = missing_stages(weekend_plan)\nprint(f\"{len(REQUIRED) - len(gaps)}/{len(REQUIRED)} stages present\")\nfor g in gaps:\n print(\"missing:\", g)\n","grader":{"kind":"stdout-equality","expected":"3/6 stages present\nmissing: validation\nmissing: gate\nmissing: monitoring hook\n","normalize":"collapse-trailing-newline"},"solution":"REQUIRED = [\n \"data pull\",\n \"validation\",\n \"train + eval\",\n \"gate\",\n \"serving stub\",\n \"monitoring hook\",\n]\n\nweekend_plan = [\"data pull\", \"train + eval\", \"serving stub\"]\n\ndef missing_stages(plan):\n return [stage for stage in REQUIRED if stage not in plan]\n\ngaps = missing_stages(weekend_plan)\nprint(f\"{len(REQUIRED) - len(gaps)}/{len(REQUIRED)} stages present\")\nfor g in gaps:\n print(\"missing:\", g)\n","hiddenTests":[]},{"type":"checkpoint","id":"capstone/portfolio-ml-system-option/s07-07-checkpoint","xp":8,"hint":[{"level":1,"body":"Three computations, one dict. Count stage membership against\n`REQUIRED` (a generator inside `sum()` works), compare the two\naccuracy fields for the gate, then AND the two conditions for\n`portfolio_ready`. Neither condition alone is enough — that's\nthe lesson wearing a boolean.\n","cost":0},{"level":2,"body":"```python\ndef readiness_report(system):\n present = sum(1 for stage in REQUIRED if stage in system[\"stages\"])\n gate = \"ship\" if system[\"eval_accuracy\"] >= system[\"gate_threshold\"] else \"hold\"\n ready = present == len(REQUIRED) and gate == \"ship\"\n return {\"stages_present\": present, \"gate\": gate, \"portfolio_ready\": ready}\n```\n","cost":0}],"personalize":false,"phase":"check","estSeconds":240,"concept":"portfolio-readiness-report","prompt":"Last exercise of the capstone. Three candidate systems sit in\nyour portfolio folder; decide which one goes in the README.\nWrite `readiness_report(system)` returning a dict with:\n\n- `stages_present`: how many of the six `REQUIRED` stages\n appear in `system[\"stages\"]`\n- `gate`: `\"ship\"` if `eval_accuracy >= gate_threshold`,\n else `\"hold\"`\n- `portfolio_ready`: `True` only when **all six** stages are\n present AND the gate says `\"ship\"`\n\nThe runner prints one line per system. Note what it punishes:\nprice-model has the best accuracy of the three and still fails\n— no monitoring hook, no portfolio. Expected output:\n```\nchurn-predictor: 6/6 stages, gate ship, ready: True\nprice-model: 5/6 stages, gate ship, ready: False\nspam-filter: 6/6 stages, gate hold, ready: False\n```\n","starter":"REQUIRED = [\n \"data pull\",\n \"validation\",\n \"train + eval\",\n \"gate\",\n \"serving stub\",\n \"monitoring hook\",\n]\n\nsystems = [\n {\n \"name\": \"churn-predictor\",\n \"stages\": [\"data pull\", \"validation\", \"train + eval\", \"gate\", \"serving stub\", \"monitoring hook\"],\n \"eval_accuracy\": 0.84,\n \"gate_threshold\": 0.80,\n },\n {\n \"name\": \"price-model\",\n \"stages\": [\"data pull\", \"validation\", \"train + eval\", \"gate\", \"serving stub\"],\n \"eval_accuracy\": 0.91,\n \"gate_threshold\": 0.80,\n },\n {\n \"name\": \"spam-filter\",\n \"stages\": [\"data pull\", \"validation\", \"train + eval\", \"gate\", \"serving stub\", \"monitoring hook\"],\n \"eval_accuracy\": 0.74,\n \"gate_threshold\": 0.80,\n },\n]\n\n# define readiness_report(system) below\n\nfor s in systems:\n r = readiness_report(s)\n print(f\"{s['name']}: {r['stages_present']}/6 stages, gate {r['gate']}, ready: {r['portfolio_ready']}\")\n","grader":{"kind":"stdout-equality","expected":"churn-predictor: 6/6 stages, gate ship, ready: True\nprice-model: 5/6 stages, gate ship, ready: False\nspam-filter: 6/6 stages, gate hold, ready: False\n","normalize":"collapse-trailing-newline"},"solution":"$4b","passThreshold":0.66}],"xpTotal":26}],"liveLessonCount":7,"liveStepCount":62,"xpTotal":219}},"chapter":"$16:1:props:tree:detail","lesson":"$16:1:props:tree:detail:lessons:6","step":"$16:1:props:tree:detail:lessons:6:steps:3","stepIndex":3,"next":{"chapterSlug":"capstone","lessonSlug":"portfolio-ml-system-option","stepIndex":4}}]]

The ML-system option — six boring stages that get a repo taken seriously — step 4 of 7