contract → tools → loop → gate → runbook
the build order is the lesson. people start with
model="whatever-is-new" and then discover they have no
inputs, no allowlist, and no way to fail a run. the model
string is a leaf. write it last, if you write it at all —
fake_model is enough to prove the loop.
1. contract
a HarnessSpec is the smallest dict that makes the loop
legible:
- name — what this harness is for (repo / PR triage)
- trigger — what lands an item on the loop
- inputs — the fields you actually load
- decisions — deterministic policy in code (route, approve, swap, compact). not a paragraph in the prompt
- writeback — the path the result hits
- eval_gate — the rule that can fail the run
if a field is missing, you do not have a harness. you have a script that calls a model.
task-harness fit
same loop skeleton. different brief. a repo / PR triage harness is not a long-running coding harness. the context it loads, the tools it may name, the policy it must enforce, and the cost it can burn are the fit. copy a coding agent's tool belt onto this brief and you will over-allow and under-gate.
2. tools
a registry (name → callable) plus an allowlist. the model proposes a name. the loop looks it up. if the name is not on the list, you skip it — you do not invent a tool because the model was confident. environment actions (read a file, run a command, write the record) live here.
3. loop
the four layers from lesson 01, now owned as loop.py:
- prep — spec + inputs → messages (the context drop)
- call —
fake_model(messages)(or a real client later) - parse — text blocks vs tool_use blocks
- dispatch — allowlisted tools only
and a cap. while True is how a harness lights money
on fire. retries and a model fallback sit on the call.
compaction of history sits on prep — deterministic, not
a "please forget the old turns" line in the prompt.
hooks exist around those four layers so that work can fire at a known point: before the call, after a tool, at the cap. people call that middleware. you do not need a framework class. you need a place that is not the prompt.
4. gate
evals.jsonl is the file. eval_gate is the rule. a run
returns pass or fail. if you cannot fail a run, you
cannot refuse to ship. that is the gate. not a screenshot.
not a vibe.
gates cover policy, human approval, and cost
(call / tool limits). if the allowlist was violated or
max_iters was hit, the gate fails — even if the model
sounded sure.
5. runbook
layer 4 writes back. the runbook is the undo sitting next to the harness — detect, decide, write the record. lesson 05 makes that a dict. here you only need the file to exist so the loop is not an orphan.
capability → build checklist
not a shopping list of class names. a checklist you hang on the order above:
| need | lives on |
|---|---|
| environment actions (fs / shell / writeback) | tools |
| memory read / write | tools (+ prep loads it) |
| retries + model fallback | loop (call) |
| context overflow / compaction | loop (prep) |
| delegation / todo progress | contract (decisions) |
| policy + human approval | gate |
| cost caps (call / tool limits) | gate + loop cap |
each concern stays isolated so the next harness inherits the pieces that already work. that is why the minimal repo is five files, not one script.
start at the contract. the model string can wait.