promptdojo_

The tool contract — make the dangerous call a validation error — step 2 of 7

ch 57 · agent studio2/7

The schema is the blast radius

A tool named update that accepts anything lets a confused model update anything. That is not a bug in the model. It is a bug in the schema, and it's yours.

The fix is the same move you'd make on any public API: narrow the surface until misuse becomes a validation error. Run the code — that's the whole tool contract for the triage agent, as a dict. Four decisions per tool:

  • A specific name. rerun_job, not run. The name is documentation the model actually reads on every single call, which is more than you can say for your wiki.
  • Required fields. open_issue without a severity isn't a smaller issue — it's a rejected call. Missing field, no execution, error string back to the model. The model course-corrects; nothing ran.
  • Allowed values. severity is an enum: p1, p2, p3. The model that invents p4 (they do) or critical-ish (they really do) gets reject: bad severity, not a garbage issue in your tracker.
  • The confirm flag. rerun_job costs compute and can mask a real failure behind a green retry. escalate wakes a human. Both carry confirm: True: the call must arrive with an explicit confirmed field set by a human or an outer policy — never by the model deciding it's probably fine.

Why data beats code here

You could write this as four if statements per tool. Don't. The registry-as-dict shape pays three times:

  1. One validator covers every tool. Adding tool number six is a dict entry, not new branches in a function someone has to re-review.
  2. The contract is diffable. When a teammate flips rerun_job to confirm: False, that's one visible line in the PR — the exact line the reviewer should fight about.
  3. The same spec feeds the model. This dict is a short serialization away from the JSON schema you'd hand messages.create as a tool definition. One source of truth for what the model sees and what the validator enforces, so they can't drift apart.

Notice what's not in the contract: any trust in the model's judgment. The model can propose rerun_job all day. The schema decides whether proposing becomes doing. That split — model proposes, harness disposes — is the single design decision that separates agent features that survive contact with production from the demos that don't.

read, then continue.