The model picker — when Sonnet is wrong and Haiku is right (step 6/8) · llm apis

Four scenarios, four models. Match each scenario to the BEST model given the volume × stakes × latency tradeoff. Which assignment is right?

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k records/day with one of 12 categories. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k error logs/day with one of 12 incident labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k campaign assets/day with one of 12 ship labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k brand assets/day with one of 12 kit labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k support tickets/day with one of 12 categories. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k claim lines/day with one of 12 clearance labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k research cuts/day with one of 12 confidence labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k tracker rows/day with one of 12 risk labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k applications/day with one of 12 screening buckets. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k handoff steps/day with one of 12 exception labels. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

# four product scenarios with the volume / quality / latency
# tradeoffs spelled out. for each, you have to pick the model
# that wins on the actual cost-vs-quality math, not the default.

scenarios = [
    "A: Tag 500k discovery docs/day with one of 12 privilege categories. Quality must be good enough for triage, but misses are recoverable because humans catch the fallout.",
    "B: Once a quarter, review a 40-page acquisition contract and surface risk clauses for the legal team. Runs maybe 6 times a year.",
    "C: A real-time autocomplete for an IDE. Suggestions appear inline as the user types. Latency budget: under 200ms to first token. Volume: ~10 calls/keystroke, millions/day.",
    "D: A coding agent loop that picks one of 20 tools per turn, calls it, reads the result, and decides what to do next. Runs hundreds of times a day per active user.",
]
for s in scenarios:
    print(s)

Four scenarios, four models. Match each scenario to the BEST model given the volume × stakes × latency tradeoff. Which assignment is right?

The model picker — when Sonnet is wrong and Haiku is right — step 6 of 8