Evaluator-optimizer — write a draft, let a judge critique it, revise — step 6 of 9
The judge is now returning free-text verdicts (the model wasn't
forced into structured output). The current loop checks
if "good" in verdict.lower() — and the judge said "this is not
good, please fix it". The substring match sees "good", exits the
loop happily, and ships the broken draft.
Fix the parsing so it requires the verdict to START with the
literal token "PASS" (case-sensitive). Anything else is a reject.
Expected output:
rejected, will revise
The judge is now returning free-text verdicts (the model wasn't
forced into structured output). The current loop checks
if "good" in verdict.lower() — and the judge said "this is not
good, please fix it". The substring match sees "good", exits the
loop happily, and ships the broken draft.
Fix the parsing so it requires the verdict to START with the
literal token "PASS" (case-sensitive). Anything else is a reject.
Expected output:
rejected, will revise
this step needs the editor
on desktop today; in the app (coming soon). save your spot and we'll bring you back here when you're ready.