Few-shot and reasoning — examples that work, and the CoT trap on reasoning models — step 8 of 9
Write build_prompt(task, examples, query, model) that:
- Always opens with the task line + a blank line.
- If
examplesis non-empty, appends each as"Input: <input>\nOutput: <output>\n\n". - Appends the query as
"Input: <query>\nOutput:". - If
modelis NOT inREASONING_MODELS, prepends a CoT line ("Think step by step.\n\n") BEFORE the examples — but ONLY if the task involves reasoning (we'll signal this withtask.startswith("Solve")). If the task is classification (theClassifyprefix in our test), skip CoT regardless of model. - Returns the prompt string.
Two cases run for you. Expected output:
--- classify on haiku (no CoT, has examples) ---
Classify sentiment.
Input: great
Output: positive
Input: bad
Output: negative
Input: okay
Output:
---
--- math on o3 (no CoT — reasoning model handles it) ---
Solve: what is 17 * 23?
Input: 17 * 23
Output:
⌘↵ runs the editor.read, then continue.
Write build_prompt(task, examples, query, model) that:
- Always opens with the task line + a blank line.
- If
examplesis non-empty, appends each as"Input: <input>\nOutput: <output>\n\n". - Appends the query as
"Input: <query>\nOutput:". - If
modelis NOT inREASONING_MODELS, prepends a CoT line ("Think step by step.\n\n") BEFORE the examples — but ONLY if the task involves reasoning (we'll signal this withtask.startswith("Solve")). If the task is classification (theClassifyprefix in our test), skip CoT regardless of model. - Returns the prompt string.
Two cases run for you. Expected output:
--- classify on haiku (no CoT, has examples) ---
Classify sentiment.
Input: great
Output: positive
Input: bad
Output: negative
Input: okay
Output:
---
--- math on o3 (no CoT — reasoning model handles it) ---
Solve: what is 17 * 23?
Input: 17 * 23
Output:
this step needs the editor
on desktop today; in the app (coming soon). save your spot and we'll bring you back here when you're ready.