The four breakage classes — sort any LLM failure before you touch the prompt — step 4 of 9
Read the trace in the editor. The prompt is clear. The retriever returned zero chunks. The raw output and post-processed output are identical. The model still produced a confident, specific date answer.
Predict the output of the script — specifically the final
class: line.
Expected output:
retrieved_chunks empty? True
prompt clear? True
raw == postprocessed? True
model answered anyway? True
class: hallucination
⌘↵ runs the editor.read, then continue.
The four breakage classes — sort any LLM failure before you touch the prompt — step 4 of 9
Read the trace in the editor. The prompt is clear. The retriever returned zero chunks. The raw output and post-processed output are identical. The model still produced a confident, specific date answer.
Predict the output of the script — specifically the final
class: line.
Expected output:
retrieved_chunks empty? True
prompt clear? True
raw == postprocessed? True
model answered anyway? True
class: hallucination
⌘↵ runs the editor.read, then continue.