promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_

Decoding, KV cache, and quantization — step 7 of 7

ch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internals7/7
Checkpoint

One last thing before we move on. pass this to mark the lesson done, or skip and keep moving. hop to the next when you're ready.

Run the quantize-then-EVALUATE drill on a one-neuron 'model'. Compute the full-precision dot product of weights and x (print f"full-precision output: {out:.4f}"). For levels in (2, 4, 16), quantize weights to round(w * levels) / levels, recompute the output, and print f"levels={levels:>2}: output={...:.4f} drift={...:.4f}". Close with the eval verdict for the 4-level model: "4-level model acceptable (drift < 0.1):" and the boolean. Quality-vs-cost is measured, never assumed.

full-screen editor opens — close anytime to keep reading.

open the editor to run it.