promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_promptdojo_

Tokenizers and context budget — step 7 of 7

ch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internalsch 44 · cnns, transformers, and useful llm internals7/7
Checkpoint

One last thing before we move on. pass this to mark the lesson done, or skip and keep moving. hop to the next when you're ready.

Build the context-budget ledger. Walk the four line items, printing f"{name:<18} {cost:>3} (running total {used})" for each, then f"remaining for the answer: {remaining}" against the 100-token budget. Finally, try to admit chunk B (15 tokens): since it exceeds what's left, print f"cannot afford {name} ({cost} > {remaining}) - drop or compact first". The window is a shared budget — spend it deliberately.

full-screen editor opens — close anytime to keep reading.

open the editor to run it.