Attention and Transformer blocks (step 7/7) · cnns, transformers, and useful llm internals

Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

Checkpoint. Compute one attention output end to end. Write attend(values, scores) that normalizes the scores into weights (score / sum) and returns the weighted average of the values, rounded to 2 decimals, formatted as:

attention output: 22.5

Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

attention output: 22.5

full-screen editor opens — close anytime to keep reading.

Attention and Transformer blocks — step 7 of 7

Attention and Transformer blocks — step 7 of 7