This serving-cost estimator recomputes keys and values for the ENTIRE sequence on every generated token — 1,055 K/V computations to generate 10 tokens after a 100-token prompt. But the prefix's K/V never change: compute them once, cache them, and each new step adds only its own. Fix the estimator to model the cache.
The break is on line 5 — but read the whole snippet first.