Cloud, GPUs, Kubernetes, cost: the operational reality
Everything so far runs somewhere, and somewhere sends invoices. You don't need to be an infrastructure engineer; you need the handful of concepts that keep the invoice sane.
GPUs: rent by the hour, pay for idleness
Training and heavy inference want GPUs, and clouds rent them hourly. The editor's arithmetic (deliberately with a placeholder rate — look up current prices, they move) is the entire cost conversation: utilization. A GPU reserved around the clock but used 40 hours a month means paying ~18× the compute you consumed. The standard levers, in the order to reach for them: run training as jobs that acquire and release hardware (chapter 37's scheduled shapes); use interruptible "spot/preemptible" capacity for fault-tolerant work — it's steeply discounted because it can be yanked, which chapter 43's checkpoints make survivable; and batch inference (chapter 46) to keep serving hardware busy or off.
Kubernetes: what it is, when it's yours
Kubernetes (k8s) orchestrates containers across a fleet: you declare "run N replicas of this ch.46 image, restart crashes, scale with load," and it converges reality to the declaration. It's the industry substrate — and the chapter 16 buy-vs-build lesson applies verbatim: one model on modest traffic is happily a managed container service or a single VM; k8s earns its operational overhead when there are many services, teams, and scaling policies to coordinate. Knowing the vocabulary (pod, deployment, service, autoscaling) lets you work with the platform team either way.
The cost review habit
Once a month, three questions against the bill: what's the biggest line item and is it utilized? (idle GPUs and forgotten dev instances are the classic leaks); does spend scale with value (per-prediction cost trending down as traffic grows, or up?); and what single change halves the biggest line — smaller/quantized model (chapter 44), batch instead of realtime (chapter 46), spot for training, caching (chapter 23)? The LLM cost-curve chapter taught the mindset; classical infra rewards it identically.
Where AI specifically gets this wrong
- Architecture astronautics. Ask Cursor to "deploy a model" and receive a k8s manifest constellation for ten requests a minute. Scale of solution should match scale of problem.
- No idle-shutdown story. Generated infra spins resources up; tearing them down is nobody's generated job. The 18× in the editor is this bug, priced.
- Hardcoded pricing "facts." Cloud prices change; any script or doc quoting them needs a date and a source (this lesson practices what it preaches — placeholder plus instruction).