Why AI Keeps Getting Cheaper (While Everything About It Gets Bigger)

Hold the last three chapters in one hand: training runs cost hundreds of millions, every reply burns metered GPU time, and the buildings are drawing reactor-scale power. Now hold this in the other: the price of using AI has been collapsing — a given level of capability costs orders of magnitude less than it did just a few years ago, one of the fastest price declines of any technology in history.

Both hands are telling the truth. This chapter is about how that's possible — the five forces crushing the cost per token — and the paradox that ensures the industry's total bill grows anyway.

Force 1: The hardware treadmill

The simplest force: GPUs keep improving. Each generation delivers several times more AI math per dollar and per watt than the last — better manufacturing, more tensor-core circuitry, faster memory. A token served on this year's chips simply costs less to produce than the identical token served two years ago. Nobody at the AI lab has to be clever for this force to work; they just have to keep buying.

Force 2: Small models learned the frontier's tricks

The more surprising force. It turns out you don't need a giant model to serve most answers — you need a giant model to discover the capability, and then a small one can learn it:

• Distillation — train a compact "student" model to imitate a huge "teacher" model's outputs. The student never reads the whole internet; it studies the teacher's answers directly, and captures a startling fraction of the quality at a tenth the size or less. A tenth the parameters means a tenth the memory streamed per token — the cost falls almost mechanically.
• Quantization — store each parameter with less numeric precision, like rounding prices to the nearest pound. The model shrinks by half or three-quarters, the memory bottleneck loosens, tokens flow faster — and quality barely moves for most uses.

Force 3: Architectures that don't wake the whole model

A quiet revolution inside the models themselves: mixture-of-experts (MoE). Instead of one monolithic network where every parameter fires for every token, an MoE model is built as many specialist sub-networks — "experts" — with a tiny router choosing a few relevant ones per token. The model can have a huge total parameter count (lots of stored knowledge) while each token only pays for the experts it actually visits — a fraction of the compute per token. Most modern frontier models are believed to use this design; it's a big part of how capability keeps growing faster than serving costs.

Force 4: Serving engineering

You met batching and prompt caching — the bus and the un-re-read prefix. The serving stack keeps finding more:

• Speculative decoding — a tiny, fast model drafts several tokens ahead, and the big model checks the whole draft in one parallel pass (checking is parallel; writing isn't). When the draft is right — which is often, since most words are predictable — the big model effectively wrote several tokens for the price of one.
• Relentless software optimisation — better attention algorithms, better scheduling, better memory management. Unglamorous, compounding, and worth as much as a hardware generation when it stacks up.

Force 5: Competition with nowhere to hide

Every force above is available to every lab — and open-weight models put a hard floor under the market: if a downloadable model is good enough for a task, no one can charge much more than the cost of hosting it. Add several frontier labs price-cutting to win developers, and margins on "good enough" intelligence keep compressing toward the cost of the electricity and hardware underneath. The consumer surplus is enormous; the producers' comfort is not — a tension the final chapter takes head-on.

The paradox that eats the savings

So: tokens get relentlessly cheaper. Here is the fact that reconciles this chapter with the three before it — we immediately spend the savings on more tokens:

• Reasoning models think for thousands of hidden tokens before answering — burning more tokens per question on purpose, because it buys quality.
• Agents work for minutes or hours, calling tools and looping — thousands of tokens where a chat reply used a hundred.
• Contexts ballooned — whole codebases and document sets stuffed into the window, re-read every turn.

This is the Jevons paradox, coined for Victorian coal: make an engine more efficient and coal use rises, because efficiency makes engines worth using everywhere. Cheaper tokens don't shrink AI budgets — they make token-hungry ambitions viable, and total spending climbs. It's precisely why the data-centre wager can coexist with collapsing prices: the industry isn't betting tokens stay expensive; it's betting we'll want unimaginably many of them.

Recap

• Using AI gets cheaper even as building it gets dearer — five forces drive the collapse in cost per token.
• Hardware: each GPU generation delivers multiples more math per dollar and per watt.
• Smaller models: distillation copies frontier quality into models a tenth the size; quantization shrinks the numbers themselves — yesterday's frontier becomes today's "mini" tier.
• Architecture & serving: mixture-of-experts wakes only part of the model per token; speculative decoding, batching, and caching squeeze more tokens from every GPU pass.
• Competition and open-weight models put a floor under prices — and the Jevons paradox ensures the savings are spent on more tokens: reasoning, agents, giant contexts. Cost per token falls; total spend rises.

Which leaves the final question of this guide — with costs collapsing at one end and spending exploding at the other: who actually makes money in AI?