Expedition 01·Beginner·12 min read
What 'Generative' Actually Means: Next-Token Prediction, Explained Simply
Generative AI sounds mysterious, but underneath it is one repeated trick: predicting the next token. Here's what that means, why it produces such convincing text, and what it tells you about how these models really work.
June 10, 2026
Generative AI gets described in ways that make it sound like magic — or like a digital brain. It is neither. Underneath ChatGPT, Claude, Gemini, and every other large language model is a single, almost embarrassingly simple idea, repeated very fast and at enormous scale.
This chapter is about that one idea. Once you have it, everything else in this guide — training, hallucinations, context windows, the strange failures — stops being mysterious and starts being predictable.
The whole thing in one sentence
A generative language model is trained to do one job:
Given a stretch of text, predict what comes next.
That's the entire trick. Not "understand the question," not "look up the answer," not "reason about the world." Just: here is some text, what is the most likely next piece?
When you ask a model a question, it isn't reaching into a database. It is treating your question as the beginning of a document and continuing it — one small piece at a time — in the way its training says such a document is most likely to continue.
"Generative" vs. the AI that came before
The word generative is doing real work. It separates today's models from the machine learning that dominated the previous decade.
Older systems were mostly discriminative: you gave them an input and they sorted it into a category. Is this email spam? Is this photo a cat or a dog? Is this transaction fraud? The model drew boundaries between buckets.
A generative model does something different in kind. Instead of choosing a label, it produces new content — a sentence, a paragraph, a function, an image — that did not exist before.
| Discriminative AI (older) | Generative AI (today) | |
|---|---|---|
| Core question | "Which bucket?" | "What comes next?" |
| Output | A label or score | New text, code, or images |
| Example | Spam filter, photo tagger | ChatGPT, Claude, image generators |
| Feels like | Sorting | Writing |
That shift — from sorting to continuing — is why this generation of AI can hold a conversation, draft an email, or explain a concept, while the previous generation mostly put things in boxes.
What is a "token"?
The models don't actually work word by word. They work in tokens — chunks of text that are often a whole word, but sometimes a piece of one. "Cat" might be one token; "unbelievable" might split into "un", "believ", and "able".
For this chapter you can read "token" as "roughly a word." We'll dig into why tokens matter — and why they cause some of AI's weirdest behaviour — in Tokens, context windows, and why AI forgets. For now, just know the model reads and writes in these small pieces.
Predicting the next token, concretely
Take that unfinished sentence: "The cat sat on the ___".
The model doesn't pick a next word out of thin air. It assigns a probability to every token it knows — its full vocabulary, often 50,000 to 200,000 tokens — and ranks them. Some are wildly likely. Most are near zero.
"Mat" wins because, across the model's training, that pattern showed up constantly. "Idea" loses because "the cat sat on the idea" almost never appears in real text. The model has no notion of cats or sitting — it has a finely tuned sense of which tokens tend to follow which.
What we call fluency is just this ranking being very, very good.
From one guess to a whole paragraph
A single prediction only gives you one token. So how do you get a full answer? You loop.
The model predicts one token, sticks it onto the end of the text, and then runs the whole thing through again to predict the next token. It repeats this until it produces a special "stop" token or hits a length limit. This is called autoregressive generation — "auto" (self) plus "regressive" (feeding its own output back in).
This is why a model "writes" the way it does — left to right, never able to go back and edit what it already said, building each sentence one committed step at a time. It is also why longer answers take longer: each token is a separate trip through the model.
Why doesn't it say the same thing every time?
If the model always picked the single highest-probability token, it would be repetitive and robotic — and the same prompt would always give an identical answer. To make output feel natural, the model samples from the top candidates instead of always taking number one.
A setting called temperature controls how adventurous that sampling is.
| Temperature | Behaviour | Good for |
|---|---|---|
| Low (≈0.2) | Almost always the top token; focused, repetitive | Facts, code, precise tasks |
| Medium (≈0.7) | Balanced; natural-sounding variety | General conversation |
| High (≈1.2) | Picks less likely tokens; creative, riskier | Brainstorming, fiction |
This is the real reason "regenerate" gives you a different answer. Nothing changed about the model or your question — it simply rolled the dice again among the plausible next tokens.
So why does it feel intelligent?
Here's the part that surprises people. If the model is "just" predicting the next token, why can it explain quantum physics, write working code, or translate between languages?
Because predicting text well is secretly very hard. To reliably finish the sentence "The capital of France is ___", you effectively have to know the answer is Paris. To continue a paragraph of logical argument convincingly, you have to track the argument. To complete a function, you have to follow the code's intent.
Across trillions of words of training, "guess the next token" quietly forces the model to absorb grammar, facts, writing styles, and the patterns of reasoning — because all of those are needed to make good guesses. Intelligence-like behaviour falls out of relentlessly optimising one simple objective.
What this immediately explains
Holding "it predicts the next token" in your head, several of AI's quirks suddenly make sense:
- It can be confidently wrong. The model is optimised to produce likely-sounding text, not true text. A fluent, plausible-but-false continuation scores well on its actual objective. (More on this in Why AI hallucinates.)
- It's not looking anything up. There is no database of facts being queried. There are only patterns baked into the model's weights — which we'll open up in What's inside a model.
- It has no memory between separate chats unless the text is fed back in. Each prediction only sees the text in front of it.
What it is — and isn't
Let's be precise, because the mental model matters:
- A generative model is a probability engine for "what token comes next," trained on a vast amount of human text.
- It is not a fact database, a search engine, or a reasoning system with a guaranteed answer. It approximates those things by being extremely good at continuation.
Keep that distinction and you'll predict the technology's behaviour far better than most people — including when to trust it and when to check its work.
Recap
- Generative AI does one thing: predict the next token, over and over.
- It differs from older AI by creating new output rather than sorting inputs.
- Each step is a ranked probability over the whole vocabulary; fluency is good ranking.
- Full answers come from an autoregressive loop — predict, append, repeat.
- Temperature adds controlled randomness, which is why answers vary.
- Useful behaviour emerges from this simple objective, but the model optimises for sounding right, not being right.
Next, we'll see how this one idea went from a research paper to a technology that reshaped an industry in just a few years — From GPT to today: a short history of the LLM era.