Chapter 12·Intermediate
Attention Explained: How an LLM Decides What Matters
01 / 06
What attention does
Lets each token focus on the relevant others.
For every token, attention decides how much each other token matters, then blends in their information. It's how the model handles context and long-range relationships.