Code Safari

Chapter 12·Intermediate

Attention Explained: How an LLM Decides What Matters

01 / 06

What attention does

Lets each token focus on the relevant others.

For every token, attention decides how much each other token matters, then blends in their information. It's how the model handles context and long-range relationships.

Attention Explained: How an LLM Decides What Matters | Code Safari