Pointer Walk

Attention

How a model knows what "it" means: every token looks at every other one, which is also why doubling the prompt quadruples the work.

AI and Machine LearningIntermediate11 min read

The word "it"

Read these two sentences.

The animal did not cross the street because it was too tired.

The animal did not cross the street because it was too wide.

In the first, "it" is the animal. In the second, "it" is the street. You worked that out without effort, using the last word and what you know about animals and streets.

A language model starts with each word as a separate list of numbers. The numbers for "it" are the same in both sentences. To understand the sentence, the model has to let "it" gather information from the other words. The mechanism that does this is called attention.

Every word looks at every other word

  1. Each token asks: which other tokens matter to me?

    It compares itself with every token in the text and gives each a score.

  2. The scores are turned into shares that add up to 100%

    A high share means "I am paying a lot of attention to you".

  3. The token updates itself with a blend of the others

    It takes a lot from the tokens it attends to strongly and little from the rest.

After this step, the numbers for "it" are no longer just "it". They carry a strong trace of whichever noun it attended to.

What does "it" refer to?Interactive
Last word
The2%animal62%did2%not1%cross5%the1%street10%because3%itwas2%too2%tired6%

"it" attends most to "animal" (62%). Streets do not get tired, so "it" must be the animal. Only the last word changed, and the heavy line moved to a different noun.

Switch the last word. Nothing else in the sentence changes, and the thick line jumps from "animal" to "street".

Nobody programmed the rule

There is no line of code saying "pronouns look at nouns". The scores come from numbers inside the model that were adjusted during training, by showing it enormous amounts of text and nudging it whenever it predicted the next word badly. Attending to the right noun turned out to help prediction, so the model learned to do it.

The price: everything looks at everything

Each token looks at every token. With n tokens, that is n × n comparisons.

8 tokens, 64 pairsInteractive
Prompt

8 × 8 = 64 pairs. Each square is one token looking at one other token. Double the prompt and the grid is twice as wide and twice as tall: four times the squares.

Example

Double the prompt, four times the work

1,000 tokens means 1,000 × 1,000 = 1,000,000 pairs.

2,000 tokens means 4,000,000 pairs.

10,000 tokens means 100,000,000 pairs. Ten times the text, a hundred times the pairs.

This is one of the main reasons models have a limit on how much text they can take in at once, why long prompts are slower and cost more, and why a great deal of research goes into making attention cheaper.

Why it changed everything

Older models read text one word at a time, passing along a summary as they went. By the end of a long paragraph, the beginning had faded.

With attention, any word can look directly at any other word, however far apart. A name on page one is one step away from a pronoun on page ten. And because every token can do its looking at the same time, the work can be done in parallel on modern hardware. This design is the Transformer, and it is the "T" in GPT.

Key takeaways

  • Attention lets each token gather information from the other tokens in the text.
  • Each token scores every other token, and updates itself with a weighted blend of them.
  • This is how "it" comes to mean the animal in one sentence and the street in another.
  • The patterns are learned from data, not written as rules.
  • The work grows with the square of the length: double the prompt, four times the pairs.

Your progress

Check your progress

0%

  1. Read the page50% of this page

    When you have gone through the page and its animations, tick it off.

  2. Take the quiz50% of this page

    9 questions, from easy to hard. Take it as often as you like: the questions change, and only your best score counts.

Saved on this device only. See all your progress