Pointer Walk

Eight AI Words, Explained

Token, context window, embedding, temperature, top-p, fine-tuning, RAG and hallucination, each shown happening.

AI and Machine LearningBeginner8 min read

Eight words that come up everywhere

Read anything about AI and the same handful of terms appears, usually without explanation. Here are eight of them. Each one has a plain definition, an example, and where possible an animation of the thing actually happening. Each links to a full page if you want more.

1. Token

A piece of text, as the model sees it. A model does not read letters or words. Text is cut into pieces from a fixed list, and each piece is replaced by a number.

Common words are one token. Rare words are several. In English a token is about three quarters of a word.

Building tokens by merging pairsInteractive

Training text

thecatsatonthematandthecatatetherat

Merges learned, in order

none yet

Start. Start with the sentence split into single characters: 35 tokens. A model with only these would need a great many tokens for every text.

Tokens in this text
35
Characters
35
Merges learned
0

Why it matters: limits and prices are counted in tokens. Full page: Tokenization.

2. Context window

The most text a model can be given at one time, measured in tokens. Your instructions, the whole conversation, any documents and the reply all have to fit inside it.

A model keeps nothing between requests. What looks like memory is the app sending the entire conversation again with every message.

Two messages in one chatInteractive
You
Chat app
saved: nothing
Model
remembers: nothing

Watch what is actually sent to the model each time.

Why it matters: when a chat gets too long, the oldest parts are dropped and the model can no longer see them. Full page: Context Windows and Memory.

3. Embedding

A list of numbers that stands for the meaning of a piece of text. Texts that mean similar things get similar numbers, so "similar meaning" becomes "nearby position", which a computer can measure.

Search by meaning: the nearest arrow winsInteractive
123456Q

QHow do I reset my password?

1Steps to recover access to your account

2Forgot your login? Start here

3How to choose a strong password

4Reset your router to factory settings

5Our refund policy

6Pasta for two in 20 minutes

Start. Every text has been turned into an arrow. Texts about similar things point in similar directions. The pink arrow is the question.

Why it matters: it is how search finds the right document even when no words match. Full page: Embeddings and Semantic Search.

4. Temperature

A setting that controls how adventurous the model's word choices are. At each step the model has a chance for every possible next token. Low temperature makes the favourite win almost every time. High temperature gives unlikely words a real chance.

"The sky is ___" at temperature 0.7Interactive
Temperature
78.4%
blue
14.1%
clear
6.0%
grey
1.1%
falling
0.3%
green
0.1%
cheese

What was picked, run by run

Start. Temperature 0.7. The bars are the chance of each word coming next. Now run the same prompt 16 times and see what gets picked.

Runs
0 / 16
Different words picked
0

Why it matters: low for answers that should be consistent, higher for variety. Full page: Temperature and Sampling.

5. Top-p

A second setting for the same choice, which cuts off the unlikely end. Sort the possible next tokens from most to least likely. Keep adding them until their chances total p. Throw the rest away.

Example

Top-p of 0.9

The next word could be "blue" (65%), "clear" (20%), "grey" (11%), or one of thousands of unlikely words sharing the last 4%.

The first three add up to 96%, which passes 90%. Only those three can be picked. "cheese" is impossible, however many times you ask.

Why it matters: it stops one bizarre word from derailing a long answer. Temperature reshapes the chances. Top-p removes the tail.

6. Hallucination

A confident, fluent answer that is wrong or made up. A model generates text that is likely given its training. When it lacks the facts, the same process still produces a plausible-sounding reply. Nothing inside it flags the difference.

The model on its ownInteractive
You
App
Your documents
Model

A question whose answer is in your documents, not in the model.

Why it matters: fluency is not evidence. Check anything important, and prefer setups where the model answers from documents it has been given.

7. RAG

Retrieval-augmented generation: look things up first, then answer. Before the model replies, the app searches your documents for relevant passages and puts them in the prompt. The model answers from what it has just read.

With retrieval (RAG)Interactive
You
App
Your documents
Model

A question whose answer is in your documents, not in the model.

Why it matters: it is the standard way to make a model answer correctly about your own, current information. Full page: RAG or Fine-Tuning.

8. Fine-tuning

Training an existing model further on your own examples. It changes the model itself, so that it reliably writes in a certain style, follows a certain format, or performs a narrow task.

It teaches behaviour, not facts. For facts, use RAG.

RAG or fine-tuning?Interactive

What is the model getting wrong?

All eight in one table

WordIn one line
TokenA piece of text, the unit a model reads and writes
Context windowThe most text a model can take in at once
EmbeddingNumbers that place a text's meaning in space
TemperatureHow much chance unlikely words are given
Top-pA cut-off that removes the least likely words
HallucinationA confident answer that is not true
RAGFetching relevant text and giving it to the model before it answers
Fine-tuningFurther training to change how a model behaves

Key takeaways

  • Tokens are the pieces a model reads. The context window is how many it can read at once.
  • Embeddings turn meaning into positions, so similar texts can be found.
  • Temperature and top-p control how the next token is picked.
  • A hallucination is likely-sounding text that is not true.
  • RAG supplies facts at answer time. Fine-tuning changes behaviour through training.

Your progress

Check your progress

0%

  1. Read the page50% of this page

    When you have gone through the page and its animations, tick it off.

  2. Take the quiz50% of this page

    7 questions, from easy to hard. Take it as often as you like: the questions change, and only your best score counts.

Saved on this device only. See all your progress