Your map of the course

What are the boxes inside the machine?

A RAG system retrieves the right passage and the model answers from its training data anyway. An agent with five tools calls the wrong one. A model writes a working parser and then says “strawberry” has two r’s. Each of these failures starts in one of about ten boxes inside and around the model, and once you know what each box does, you can usually say which one to look at first.

Drawn as boxes, the machine starts with a box for text and an arrow to tokens. Then come numbers, a network, a mechanism called attention, a stack of blocks, a loop for training and a loop for generating. Off to the right sit three more boxes: RAG, fine-tuning and agents.

The strawberry miscount starts in the tokenizer, which hands the model a few multi-letter pieces and never the letters themselves. The ignored passage is a question about attention, and about what training taught the model to trust. The wrong tool call usually starts in the agent box: tool descriptions reach the model as ordinary tokens in the prompt, and the model picks a tool the way it picks any next token. Each box answers one question about the machine, and the map below takes them in the order they are built.

How people get lost learning LLMs

Pause and predict There are two common ways to start: study a semester of maths first, or memorise the vocabulary (“attention”, “RAG”) first. Which of the two leaves people lost?

One picture, three layers

That is a lot of boxes. They fall into three layers, plus a final engineering box that builds on all of them. Keep the three layers in your head and the rest is detail.

Pause and predict Where on the map does RAG, the box that looks things up in documents, belong?

What real models add

A tiny GPT and a frontier model share these boxes. The large models also differ from it in ways worth knowing:

• Why LLMs know things, Modern LLM architecture and From raw text to assistant: where knowledge lives, what changed inside the block, and how a text predictor becomes an assistant.
• Small models from big ones: how a small model learns from a much larger one.
• Alignment and safety: how you teach a model what it should not do, and how you check that it learned.
• Models that see and hear: how a model that only reads tokens looks at a photo.
• Looking inside the model: what we can and cannot read from the numbers.
• Reasoning models: what changes when a model is allowed to think before answering.

How each lesson works, and how to use it

Every lesson moves through the same five stages, in the same order.

The order is the teaching method: why before how, how before maths, maths before code. You learn each idea by building it.

Conceptual lessons drop the numbers, maths or code sections when there is honestly nothing to put in them. The experiment, the exercises and the recall are never dropped.

What you should remember

1. Three layers: text into numbers, the model (how it works, how it is trained, how it is run), and what you build around it.
2. Every box on the map exists to answer a question. Always know which question you are on.
3. Each lesson moves through five stages: Story → Idea → See it → Practice → Own it. Why comes before how, and maths before code.
4. When a model misbehaves, ask which box the failure starts in: the tokenizer, the model, its training, how it is run, or what was built around it.
5. The repository’s Python files are the ground truth. Python 3 + NumPy to start; PyTorch from Build GPT onward.

Where this appears in a real LLM

Is this map the real thing, or a teaching simplification? Mostly the real thing, at a very different scale.

In this lesson

• You build every box yourself, in a few hundred lines each
• Models with thousands to a few million parameters
• Trains in minutes on a laptop CPU (or, for the tiniest GPT, in your browser)
• Writes Shakespeare-flavoured gibberish

In a real system

• The same boxes, each owned by a specialist team
• Billions to trillions of parameters
• Trains for weeks to months on thousands of GPUs
• Extra stages we only describe: instruction tuning, preference tuning, safety work, serving infrastructure

Every box is a question. The boxes need a small amount of school maths first, starting with vectors and the dot product.