Nudges: derivatives and the chain rule

How do you find out which way to turn a billion knobs?

GPT-2 small has 124 million adjustable numbers, its parameters. When training starts they are random, and the loss on the first batch of text is about 10.8, which is what you get by spreading belief evenly over all 50,257 tokens. A fully trained GPT-2 small brings the loss below 3. Every training step between those two numbers has to answer the same question for each of the 124 million parameters: if I turn this one up a hair, does the loss go up or down, and how strongly?

Think of each parameter as a knob, with one meter beside all of them: the loss, the “surprise” number from Softmax. Lower is better, and finding good settings for the knobs is what training means.

Pause and predict Suppose you have no formula for the loss, only the ability to run the model and read the loss. How could you find out, for one knob, whether turning it up helps or hurts, and by how much?

A mental model

This lesson uses no limits, integrals or trigonometry. Everything comes from nudging an input by a small amount and watching what the output does.

Here is what one such measurement looks like. Say you raise one weight by 0.01 and the loss rises by 0.02. The sensitivity of the loss to that weight is 0.02 ÷ 0.01 = 2. That one number tells you two things: the direction (raising this weight makes the loss worse, so lowering it should help) and the strength (the loss moved twice as far as your nudge).

The reading holds only close to where you took it. Raise the same weight by a further 0.01 and the loss may move by 0.015 this time, or by 0.03, because a function’s sensitivity changes as you move along it. So it has to be measured again after every move.

Pause and predict f(x) = x² at x = 3. Nudge x by 0.001. About how far does the output move?

Derivative

The answer to “if I nudge the input a tiny bit, how much does the output move, per unit of nudge?”. An amplification factor with a sign.

For example: For f(x) = x² at x = 3: nudge x by 0.001 and the output moves by about 0.006. The derivative there is 6.

You can always find a derivative by experiment: nudge the input, measure the output again, and divide the change in the output by the size of the nudge.

Many knobs: the gradient

A model has many inputs we can adjust. Nothing new is needed: do the nudge experiment for one knob at a time, holding the others still.

Gradient

One derivative per knob, collected in a list. For each knob: “if I turn you up a hair, does the error go up or down, and how fast?”

For example: f(a, b) = a × b at a = 3, b = 4. If you nudge a, the output moves 4 times as far as the nudge. If you nudge b, it moves 3 times as far. So the gradient is [4, 3].

Nudges through a pipeline: the chain rule

A neural network is a pipeline: layer feeds layer feeds layer. So how does a nudge at the start arrive at the end?

Start with a pipeline you already know: changing money from one currency to another, and then to a third.

Pause and predict Dollars to euros at 0.9, then euros to yen at 160. How many extra yen does one extra dollar give?

That is the chain rule: every stage has a local amplification, and the end-to-end amplification is their product. You will try it below.

Let's see the numbers

The nudge experiment on f(x) = x² at x = 3, with h = 0.001:

And with smaller and smaller nudges:

It is heading for 6. At x = 3, this function amplifies tiny nudges 6 times.

The pipeline at x = 2

Pause and predict Square, add 1, times 3, at x = 2. What is the end-to-end amplification?

You can check this by treating the whole pipeline as a black box and nudging it. f(2) = 15 and f(2.001) = 15.012003, so the output moved by 0.012003. Divided by the nudge of 0.001, that gives 12.003, which agrees with the chain rule’s 12.

Backpropagation, the algorithm that trains every neural network, is built on this multiplication. A 50-layer network is a 50-stage pipeline, walked from the output back to the input, multiplying local amplifications as it goes. (When a value feeds several later stages, the effects along the separate routes add up.) You will build it in Backpropagation, and the “hard part” will be a multiplication you already understand.

The math

Formulas for derivatives are shortcuts that save you the experiment. These five are the only ones this course ever uses, and the repository verifies each one by nudging:

What you should remember

1. A derivative is a measured sensitivity: nudge the input, see how far the output moves, and divide by the size of the nudge. Its sign tells you the direction, and its size tells you the strength.
2. It is local: it is valid only for tiny nudges around where you stand, so after every move you must measure it again. A gradient is one derivative per knob.
3. The chain rule: in a pipeline, local amplifications multiply. At x = 2, square → +1 → ×3 gives 4 × 1 × 3 = 12. Backpropagation is built on that multiplication, and the nudge experiment is its unit test.

Where this appears in a real LLM

In this lesson

• One input x, three stages
• Derivative measured by nudging: two function calls
• Amplifications are single numbers

In a real system

• Billions of parameters, hundreds of stages (every matrix multiply, softmax and activation is a stage)
• Gradient computed by backpropagation: one forward pass plus one backward pass, for all parameters at once
• Amplifications are matrices (the proper name is Jacobian), and “multiply” is a matrix multiply (this is where the transpose from Matrices shows up)

In frameworks like PyTorch you never write the backward sweep yourself. Every operation records what it needs to work out its local amplification as it runs, and loss.backward() multiplies them together in reverse. This is called automatic differentiation. In Backpropagation you will write it by hand once, in NumPy, and check it with the nudge experiment, so that .backward() is never magic again.

You now have all four tools:

• dot products to compare,
• matrix multiplies to do it in bulk,
• softmax to turn scores into probabilities and a loss,
• and derivatives to find out which way to turn the knobs.

Next, Gradient descent puts the last two together and watches a program learn, by small turns of every knob with a fresh measurement after each.