The Math Toolkit

Before you start

How to use this chapter

If mathematics has ever felt like a wall of symbols, this chapter is built for you. It covers six small toolkits, and each one exists because AI needed to solve a specific problem:

Every idea follows the same pattern: intuition → picture → tiny numbers → the equation → what the equation is doing → where it appears in AI → something to try. The sections are independent. Do them in order, or jump in when a later chapter sends you back here.

2.1 · The shape of data

Linear algebra

The problem: a model can only compute with numbers, and it needs to process a lot of them at once.

The answer is to turn everything — a house, a word, an image — into a vector: a list of numbers, pictured as an arrow or a point in space. Once things are points, "similar" can mean "pointing the same way", and the dot product measures exactly that.

The second big idea: a matrix is a machine that transforms vectors. Every output number is a dot product of one row of the matrix with the input, and geometrically the whole plane gets rotated, stretched, sheared or squashed. That is literally what one layer of a neural network does — plus a nonlinearity, which you can toggle below.

2.2 · Reasoning with uncertainty

Probability

The problem: the world is uncertain, and a model that outputs one hard answer can't say how sure it is — or learn from being "a bit wrong".

Modern models output distributions: "72% cat, 25% dog, 3% car"; a probability for each of 50,000 possible next tokens. Softmax turns raw scores into such a distribution.

Conditional probability is the heart of it: how likely is A given B? A language model is nothing but an estimator of $P(\text{next token} \mid \text{everything so far})$. And Bayes' theorem tells you how to reverse a condition — which is where intuition most often fails:

Chain conditional probabilities together and you can score an entire sentence, one token at a time. This identity is the mathematical reason "predict the next token" is a complete recipe for modelling language.

2.3 · Which way is downhill?

Calculus

The problem: a model has millions of parameters and one number — the loss — that says how wrong it is. For each parameter we need to know: if I nudge you, does the loss go up or down, and how much?

That's a derivative. Collect one for every parameter and you have the gradient, an arrow pointing uphill. Shrink the nudge below until the secant becomes the tangent — then use it to step downhill.

A network is a long chain of functions — layer after layer — and the loss sits at the end. The chain rule says the sensitivity of the end to the beginning is the product of the sensitivities along the way. Walk that product backward from the loss and you have backpropagation, the algorithm that trains every neural network (Chapter 4).

2.4 · Learning as walking downhill

Optimization

The problem: "make the model better" isn't an instruction a computer can follow.

First make it precise: a loss function turns every prediction into a number, and learning becomes minimize the average loss. Then minimize it the only way that scales to billions of parameters: compute the gradient, take a small step against it, repeat. That's gradient descent, and nearly everything in modern AI is trained with some version of it.

Three things to notice in the playground. The learning rate is a knife-edge: too small and nothing happens, too large and training explodes. Ravines — steep one way, flat the other — make plain gradient descent zig-zag, which is exactly why momentum and Adam exist. And on the two-valley surface, where you start decides where you end.

2.5 · Not fooling yourself

Statistics

The problem: every number you measure — an accuracy, a benchmark score — comes from a finite sample, so it wobbles. A paper that reports 81.3% vs 80.9% on a small test set may be reporting noise.

Chapter 3 builds on this with training, validation and test splits, overfitting and generalization — the statistics of learning rather than of measurement.

2.6 · Scoring a guess

Information theory

The problem: a model's prediction is a whole probability distribution. How do we score it against the one thing that actually happened?

Claude Shannon's 1948 answer starts with surprise: an outcome you gave probability $p$ surprises you by $-\log p$. Entropy is average surprise — how uncertain a distribution is. Cross-entropy is the surprise of what actually happened, under the model's predictions — and it is the loss every language model is trained to minimize.

Two relatives complete the toolkit. Perplexity is cross-entropy made human-readable: roughly how many tokens the model is choosing between. KL divergence measures how far one distribution is from another; you'll meet it as a penalty in RLHF and as the target in distillation.

Putting it together

One training step, in six tools

Here is a single training step of a language model, told entirely with this chapter:

1. 1

   Linear algebra

   Token vectors flow through stacks of matrix multiplications — shapes [batch × tokens × d_model].
2. 2

   Probability

   Softmax turns the final scores into a distribution over the next token, for every position, via the chain rule of probability.
3. 3

   Information theory

   Cross-entropy scores those distributions against the actual next tokens; its exponential is the perplexity you'll see reported.
4. 4

   Calculus

   The chain rule carries the gradient of that loss back to every one of the parameters.
5. 5

   Optimization

   AdamW nudges each parameter against its gradient, using a random mini-batch.
6. 6

   Statistics

   Validation loss on held-out text — with its sampling noise in mind — tells you whether it's really improving.

That loop, repeated hundreds of thousands of times over trillions of tokens, is how a modern LLM is pretrained. Everything in the rest of this site is detail on top of it.