3Blue1Brown
Vectors | Chapter 1, Essence of linear algebra
Builds the picture you need for everything else: a vector as an arrow, as a list of numbers, and why both views matter.
Covers: Vectors, vector addition, scaling.
Must know
Lectures & explainers
Watch
A few exceptional videos rather than many mediocre ones — 36 so far, about 18 h 57 min in total. Each is placed where it fits in the curriculum, with a note on why it's worth your time.
Chapter 2
The Math Toolkit
3Blue1Brown
Linear transformations and matrices | Chapter 3, Essence of linear algebra
The single best explanation of why a matrix is a transformation of space — which is exactly what a neural-network layer does.
Covers: Matrices as functions that move vectors; matrix–vector multiplication.
Must know
3Blue1Brown
Dot products and duality | Chapter 9, Essence of linear algebra
Explains why the dot product measures alignment, connecting the arithmetic to the geometry.
Covers: Dot product, projection, the link between dot products and linear maps.
Should know
3Blue1Brown
Matrix multiplication as composition | Chapter 4, Essence of linear algebra
Shows why multiplying matrices means applying one transformation after another — exactly what stacking neural-network layers does.
Covers: Matrix–matrix multiplication as composition of linear maps.
Must know
3Blue1Brown
Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra
The clearest visual intuition for eigenvectors: directions a transformation only stretches.
Covers: Eigenvectors, eigenvalues, why they matter for understanding a matrix.
Should know
3Blue1Brown
Bayes theorem, the geometry of changing beliefs
Turns Bayes from a formula into a picture of restricting your view to the evidence.
Covers: Prior, likelihood, posterior, and the area picture of Bayes theorem.
Must know
3Blue1Brown
The medical test paradox, and redesigning Bayes' rule
Why a 90%-accurate test can still be wrong most of the time — base rates, made unforgettable.
Covers: Base rates, false positives, the odds form of Bayes rule.
Should know
3Blue1Brown
The essence of calculus
Rebuilds the idea of a derivative from scratch, visually, without assuming you remember school calculus.
Covers: What derivatives and integrals are really about.
Must know
3Blue1Brown
Visualizing the chain rule and product rule | Chapter 4, Essence of calculus
The chain rule is the engine of backpropagation; this makes it feel obvious rather than memorized.
Covers: Chain rule and product rule as nudges propagating through functions.
Must know
3Blue1Brown
Gradient descent, how neural networks learn | Deep Learning Chapter 2
Connects the abstract idea of minimizing a function to how a real network learns to recognise digits.
Covers: Cost functions, gradients in many dimensions, gradient descent.
Must know
StatQuest with Josh Starmer
Gradient Descent, Step-by-Step
Works gradient descent out by hand on a tiny regression problem, one step at a time.
Covers: Loss, derivatives, step size, stochastic gradient descent.
Should know
Aurélien Géron
A Short Introduction to Entropy, Cross-Entropy and KL-Divergence
Ten minutes that tie entropy, cross-entropy and KL divergence together — the three quantities behind language-model training.
Covers: Entropy as average surprise, cross-entropy as loss, KL as the gap between them.
Must know
3Blue1Brown
Solving Wordle using information theory
A playful but rigorous introduction to information and entropy as expected bits of surprise.
Covers: Information in bits, entropy, choosing actions that maximize expected information.
Should know
StatQuest with Josh Starmer
Confidence Intervals, Clearly Explained!!!
Short and concrete — what a confidence interval actually tells you, via bootstrapping.
Covers: Bootstrapping, confidence intervals.
Should know
Chapter 3
Machine Learning
StatQuest with Josh Starmer
Machine Learning Fundamentals: Bias and Variance
Seven minutes on the most important trade-off in machine learning.
Covers: Bias, variance, overfitting, underfitting.
Must know
StatQuest with Josh Starmer
Linear Regression, Clearly Explained!!!
Least squares, R², and what a fitted line actually tells you — from the ground up.
Covers: Fitting a line, residuals, R², p-values.
Should know
StatQuest with Josh Starmer
StatQuest: Logistic Regression
A short, clear picture of how logistic regression turns a line into probabilities.
Covers: The S-shaped curve, classification with probabilities, maximum likelihood.
Must know
StatQuest with Josh Starmer
Decision and Classification Trees, Clearly Explained!!!
How a tree chooses its questions — the most interpretable model in this chapter.
Covers: Splits, impurity (Gini), leaves, building a tree.
Should know
StatQuest with Josh Starmer
StatQuest: Random Forests Part 1 - Building, Using and Evaluating
Why many random trees beat one careful tree.
Covers: Bootstrapping, random feature subsets, voting, out-of-bag error.
Should know
StatQuest with Josh Starmer
StatQuest: Principal Component Analysis (PCA), Step-by-Step
PCA built up geometrically, with no linear algebra assumed beyond the basics.
Covers: Principal components, variance explained, projection.
Should know
StatQuest with Josh Starmer
StatQuest: K-means clustering
The k-means loop, animated.
Covers: Assign, update, repeat; choosing k.
Should know
StatQuest with Josh Starmer
ROC and AUC, Clearly Explained!
How to evaluate a classifier across every possible threshold.
Covers: Confusion matrices, true/false positive rates, ROC curves, AUC.
Should know
StatQuest with Josh Starmer
Machine Learning Fundamentals: Cross Validation
Why you never test on your training data, and how to use limited data well.
Covers: Train/test splits, k-fold cross-validation.
Must know
StatQuest with Josh Starmer
Regularization Part 1: Ridge (L2) Regression
Regularization made concrete: accept a little bias to reduce a lot of variance.
Covers: Ridge penalty, λ, bias–variance, fitting with few data points.
Should know
Chapter 4
Neural Networks
Andrej Karpathy
The spelled-out intro to neural networks and backpropagation: building micrograd
Builds automatic differentiation from nothing; afterwards backpropagation stops feeling like magic.
Covers: Derivatives, the chain rule, computational graphs, backprop, a tiny neural network.
Must know
3Blue1Brown
But what is a neural network? | Deep learning chapter 1
The clearest visual introduction to what a neural network actually computes, layer by layer.
Covers: Neurons, weights, biases, layers, activations — all on digit recognition.
Must know
3Blue1Brown
Backpropagation, intuitively | Deep Learning Chapter 3
Builds intuition for how each training example 'nudges' every weight — before any calculus.
Covers: What backpropagation does, and why stochastic gradient descent uses mini-batches.
Must know
3Blue1Brown
Backpropagation calculus | Deep Learning Chapter 4
The chain rule applied to a network, step by step — the formal version of the previous video.
Covers: Derivatives through layers, the chain rule, sensitivity of the cost to each weight.
Should know
Chapter 6
Language Before Transformers
Andrej Karpathy
The spelled-out intro to language modeling: building makemore
Builds a character-level bigram model from counts, then trains a one-layer neural network that learns the same table by gradient descent.
Covers: Language modeling, count tables, smoothing, negative log-likelihood, and swapping counting for a trained network.
Should know
Andrej Karpathy
Building makemore Part 2: MLP
Implements the Bengio et al. 2003 neural language model at character level: an embedding lookup, a hidden layer and a softmax over the next character.
Covers: Learned embeddings, a fixed context window, training splits and why sharing vectors beats exact-count tables.
Should know
Chapter 7
Transformers
3Blue1Brown
Transformers, the tech behind LLMs | Deep Learning Chapter 5
A visual tour of a GPT from input text to next-token probabilities — ideal before or right after Chapter 7.
Covers: Tokens, embeddings, the flow of data through a GPT, softmax and temperature.
Must know
3Blue1Brown
Attention in transformers, step-by-step | Deep Learning Chapter 6
Animates exactly what queries, keys and values do. Watch it alongside the Attention Explorer.
Covers: Queries, keys, values, the attention pattern, masking, multi-head attention.
Must know
3Blue1Brown
How might LLMs store facts | Deep Learning Chapter 7
The feed-forward half of a Transformer block gets less attention than attention; this fixes that.
Covers: MLP sublayers, directions in embedding space, superposition (intuition).
Should know
Andrej Karpathy
Let's build GPT: from scratch, in code, spelled out.
The best way to reach IMPLEMENT level on Transformers: write one yourself, line by line, in PyTorch.
Covers: Self-attention, multi-head attention, masking, residuals, layer norm — all built from a bigram baseline up.
Must know
Chapter 8
The Rise of Large Language Models
Andrej Karpathy
Let's build the GPT Tokenizer
Many odd LLM behaviours trace back to tokenization; this shows you why by building a BPE tokenizer.
Covers: Unicode, bytes, byte-pair encoding, GPT-2/GPT-4 tokenizers, special tokens.
Should know
Andrej Karpathy
[1hr Talk] Intro to Large Language Models
A clear one-hour overview of what LLMs are, how they are trained, and where they're going — good orientation for Part III.
Covers: Pretraining, fine-tuning, scaling, tool use, security issues.
Must know