Skip to content
Road to Intelligence

Concept · Chapter 2: The Math Toolkit

Matrix Multiplication

Must knowImplement30 minDifficulty

Multiplying a vector by a matrix transforms it — every output number is a dot product of one matrix row with the input — and that is exactly what a neural-network layer does.

The problem

We need one operation that can take a vector of features and produce a new vector of features — rotating, stretching, mixing and selecting information — with learnable parameters.

The solution

Arrange the parameters in a grid W. Each output entry is the dot product of one row of W with the input, so W·x computes many weighted sums at once.

The consequence

Neural networks become stacks of matrix multiplications with nonlinearities between them, and GPUs — machines built for fast matmul — become the engines of modern AI.

You should understand first

  1. Vectors
  2. Dot Product
  3. Matrix Multiplication

Intuition

A matrix is a machine that takes a vector in and gives a vector out. Feed it the feature vector of a house, and it can produce a new vector whose first entry is "price-relevant signal", whose second entry is "size-to-age ratio signal", and so on — each one a different weighted sum of the inputs.

Geometrically, a matrix moves space around: it can rotate, stretch, shear, or flatten it. Lines stay lines and the origin stays put — that's what "linear" means.

Tiny numeric example

  1. Set up

    W=[1234],x=[56]W = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad x = \begin{bmatrix} 5 \\ 6 \end{bmatrix}

  2. Row 1 · x

    [1,2]⋅[5,6]=5+12=17[1, 2] \cdot [5, 6] = 5 + 12 = 17

  3. Row 2 · x

    [3,4]⋅[5,6]=15+24=39[3, 4] \cdot [5, 6] = 15 + 24 = 39

  4. Result

    Wx=[17,39]Wx = [17, 39] — two dot products, one per row.

The equation

(Wx)i=∑j=1nWij xj(AB)ik=∑jAijBjk(Wx)_i = \sum_{j=1}^{n} W_{ij}\, x_j \qquad\qquad (AB)_{ik} = \sum_{j} A_{ij} B_{jk}

Try it

Try it

Matrix as Transformation

Edit a 2×2 matrix and watch it stretch, rotate, shear or collapse the plane — then add ReLU and see how a neural-network layer folds space.

Know well8 min

Where it appears in AI

  • A layer: y=σ(Wx+b)y = \sigma(Wx + b). WW mixes the input features; the nonlinearity σ\sigma (e.g. ReLU) keeps stacked layers from collapsing into one big matrix.
  • Attention: Q=XWQQ = XW_Q, K=XWKK = XW_K, V=XWVV = XW_V, and the score table QK⊤QK^\top — all matrix products.
  • Batching: put 64 inputs as the rows of XX and one matrix product XWXW processes all of them at once. This is why GPUs are so effective.
Deep diveEigenvectors in one paragraphShould know

Most vectors change direction when a matrix is applied. Eigenvectors are the special directions that only get stretched: Wv=λvWv = \lambda v, where the stretch factor λ\lambda is the eigenvalue. They reveal what a matrix "really does". In ML you'll meet them in PCA (the top eigenvectors of a data covariance matrix are the directions of greatest variance) and in analyses of training stability, where very large eigenvalues of the loss curvature limit how big a learning rate can be. Understanding-level is enough for now.

Why should I care?

As a researcher

Papers describe models almost entirely as matrix products (XW, QKᵀ, W₂φ(W₁x)). Reading them requires tracking what each matrix multiplies and what shapes come out.

As an engineer

Nearly all the compute in training and serving a model is matrix multiplication; its cost is what you pay for in GPU hours, and its shapes decide memory use.

Modern systems that depend on it

  • Every neural-network layer
  • Q/K/V projections in attention
  • Embedding lookups (a one-hot vector times a matrix)
  • GPU and TPU hardware design

Historical context

Before

Linear algebra dates to the 19th century; early ML used it for linear models and PCA, with features chosen by hand.

After

Deep learning stacked learned matrices with nonlinearities, and hardware (GPUs, TPUs, tensor cores) was specialized for matrix multiplication.

Used today

Every forward and backward pass of every neural network. Most of the arithmetic in an LLM is matrix multiplication.

What to remember

  • W·x: each output entry = (row of W) · x.
  • Shapes: [m × n] · [n × p] = [m × p]; the inner dimensions must match.
  • The columns of W are where the basis vectors land.
  • Multiplying matrices = applying one transformation after another.
  • A neural-network layer is σ(Wx + b): a linear transformation, then a nonlinearity.

Watch