Chapter 11 · Inside Modern LLMs
Squeeze the Weights
Round a row of weights to 8, 4, 3 or 2 bits, add one outlier, and see why a shared scale fails and per-group scales rescue it.
Try it · toy model
7 min
Read the concepts behind it
Chapter 11 · Inside Modern LLMs
Round a row of weights to 8, 4, 3 or 2 bits, add one outlier, and see why a shared scale fails and per-group scales rescue it.
Try it · toy model
Read the concepts behind it