Skip to content
Road to Intelligence
← The Lab

One short sentence, one question: which words should each word pay attention to? Build attention step by step — similarity, softmax, weighted sum, then queries and keys.

Try it · toy model

Attention from Scratch

Know well10 min