Chapter 11 · Inside Modern LLMs

Draft and Verify

Run speculative decoding's accept-or-resample rule thousands of times and watch the output keep the large model's distribution, then trade acceptance rate, draft length and draft cost for speed.