speculative decoding
A speed trick that costs no quality. A small draft model guesses the next few tokens, the big model checks all of them in one pass and keeps the ones it agrees with. The output is identical to running the big model alone, and the speedup is typically 1.5 to 2x, entirely determined by how often the small model guesses right.
It works because decode wastes the chip. Generating one token reads every active weight out of memory while the arithmetic units sit mostly idle, so verifying five proposed tokens in that same pass costs almost nothing extra. Accepted tokens are close to free. A rejected one falls back to ordinary one-at-a-time decoding from that point, which is why the acceptance rate is the whole game.
Acceptance depends on the task. Code and structured output run high, because the next token is often forced, and reported speedups reach 2 to 5x. Creative prose runs low and the gain can vanish. One published run pairing a draft model with Qwen3.6-35B-A3B on a single RTX 3090 found no net speedup at all.
In LM Studio it shipped in v0.3.10 and is off by default: you load the main model, then pick a compatible draft model from a sidebar, and it checks that the two share a tokenizer before letting you. Which also means any benchmark you read was run without it unless somebody says otherwise.