multi-token prediction
Speculative decoding with the draft model built in. The model is trained with an extra head that predicts the token after next, so it can propose its own continuations and verify them in one pass, with no second model to load or keep in memory. DeepSeek V3 popularized it.
The mechanism is the same as speculative decoding and so is the guarantee: the output matches what the model would have produced token by token. The difference is where the guess comes from. An extra prediction head trained alongside the main one drafts for free, which removes the two awkward parts of the classic setup, finding a small model that shares a tokenizer and paying memory for it.
DeepSeek reported roughly an 80 to 90% acceptance rate on the second token and about a 1.8x generation speedup. That is one model's number and does not generalize. Implementations vary widely in how much they actually deliver, and some ship it correctness-gated and report barely any gain at all.
It is part of why a model can feel quicker than its parameter count and its tokens-per-second figure suggest it should.