A. Smyntyna / Code
ENRU

TTFT

Time To First Token: how long you wait after hitting enter before the model produces anything at all. It is dominated by prefill, the model reading your whole prompt, so a long prompt or document makes it worse. On a 27B dense model a large prompt can mean a minute or two of waiting before anything shows up.

TTFT and tokens per second measure the two halves of a request and they can disagree completely. On a measured M4 Max run with the same 4,000-token prompt, a dense 27B waited 16 seconds before writing anything and a 35B A3B waited 2.66. The dense model puts nine times as many parameters to work on every token of the prompt, and the wait tracks that almost directly.

Which number matters depends on the shape of your work rather than on the model. Short question, long answer: the wait is nothing and tokens per second decides. Long document, short answer: the wait is nearly all of it. An agent harness sits at the far end of that range, because it re-sends the entire conversation on every step and pays the full reading cost before every tool call.

On a reasoning model the two clocks come apart. Every tool that reports TTFT stops it at the first token the model generates, and that token is a thinking token, which the interface may hide from you. The wait you actually experience is TTFT plus the whole thinking pass. On the M4 Max run above, the A3B's 2.66 seconds was followed by 27 seconds of thinking before the answer began, and the 122B's 7.6 seconds by 35. Thinking is not extra time bolted on; it is the front slice of the writing phase. It is still time spent looking at nothing.

The one thing that reliably improves it is fewer active parameters, which is the whole argument for a mixture of experts. Buying more memory bandwidth does not help here, because prefill is not waiting on memory.

Read next