A. Smyntyna / Code
ENRU

reasoning model

A model trained to write out its working before it answers. The thinking pass is generated text like any other, so it costs time and tokens, and most interfaces hide it. On a measured M4 Max run a 35B A3B reached its first token in 2.66 seconds and then thought for 27 more.

The thinking is not a separate mode the model enters. It is the front slice of the writing phase, produced one token at a time at the same tokens-per-second the answer runs at, and paid for at the same rate.

Which breaks every timing number that stops at the first token. TTFT stops there, and on a reasoning model that token is a thinking token nobody shows you, so the reported wait can be a tenth of the wait you sit through.

It also changes what a long answer costs. A model that thinks for two minutes and a model that reads your prompt for two minutes feel identical from the outside and are fixed by opposite things: fewer active parameters helps the second, and only a shorter thinking budget helps the first.

Read next