tokens per second
How fast a model writes, and the number every benchmark leads with. Estimate it as memory bandwidth divided by the bytes of active parameters: 3 billion active params at 8 bits is 3 GB per token, so an M4 Max at 546 GB/s tops out near 180. Real measurements land at half to three quarters of that.
Roughly 7 tokens a second is reading pace for a person. Anything above about 20 feels immediate in a chat window. Below 10 you watch it type.
The number describes decode only, so it says nothing about the wait before the first word. For a short question that is most of the story. For a long prompt it is a minority of the total time, and for an agent re-sending 60,000 tokens of context every turn it is close to irrelevant. Two models can differ by 5x in tokens per second and by 6x in the other direction on how long you sit there.
It is also not portable between setups. Quantization, context length, runtime, whether the cache was warm, and thermal throttling on a laptop under sustained load all move it. A tok/sec figure is worth something when the same machine ran both sides of the comparison, and worth very little otherwise.