prefill
The reading phase. Before a model writes anything it runs your whole prompt through itself, every token of it, filling the KV cache. This is where the wait before the first word comes from, and it is compute-bound: a dense 27B does 27 billion parameters of arithmetic per token you sent it, an A3B does 3 billion.
Prefill runs every token of the prompt at once rather than one after another, which is why it is limited by how much arithmetic the chip can do rather than by how fast it can read memory. Decode is the opposite on both counts.
The practical consequence is that prefill cost scales with the prompt, and nothing else in the pipeline does. Doubling the prompt roughly doubles the wait. Doubling the model's active parameters does the same. An agent harness re-sends its whole conversation on every step, so it pays that bill again on every tool call, which is why the same model can feel instant in a chat window and unusable inside an agent.
One quirk worth knowing: prefill throughput usually goes up on longer prompts, because there is more work to batch and the chip stops idling. The total wait still climbs, just not quite as fast as the token count.