router
The small network inside a mixture of experts that decides, for every token, which few experts run. It is what makes a 35B model do the work of a 3B one: 35 billion parameters sit in memory and the router picks about 3 billion of them each time. Kimi K3 routes to 16 experts out of 896.
The router runs before the experts do, scores them, and takes the top few. It picks a different set for the next token, which is why every expert has to stay resident in memory even though almost none of them run at any moment.
That is the whole shape of the trade. You pay for the full parameter count in RAM and you pay for the active count in time.