MoE
Mixture of Experts. The model is split into many small sub-networks (experts), and a router picks only a few to run for each token. A 35B model with 3B active per token stores all 35B in memory but does the math of a 3B model, so it answers much faster than a dense model of the same total size.
The trade is memory for speed: you still need RAM for every expert, but decode speed tracks the active slice, not the full parameter count. Qwen3.6-35B-A3B keeps 35B in memory and activates roughly 3B per token.
Kimi K3 is the same idea at the far end of it: 896 experts, 16 of them running per token. 2.8 trillion parameters held, 104 billion of them read to produce one word.