pruning
Deleting parts of a trained model and keeping the rest, with no retraining. On a mixture-of-experts model that means dropping whole experts: Cerebras' REAP method cut Kimi K3 from 896 experts to 179, taking the file from 1.56 TB to 350 GB in two days. Cheap, fast, and it costs you capability that nothing puts back.
Pruning is the least respectful of the shrinking methods and the quickest. Quantization keeps every parameter and stores each one in fewer bits. Distillation trains a new smaller model to behave like the big one. Pruning just removes pieces and ships what is left.
What breaks is uneven. The experts that get cut were specialists, so the loss lands on whatever they specialized in: a pruned model can hold its coding ability and start looping in a language nobody tested.