quantization
Storing each weight in fewer bits to shrink the model. Full precision is 16 bits per weight; 4-bit quantization uses 4, cutting the file to about a quarter of its size. A 27B model drops from 54 GB at 16-bit to around 16 GB at 4-bit, with a small, usually acceptable, loss in quality.
4 bits means each weight is snapped to one of 16 possible values. Weights cluster between −1 and +1, so those 16 values land 2 ÷ 15 = 0.133 apart, and every weight in the file rounds to the nearest one.
It is easier to see than to argue about. Drag the depth down and the sky bands, because a smooth gradient has nowhere to hide when it only has a few values to be made of. Weights behave the same way, and the error shows up as a model that is slightly worse at everything rather than broken at one thing.
crushing the photo · drag to stop
2-bit · 4 levels
all 4 ticks drawn
Mixed costs 12.3 GB against 9.3 GB for uniform 2-bit, and 29.5 GB for uniform 8-bit.
Same budget, two ways to spend it
For the same file size, mixed precision keeps the sun and the ridge crisp and spends the banding on sky nobody was reading. Which weights count as the sun is the whole question, and a recipe name is somebody else's answer to it.
Uniform 4-bit at 16.0 GB, against mixed precision with a 3-bit background at 15.2 GB. Which photo survives?
Good recipes keep the roughly 15% of load-bearing weights at higher precision and push the rest down, which is what the mixed-precision toggle above does. Below about 3 bits the losses stop being subtle, unless the recipe is doing something clever about which weights get crushed.