A. Smyntyna / Code
ENRU

dynamic range

The distance between the largest and smallest value a format can hold. In a photograph it is the number of stops between the brightest highlight and the darkest shadow that still carry detail. In a number format it is set by the exponent bits, which is the whole reason BF16 exists.

Range and precision are separate things, and quantization spends them separately. A format can cover an enormous range in coarse steps, or a narrow range in fine ones.

That distinction is why block formats like MXFP4 work at all. They give a block of weights one shared scale factor, which restores range, and then spend the remaining bits on precision inside that block.

Read next