BF16
Brain floating point, a 16-bit number format Google Brain designed for machine learning. It keeps the same 8-bit exponent as a full 32-bit float, so it reaches the same range of magnitudes, and pays for that by cutting the mantissa to 7 bits. Most open weights ship in BF16 at 2 bytes per parameter, which is why a 27B model is a 54 GB download.
A floating-point number is stored as sign, mantissa and exponent: scientific notation in binary. The exponent sets how big or small the number can get. The mantissa sets how many meaningful digits survive.
FP16 splits its 16 bits as 1 sign, 5 exponent, 10 mantissa. BF16 splits them as 1 sign, 8 exponent, 7 mantissa. Same total size, different bet. Google Brain's bet was that a neural network shrugs off fuzzy digits and dies on values that overflow, so keep the range and spend the mantissa.
In photography terms, the exponent is dynamic range, how many stops you can capture at all. The mantissa is bit depth per stop, how finely each one gets sliced. BF16 keeps every stop and slices them coarser.