Q4_K_M
llama.cpp's most-downloaded 4-bit recipe, and the safe default. Q4 is four bits per weight, K means the K-quant scheme with per-block scales, and M is the medium size within it, where the attention and feed-forward layers that matter most get bumped above four bits. A 27B model lands near 16 GB.
It is not four bits. Counting the block scales and the promoted layers, a Q4_K_M file averages 4.83 bits per weight, so a 27B model comes out at 16.3 GB where a true four bits would be 13.5. That 2.8 GB is what the recipe spends holding on to the layers a flat squeeze would have wrecked.
The letters read left to right as a recipe. Q8_0 is uniform 8-bit with no
per-layer cleverness. Q4_K_S, Q4_K_M and Q4_K_L are the small, medium and
large versions of the same 4-bit scheme, differing in how many layers get
promoted. IQ3_XXS and its neighbours use an importance matrix, measured by
running real text through the model, to decide which weights can afford to be
crushed.
If you are picking blind, take the _K_M build at the bit depth your memory
allows. It is the one everybody else is running, which also means it is the one
that gets tested.