K-quant
llama.cpp's family of mixed-precision recipes, the ones with a K in the name. Weights are stored in blocks that share a scale factor, and a fixed rule promotes certain layer families above the nominal bit depth. The trailing S, M or L says how much promoting the recipe does.
Read a name like Q4_K_M right to left. M is medium, the amount of
promotion. K is the block-scale scheme. Q4 is what most of the file is
stored at.
On a 27B model the S and the M differ by about 0.9 GB, which buys back the
attention value projections, the feed-forward down-projection in the early
blocks, and the embeddings. That is a small enough premium that almost nobody
takes the S, and it is why _K_M is the default nearly everyone is running.