grouped-query attention
An attention layout that shares one key and value across several query heads instead of giving every head its own. It shrinks the KV cache by a factor of four to eight with almost no quality cost, which is why a 27B-class model at 32k tokens wants 1 to 4 GB of cache rather than 10.
The cache stores a key and a value for every token the model has seen. In the original layout every attention head kept its own pair, so the cache scaled with the head count. Grouped-query attention gives a group of query heads one shared key and value, and the cache scales with the number of groups instead.
Multi-query attention is the far end of the same idea, one shared pair for every head in the layer. Cheapest cache, most quality lost, which is why most models shipping now sit in the middle.
This is the reason two models with the same parameter count can want very different amounts of room for the same conversation. Parameter count sets the weights. The attention layout sets the cache, and nobody puts it in the model name.