GQA (Grouped-Query Attention)

Attention scheme where several query heads share the same group of Key and Value matrices, reducing the KV-head count compared to MHA. Lowers the cost and memory-access speed during decoding with a minimal quality trade-off.