WUSH-KV: KV cache quantization with data-adaptive transforms
ResearchSource:arXiv
A new method, WUSH-KV, cuts the memory and bandwidth the KV cache takes up in long-context Inference by quantizing it at low precision with data-calibrated transforms. At 2 bits, the approach matches or beats the OSCAR transform across all tested models and tasks. It's a sign that memory optimization is no longer only about the weights, but also the context memory.
Related articles
KV cache: why your RAM needs more than just the model
What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.
How to Read a Model's Spec Sheet: Family, Size, Context
Family, parameters, context, quantization, benchmarks and license: how to decode an AI model's spec sheet and choose well.
Quantization: Get More From the Same Hardware
What quantization is, the Q4/Q5/Q8 levels, and how to pick the right one for your VRAM, with a concrete example on Qwen2.5 14B.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.