I think this might be the first KV quantisation that will work well...

华为在 GitHub 开源 KVarN 把大模型推理中最大的内存开销直接压缩。 同样的模型，内存少用一半，账单少付一半 ，果汁分你一半

The weird part about KV cache quantization failing on reasoning isn't angle distortion, it's just magnitude error. Huawei's KVarN fixes this with a Hadamard rotation and dual-axis variance…

🤖 KVarN ⭐ 115 stars Get 3-5x more context for your LLM agents with throughput that beats FP16. No calibration, lossless accuracy, and enabled with a single flag. 🔗 🌐 #AI #MachineLearning

KVarN ⚡️ Built for agentic and long-context workloads. 💡 KVarN delivers 3-5x more KV-cache capacity and up to ~1.3x the throughput of FP16, so you fit far longer contexts and serve more concurrent…

this actually has the potential to be quite impactful for local models (though currently only available in vLLM). KV growth can be a major limit factor for long ctx use (e.g., ~8GB of memory at 8/8…

Huawei released KVarN, a KV-cache compression method for LLMs It delivers 3-5x more context length, beats FP16 throughput, and keeps FP16 accuracy. One flag in vLLM. Zero calibration.