SJTU-Storage-Lab/CacheSlide

Unified KV Cache Compression Methods for Auto-Regressive Models

Code for the paper "Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning" https://arxiv.org/abs/2609.03430

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

🏆 OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond — redefining the accuracy-efficiency Pareto front for X-LLMs KV quantization.

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Persist and reuse KV Cache to speedup your LLM.

High Performance KV Cache for LLM Inference

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.