alibaba/tair-kvcache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

KV cache store for distributed LLM inference

Distributed KV cache scheduling & offloading libraries

High Performance KV Cache for LLM Inference

Persist and reuse KV Cache to speedup your LLM.

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

Unified KV Cache Compression Methods for Auto-Regressive Models