taco-project/FlexKV

KV cache store for distributed LLM inference

High Performance KV Cache for LLM Inference

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Persist and reuse KV Cache to speedup your LLM.

Unified KV Cache Compression Methods for Auto-Regressive Models

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training