ModelEngine-Group/unified-cache-management

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

KV cache store for distributed LLM inference

High Performance KV Cache for LLM Inference

Unified KV Cache Compression Methods for Auto-Regressive Models

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response

[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.