feichai0017/orbitkv

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

KV cache store for distributed LLM inference

Persist and reuse KV Cache to speedup your LLM.

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

Unified KV Cache Compression Methods for Auto-Regressive Models

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

Dynamic Memory Management for Serving LLMs without PagedAttention