bytedance/InfiniStore

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Persist and reuse KV Cache to speedup your LLM.

High Performance KV Cache for LLM Inference

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

Unified KV Cache Compression Methods for Auto-Regressive Models

Disaggregated serving system for Large Language Models (LLMs).

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Achieve state of the art inference performance with modern accelerators on Kubernetes