xcena-dev/maru

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

KV cache store for distributed LLM inference

High Performance KV Cache for LLM Inference

Persist and reuse KV Cache to speedup your LLM.

Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.