thu-nics/AKV

Persist and reuse KV Cache to speedup your LLM.

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.

Unified multi-layer caching library for AI/agent pipelines — LangChain, LangGraph, AutoGen, CrewAI, Agno, A2A

High Performance KV Cache for LLM Inference

Distributed KV cache scheduling & offloading libraries

Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.