ovg-project/kvcached

Dynamic Memory Management for Serving LLMs without PagedAttention

High Performance KV Cache for LLM Inference

Persist and reuse KV Cache to speedup your LLM.

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

LLM inference in C/C++