microsoft/vattention

Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Fast and memory-efficient exact attention

FlashInfer: Kernel Library for LLM Serving

LLM inference in C/C++

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

CUDA kernels for linear attention variants, written in CuTe DSL and CUTLASS C++.

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

High Performance KV Cache for LLM Inference