microsoft/RetrievalAttention

Sparse Decoupled Attention for Efficient Long-Context LLM Inference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

A sparse-first inference engine (SparseEngine).

Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

[ICML 26 Spotlight, NeurIPS 25 @ ER] Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference

Accelerating Long Context LLM Inference with Accuracy-Preserving Context Optimization in SGLang, vLLM, llama.cpp, OpenClaw, RAG, and Agentic AI.