microsoft/MInference

MoBA: Mixture of Block Attention for Long-Context LLMs

Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

Unified KV Cache Compression Methods for Auto-Regressive Models

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration