flagos-ai/FlagAttention

Fast and memory-efficient exact attention

Helpful tools and examples for working with flex-attention

FlashMLA: Efficient Multi-head Latent Attention Kernels

Trainable fast and memory-efficient sparse attention

🚀 Efficient implementations for emerging model architectures

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

🚀🚀 Efficient implementations of Native Sparse Attention

FlashKDA: high-performance Kimi Delta Attention kernels

TriAttention — Efficient long reasoning with trigonometric KV cache compression. Enables OpenClaw local deployment on memory-constrained GPUs.