Relaxed-System-Lab/Flash-Sparse-Attention

Trainable fast and memory-efficient sparse attention

Fast and memory-efficient exact attention

FlashMLA: Efficient Multi-head Latent Attention Kernels

🚀 Efficient implementations for emerging model architectures

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

A collection of memory efficient attention operators implemented in the Triton language.

DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers

Helpful tools and examples for working with flex-attention