mjun0812/flash-attention-prebuild-wheels

Fast and memory-efficient exact attention

FlashKDA: high-performance Kimi Delta Attention kernels

Helpful tools and examples for working with flex-attention

FlashMLA: Efficient Multi-head Latent Attention Kernels

🚀 Efficient implementations for emerging model architectures

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

Hackable and optimized Transformers building blocks, supporting a composable construction.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Enabling PyTorch on XLA Devices (e.g. Google TPU)