NVlabs/GatedDeltaNet-2

Official Project Page for Deep Delta Learning (https://arxiv.org/abs/2601.00417)

🚀 Efficient implementations for emerging model architectures

Implementation of FlashAttention-2 for Nvidia Tesla V100 / Titan V

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention

An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"

[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (https://arxiv.org/abs/2501.06425)

The official repo of the paper: $delta$-mem: Efficient Online Memory for Large Language Models

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation (NeurIPS 2025)