lucidrains/halonet-pytorch

Helpful tools and examples for working with flex-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Code for ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Fast and memory-efficient exact attention

Fast and memory-efficient exact attention

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention

A concise but complete full-attention transformer with a set of promising experimental features from various papers