4rtemi5/ssog

Official repository of Polarity-aware Linear Attention for Vision Transformers (ICLR 2025)

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

[CVPR 2026 Highlight] Official implementation of Log-linear Sparse Attention (LLSA).

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention

This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.

Generalized Optimal Transport Attention with Trainable Priors

[NeurIPS 2021 Spotlight] & [IJCV 2024] SOFT: Softmax-free Transformer with Linear Complexity

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch