h9-tec/llm-architecture-ar

Explorations into the recently proposed Taylor Series Linear Attention

[CVPR 2026 Highlight] Official implementation of Log-linear Sparse Attention (LLSA).

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

🚀🚀 Efficient implementations of Native Sparse Attention

🚀 Efficient implementations for emerging model architectures

[Technical Report] Multi-Stream Attention Residuals

Official repository for Parallax (Parameterized Local Linear Attention)

Code for Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention