jianke0604/MSAR

[Tech Report, NeurIPS 2026] Expanded Hyper-Connections

Official Project Page for Deep Delta Learning (https://arxiv.org/abs/2601.00417)

This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.

[CVPR 2026 Highlight] Official implementation of Log-linear Sparse Attention (LLSA).

DepthBench: Measuring How Residual Connections Enable More Computational Depth

[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

The official implementation of the paper "MLP Memory: A Retriever-Pretrained Memory for Large Language Models". (ICLR 2026)