tsinghua-ideal/Twilight

Sparse Decoupled Attention for Efficient Long-Context LLM Inference

[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

[ICML 26 Spotlight, NeurIPS 25 @ ER] Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

Trainable fast and memory-efficient sparse attention

MoBA: Mixture of Block Attention for Long-Context LLMs

RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse

A sparse-first inference engine (SparseEngine).