HKUDS/SepLLM

[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

Sparse Decoupled Attention for Efficient Long-Context LLM Inference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

latent context language models

Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

[NeurIPS 2026] Official implementaion for Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)