EverMind-AI/MSA

MoBA: Mixture of Block Attention for Long-Context LLMs

Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

Empowering RAG with a memory-based data interface for all-purpose applications!

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model.

[ICML'26] SimpleMem: Efficient Lifelong Memory for LLM Agents — Text & Multimodal