rednote-machine-learning/RedKnot

Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

MoBA: Mixture of Block Attention for Long-Context LLMs

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

Unified KV Cache Compression Methods for Auto-Regressive Models

Empowering RAG with a memory-based data interface for all-purpose applications!