tomaarsen/attention_sinks

[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

Recursive Language Models for efficient long-context processing. Analyze 1M+ tokens by storing context in a Python REPL while reducing LLM token usage.

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

Gemma 2B with 10M context length using Infini-attention.

MoBA: Mixture of Block Attention for Long-Context LLMs

Large Language Model Text Generation Inference