grishahq/recursive-llm

A feature rich implementation of Recursive Language Models, with ACP integration, REPL tool support, structured IO, advanced visualization, logging tools.

[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

General plug-and-play inference library for Recursive Language Models (RLMs), supporting various sandboxes.

Extend existing LLMs way beyond the original training length with constant memory usage, without retraining

Drop-in prompt compression for production LLM apps. Cut your token bill 40-60% without changing your code. Python SDK, LLMLingua-2, MIT.

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Access large language models from the command-line

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Gemma 2B with 10M context length using Infini-attention.