token-gate-llms/TokenGate

Accelerating Long Context LLM Inference with Accuracy-Preserving Context Optimization in SGLang, vLLM, llama.cpp, OpenClaw, RAG, and Agentic AI.

Recursive Language Models for efficient long-context processing. Analyze 1M+ tokens by storing context in a Python REPL while reducing LLM token usage.

Query-aware context compression for LLMs and coding agents. Cut ~65% of input tokens.

Local proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history, tool output, and code before they're sent: -31% input / -74% output, measured live. Any provider, no extra model calls. Also an MCP server and embeddable library (Rust, Python, Ruby, Kotlin, Swift, JS/TS).

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

Drop-in prompt compression for production LLM apps. Cut your token bill 40-60% without changing your code. Python SDK, LLMLingua-2, MIT.

Optimizing inference proxy for LLMs

A lossless token-optimization codec for LLM agents — cut input tokens up to 82%, nothing dropped, every fact kept. Drop-in proxy + one-line Python API for Claude Code, Codex, Cursor, OpenAI, Anthropic & Ollama. pip install foveance