messkan/prompt-cache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

Self-hosted LLM router with a managed safety net. OpenAI-compatible. BYOK. Single-workspace. Streaming. For more advanced routing choose hosted OrcaRouter

Unified large model proxy gateway, providing load balancing, fault transfer, current limiting, budget management, content auditing, caching, and MCP gateway capabilities.

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.

Universal LLM Gateway: One API, every LLM. OpenAI/Anthropic-compatible endpoints with multi-provider translation and intelligent load-balancing.

The fastest, lightest, and easiest-to-integrate AI gateway on the market. Fully open-sourced.

Source-available LLM router and AI cost optimizer. Routes simple prompts to cheap or local models and complex ones to premium models, automatically. Drop-in OpenAI-compatible proxy for Claude Code, Codex, Cursor and OpenClaw. Self-hosted, no middleman.