airnsk/proxycache

Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response

A reverse proxy server to cache the slow expensive requests to the openai api. Built with nginx and docker.

Optimize DeepSeek prefix cache hit rates — reduce API input costs by 95%+ in multi-turn coding sessions

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

Accelerating Long Context LLM Inference with Accuracy-Preserving Context Optimization in SGLang, vLLM, llama.cpp, OpenClaw, RAG, and Agentic AI.

Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets.

Give any local LLM a billion-token memory. Open-source, local-first context + memory engine for Ollama and AI agents — runs on your own machine, free. 🧠⚡

LLM KV cache compression made easy