mosafariuk/TokenFlow-Gateway — GitHub trending stats & insights

Featured

mosafariuk/TokenFlow-Gateway

TokenFlow is an enterprise-grade, token-aware load balancer and AI gateway for vLLM. It prevents GPU OOM crashes using Redis-backed admission control, semantic caching, and dynamic token routing. Built in TypeScript for extreme bare-metal performance.

Visit GitHub

TypeScript

1 contributors

MIT License

Social mentions

Recent discussions about this repository across the web

Built a token-aware gateway/load balancer for local LLM stacks — because nginx has no idea what a token costs

r/LocalAIStack

TokenFlow Gateway – a token-aware load balancer/gateway for self-hosted LLMs (vLLM/Ollama), because nginx has no idea what a token is

r/coolgithubprojects

Repository activities

repository's daily and monthly activities across stars, forks, merged PRs, issues, and closed issues