ray-project/ray-llm

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

list of awesome platforms, tools, and resources run for LLMs locally

Agentic components of the Llama Stack APIs

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

LangServe 🦜️🏓

A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally

A simple TUI for serving local LLM models. Pick a model, pick a backend, serve it

🎉 An awesome & curated list of best LLMOps tools.