EM-GeekLab/LLMOne

Universal LLM Deployment Engine with ML Compilation

Enterprise-grade Private Model-as-a-Service Platform

A deployment, monitoring and autoscaling service towards serverless LLM serving.

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Scale LLM Engine public repository

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.