vllm-project/production-stack

Achieve state of the art inference performance with modern accelerators on Kubernetes

Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.

Cost-efficient and pluggable Infrastructure components for GenAI inference

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

A Datacenter Scale Distributed Inference Serving Framework

Kubernetes AI Toolchain Operator

Helm chart for Ollama on Kubernetes

Scale LLM Engine public repository

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes