kaito-project/kaito

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Achieve state of the art inference performance with modern accelerators on Kubernetes

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Cloud Native Agentic AI | Discord: https://bit.ly/kagentdiscord

Cost-efficient and pluggable Infrastructure components for GenAI inference

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization