kubeai-project/kubeai

Achieve state of the art inference performance with modern accelerators on Kubernetes

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Open-source inference server and production cluster for all the models your agent needs.

Cost-efficient and pluggable Infrastructure components for GenAI inference

A flexible, high-performance serving system for machine learning models

Universal LLM Deployment Engine with ML Compilation