openvinotoolkit/model_server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

A flexible, high-performance serving system for machine learning models

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Achieve state of the art inference performance with modern accelerators on Kubernetes

Serve, optimize and scale PyTorch models in production

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.