SeldonIO/MLServer

A flexible, high-performance serving system for machine learning models

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

A scalable inference server for models optimized with OpenVINO™

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

The simplest way to serve AI/ML models in production

Serve, optimize and scale PyTorch models in production

A Datacenter Scale Distributed Inference Serving Framework