deepjavalibrary/djl-serving

A flexible, high-performance serving system for machine learning models

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

The simplest way to serve AI/ML models in production

A scalable inference server for models optimized with OpenVINO™

⚡ Fastest way to serve open source ML models to millions

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Serve, optimize and scale PyTorch models in production