PaddlePaddle/Serving

A flexible, high-performance serving system for machine learning models

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

A scalable inference server for models optimized with OpenVINO™

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

Deep Learning Server and CLI for Torch and TensorRT

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization