jolibrain/deepdetect

Serve, optimize and scale PyTorch models in production

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

A flexible, high-performance serving system for machine learning models

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

Containers for machine learning

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.