triton-inference-server/client

This repository contains tutorials and examples for Triton Inference Server

Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Deep Learning Server and CLI for Torch and TensorRT

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

The Triton TensorRT-LLM Backend

High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI