triton-inference-server/perf_analyzer

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

GenAI inference performance benchmarking tool

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

Inference server benchmarking tool

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

Reference implementations of MLPerf® inference benchmarks

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

A profiling and performance analysis tool for machine learning