microsoft/superbenchmark

AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

Open Source Inference Research Platform Standard / 开源推理研究平台

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Open-source library for scalable, reproducible evaluation of AI models and benchmarks.

An open source benchmarking framework for IT automation

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

A benchmark of real-world DL kernel problems

Reference implementations of MLPerf® inference benchmarks

AISBench Benchmark is a model evaluation tool built on OpenCompass, compatible with OpenCompass’s configuration system, dataset structure, and model backend implementation, while extending support for service-based models.