ai-dynamo/aiperf

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

llama-benchy - llama-bench style benchmarking tool for all backends

AISBench Benchmark is a model evaluation tool built on OpenCompass, compatible with OpenCompass’s configuration system, dataset structure, and model backend implementation, while extending support for service-based models.

Open Source Inference Research Platform Standard / 开源推理研究平台

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

A validation and profiling tool for AI infrastructure

Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.

Provider-agnostic, open-source evaluation infrastructure for language models

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.