local-inference-lab/llm-inference-bench

Local LLM Inference Speed Test Tool

📊 LLM Context Benchmarks - A comprehensive benchmarking tool for testing LLMs with varying context sizes using Ollama. Features dual benchmark modes (API/CLI), automatic hardware detection (optimized for Apple Silicon), visual performance charts.

The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoints across different concurrency levels.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Inference server benchmarking tool

Benchmark and optimize LLM inference across frameworks with ease

llama-benchy - llama-bench style benchmarking tool for all backends

LLM Benchmark for Throughput via Ollama (Local LLMs)

Benchmark suite for LLMs from Fireworks.ai