Yoosu-L/llmapibenchmark

llama-benchy - llama-bench style benchmarking tool for all backends

Benchmark suite for LLMs from Fireworks.ai

Local LLM Inference Speed Test Tool

LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context lengths. Supports SGLang and vLLM engines.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

LLM Benchmark for Throughput via Ollama (Local LLMs)

📊 LLM Context Benchmarks - A comprehensive benchmarking tool for testing LLMs with varying context sizes using Ollama. Features dual benchmark modes (API/CLI), automatic hardware detection (optimized for Apple Silicon), visual performance charts.

Inference server benchmarking tool

Benchmark local LLM serving by replaying real agent trajectories