ivanfioravanti/llm_context_benchmarks

LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context lengths. Supports SGLang and vLLM engines.

Local LLM Inference Speed Test Tool

Benchmark and optimize LLM inference across frameworks with ease

llama-benchy - llama-bench style benchmarking tool for all backends

LLM Benchmark for Throughput via Ollama (Local LLMs)

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoints across different concurrency levels.

Test LLMs on real tasks. Compare models side-by-side.

Benchmark suite for LLMs from Fireworks.ai