stevibe/BenchLocal

A multi-platform desktop application to evaluate and compare LLM models, written in Rust and React.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

Provider-agnostic, open-source evaluation infrastructure for language models

An LLM playground you can run on your laptop

Advanced CLI tool that scans your hardware and tells you exactly which LLM or sLLM models you can run locally, with full Ollama integration.

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends