EleutherAI/lm-evaluation-harness

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

A unified evaluation framework for large language models

Inspect: A framework for large language model evaluations

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

LiveBench: A Challenging, Contamination-Free LLM Benchmark

The LLM Evaluation Framework

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.