huggingface/evaluate

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Inspect: A framework for large language model evaluations

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

Code for the paper "Evaluating Large Language Models Trained on Code"

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Supercharge Your LLM Application Evaluations 🚀

The LLM Evaluation Framework