allenai/OLMo-Eval-Legacy

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

A framework for few-shot evaluation of language models.

Comprehensive LLM evaluation at scale: A production-ready framework for evaluating large language models across multiple benchmarks.

Provider-agnostic, open-source evaluation infrastructure for language models

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Inspect: A framework for large language model evaluations

Automated system for LLM evaluation via agents. Doc as below:

Benchmark your local LLMs with real evaluation suites. Not vibes - actual scores.