allenai/olmo-eval

Evaluation suite for LLMs

A framework for few-shot evaluation of language models.

Provider-agnostic, open-source evaluation infrastructure for language models

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

Comprehensive LLM evaluation at scale: A production-ready framework for evaluating large language models across multiple benchmarks.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Benchmark your local LLMs with real evaluation suites. Not vibes - actual scores.

LiveBench: A Challenging, Contamination-Free LLM Benchmark