princeton-nlp/HELMET

BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Official repo for "Make Your LLM Fully Utilize the Context"

Benchmarking long-form factuality in large language models. Original code for our paper "Long-form factuality in large language models".

A Comprehensive Survey on Long Context Language Modeling

A unified evaluation framework for large language models

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]