dustalov/evalica

Source Code of Arena Leaderboard Methodology

provides a common interface to many IR measure tools

Statistical inference for AI evaluations, from model comparisons to corrections for LLM judge bias, optimized for small sample sizes. All defaults battle-tested in Monte Carlo simulations.

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker

Evaluation software used in the Text Retrieval Conference

Fast computation of Krippendorff's alpha agreement measure in Python.

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices.