IBM/eval-assist

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

The LLM Evaluation Framework

Inspect: A framework for large language model evaluations

A Python library encapsulating best practices for rubric-based evaluation of LLM/VLM outputs using AI judges such as LLMs and decision models.

Automated system for LLM evaluation via agents. Doc as below:

Readymade evaluators for your LLM apps

Simple LLM Evaluation Using LLM As a Judge 👩‍⚖️

LiveBench: A Challenging, Contamination-Free LLM Benchmark

bloom - evaluate any behavior immediately  🌸🌱