cvs-health/langfair

Deliver safe & effective language models

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

Inspect: A framework for large language model evaluations

Simple LLM Evaluation Using LLM As a Judge 👩‍⚖️

The LLM Evaluation Framework

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Auditing LLM-as-judge systems with paired synthetic controls, bootstrap CIs, and BH-FDR bias detection.

A unified evaluation framework for large language models

bloom - evaluate any behavior immediately  🌸🌱