wandb/rai-toolkit

Evaluation and Tracking for LLM Experiments and AI Agents

🐢 Open-Source Evaluation & Testing library for LLM Agents

Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding, RAG), TigerTune (fine-tuning)

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

An open-source compliance-centered evaluation framework for Generative AI models

The LLM Evaluation Framework

Data-Driven Evaluation for LLM-Powered Applications

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

AI Observability & Evaluation