Giskard-AI/giskard-oss

The LLM Evaluation Framework

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Evaluation and Tracking for LLM Experiments and AI Agents

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

Supercharge Your LLM Application Evaluations 🚀

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

the LLM vulnerability scanner

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.