ianarawjo/ChainForge

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.

Your Automatic Prompt Engineering Assistant for GenAI Applications

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Inspect: A framework for large language model evaluations

bloom - evaluate any behavior immediately  🌸🌱

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

The LLM Evaluation Framework

Supercharge Your LLM Application Evaluations 🚀

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.