Bryley/evalt

A comprehensive evaluation framework for AI agents and LLM applications.

Typed prompt-and-judge SDK, CLI, and evidence viewer for agent evaluations

🧪💥 Evaluation framework based on Vitest, the testing framework you familiar with, for agents, models, and more.

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

Requirement-driven evaluation harness for AI agents and LLM applications. Generate behavior-specific test cases, run them against any target (hosted models, callable wrappers, OTel-traced agents), and inspect local-first artifacts.

Invarium is pytest for AI agents. Test behavior, not exact text.

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

Python SDK for DefuzeX agent behavior testing.

Agent eval on one running base. Swap the agent under test with plugins; run the same dataset anywhere.