microsoft/OpenRCA

[FSE'26, WWW'25, ASE'24] RCAEval: A Benchmark for Root Cause Analysis.

Comprehensive LLM Error Analysis and Reporting

AI Agent failure replay and root cause analysis platform

Benchmark measuring whether an LLM actually reads cluster evidence before diagnosing a Kubernetes incident, or just guesses confidently. Chaos injection, evidence ablation, cross-model scoring.

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

AgenTracer: A Lightweight Failure Attributor for Agentic Systems

Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

Open-source agent reliability platform for discovering, investigating, and fixing production agent failures