usnistgov/caisi-cyber-evals

CyberGym is a large-scale, high-quality cybersecurity evaluation framework designed to rigorously assess the capabilities of AI agents on real-world vulnerability analysis tasks.

CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities

CyberGym-E2E is a large-scale benchmark built from real-world vulnerabilities in widely used open-source projects to evaluate AI agents' end-to-end cybersecurity capabilities, from discovering vulnerabilities to generating proof-of-concept to writing patches.

An Open Harness and Benchmark for AI in Cybersecurity Operations.

Red-teaming and evaluation framework for AI agents, built around an OWASP-inspired ASI01–ASI10 taxonomy

Benchmarking LLM agents on Cyber Threat Investigation.

ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities designed to evaluate AI agents' ability to develop exploits.

Damn Vulnerable AI Agent is a deliberately vulnerable AI agent platform for security testing and education.