cve-bench

CVE-Bench is a benchmarking tool designed to evaluate AI agents' capacities to exploit real-world web application vulnerabilities, specifically utilizing critical-severity Common Vulnerability and Exposure (CVE) data. The tool enables automated assessments through tasks like denial of service, remote code execution, and unauthorized access, leveraging Docker for consistent environments and reproducible results. Notable features include a focused dataset of 40 CVEs, integration of automatic exploits, and support for various attack vectors, making it a pivotal resource for researchers in AI safety and security evaluation.