benchjack

BenchJack is a vulnerability scanner designed to assess whether AI benchmarks can be manipulated, highlighting weaknesses before adversarial agents can exploit them. It employs a multi-phase audit process that combines static analysis tools with AI-driven deep inspection, categorizing vulnerabilities into eight distinct classes and providing real-time results via a web dashboard. Notable features include proof-of-concept exploit code generation and future integration of Docker sandboxing for enhanced security during analysis.