RedCode

RedCode is a benchmark tool designed for the evaluation of code agents' capabilities in safely executing and generating code across various programming languages and formats. It features two primary components: RedCode-Exec, which assesses the agents' ability to manage unsafe code execution with over 4,000 test cases, and RedCode-Gen, which evaluates the generation of potentially harmful code through 160 prompts. The tool aims to enhance the safety and reliability of code agents by providing a comprehensive dataset and structured evaluation environment.