promptfoo

Promptfoo is a command-line interface (CLI) and library designed for evaluating and red-teaming large language model (LLM) applications. Its primary use case involves automated testing of prompts, vulnerability scanning, and model comparison, enabling developers to enhance security and reliability in their AI applications while running evaluations locally without exposing data. Notable features include integration with CI/CD workflows, comprehensive security reporting, and support for multiple LLM providers, allowing for a developer-centric, flexible, and data-driven approach to AI application development.