Directory / Promptfoo
Promptfoo
Verified
Open evals and red-team for LLM apps and agents.
Promptfoo is an open-source eval framework: YAML test cases, assertions, red-teaming, and CI hooks. Teams use it as a quality gate for prompts, tools, and sometimes coding agents. A cloud product exists around the OSS core.
- Model independence
- Multi-model
- Deployment
- Cloud, Self-hosted
- Open source
- Yes
- License
- MIT
- Pricing model
- Mixed
- Added / updated
- 19 Aug 2026 / 19 Aug 2026
www.promptfoo.dev · Docs · Source
Editorial take
Promptfoo is the eval gate that looks like unit tests, which is why it belongs in a factory. It will not review your PR style. Use it on hold-out cases the agent cannot edit. MIT core; cloud is optional.
Strengths
- MIT evals you can run in CI.
- Model-agnostic, assertion-based.
- Red-team features documented, not only happy-path evals.
Limitations
- You must write the cases. The tool will not invent a good spec.
- Cloud UI is not the open runner.
- Easy to eval the prompt and skip the product.
Compare
- vs deepeval — Two eval harnesses used as factory quality gates.