Directory / Promptfoo
Promptfoo
Verified
Open evaluation and red-team framework for LLM applications.
Promptfoo is an open-source eval framework: YAML test cases, assertions, red-teaming, and CI hooks. Teams use it as a quality gate for prompts, tools, and sometimes coding agents. A cloud product exists around the OSS core.
- Model support
- Multi-model
- Deployment
- Cloud, Self-hosted
- Open source
- Yes
- License
- MIT
- Pricing model
- Mixed
- Added / updated
- 19 Aug 2026 / 19 Aug 2026
www.promptfoo.dev · Docs · Source
Assessment
Promptfoo fits CI evaluation of prompts, tools, and agents through cases and assertions. Use hold-out cases the agent cannot edit. The MIT runner and optional cloud product are separate surfaces.
Strengths
- MIT evals you can run in CI.
- Model-agnostic, assertion-based.
- Documented red-team features alongside happy-path evals.
Limitations
- Case design remains user-owned; the tool does not define target behavior.
- Cloud UI is not the open runner.
- Prompt-only cases can miss product behavior.
Compare
- vs deepeval : Two evaluation harnesses for software quality gates.