Layer / What the factory did
Observability
Run history, cost, cycle time, and audit trails so a team can manage a portfolio of agent work instead of a pile of chat logs.
- AgentOps
Observability aimed at agent runs, not only chat completions.
- Braintrust
Eval-first platform: datasets, experiments, and logging.
- Factory
Droids across the SDLC, sold as a software factory.
- Galileo
Evaluation and observability for production AI systems.
- Helicone
Open LLM gateway with logging, caching, and spend controls.
- Langfuse
Open-source LLM observability you can self-host.
- LangSmith
LangChain’s commercial tracing, eval, and prompt hub.
- LangWatch
Observability and evaluation platform for LLM products.
- Logfire
Pydantic’s observability product, including LLM/agent traces.
- Phoenix
Arize’s open observability and eval notebook for LLM apps.
- Portkey
AI gateway: routing, guardrails, and observability in one proxy.
- SpecStory
Capture AI IDE sessions so intent does not die in the chat.
- Warp
Open terminal, multi-model agents, and factories in early access.