Observability developer tools
Every listing tagged observability, A to Z: 10 tools across 3 categories. Listed in alphabetical order, not ranked.
Arize Phoenix
Arize's AI observability platform, source-available under the Elastic License 2.0: OpenTelemetry tracing through OpenInference for OpenAI, Anthropic, LangGraph, the Vercel AI SDK and more, with LLM evals, datasets, experiments and a prompt playground; runs locally, in Docker or on Kubernetes.
Cekura
Automated QA and observability for voice and chat AI agents: simulate conversations across personas before launch, monitor production calls for instruction-following and tool-call errors, and plug tests into CI/CD.
guilds.run
Open-source (AGPL-3.0) multi-agent teams you can inspect: debug every prompt, tool call, memory change and agent interaction, control permissions and pick the model per agent. Self-host free on your own machine with Docker Compose; hosted plan from $19/month coming soon.
Inferock
Accountable LLM inference: watch your own OpenAI, Anthropic or Gemini calls for failures and overbilling, run serverless inference with credits back on eligible failed calls, or rent dedicated capacity; open-source inferock-bench proxy.
KloudMate
AI-powered observability and SRE-Ops: logs, metrics, traces, infrastructure, Kubernetes, RUM, synthetic and database monitoring in one platform, with an AI assistant for root-cause analysis; Pro from $99/month.
Langfuse
Open-source LLM engineering platform: traces and sessions for LLM apps and agents with token and cost tracking, prompt management, datasets, LLM-as-judge and human evals; OpenTelemetry-based, self-host free or use Langfuse Cloud in the US, EU or Japan.
LangSmith
LangChain's platform for agents: tracing and online and offline evals for any framework, plus managed agent deployment, sandboxes and an LLM gateway; cloud in the US or EU, hybrid or self-hosted on Enterprise.
Latitude
Open-source (MIT) observability and quality platform for AI agents: traces production runs, groups failures into tracked issues with evals attached, dispatches your coding agent to fix them and replays the fix against the failing traces. MCP server and CLI. Free Starter plan with 20K credits a month; Pro $99 a month.
Parseable
Open-core (AGPL-3.0) observability platform written in Rust: logs, metrics and traces in one binary on S3-compatible object storage, with SQL and PromQL, OpenTelemetry-native ingest, self-hosted or as Parseable Cloud.
XTrace
Open-source macOS app (Apple Silicon) that reads your local Claude Code, Codex and Cursor history to show agent activity, estimated human time, API-equivalent spend and how fast you are using your Claude and Codex limits. Apache-2.0.