LLM observability tools
Agents fail quietly. The call returns 200 and the answer is still wrong. These tools record every LLM call, tool call and step, so you can see what happened, score it and catch it next time. Langfuse, Phoenix and Latitude publish their code and run on your own servers. The rest are cloud-first, billed on traces, spans, credits or calls. 6 tools, in the order they appear in the directory. This is not a ranking.
Grouped by job: AI Agent Infrastructure
Cekura
Automated QA and observability for voice and chat AI agents: simulate conversations across personas before launch, monitor production calls for instruction-following and tool-call errors, and plug tests into CI/CD.
Promptic
Optimization platform for GenAI apps: OpenTelemetry-native LLM tracing, model benchmarking and one-click prompt and agent tuning scored on your own quality and cost metrics; free tier, team plans from €149/month.
Latitude
Open-source observability and quality platform for AI agents: traces production runs, turns failures into tracked issues with evals attached and dispatches your coding agent to fix them; paid team plans.
Langfuse
Open-source LLM engineering platform: traces and sessions for LLM apps and agents with token and cost tracking, prompt management, datasets, LLM-as-judge and human evals; OpenTelemetry-based, self-host free or use Langfuse Cloud in the US, EU or Japan.
LangSmith
LangChain's platform for agents: tracing and online and offline evals for any framework, plus managed agent deployment, sandboxes and an LLM gateway; cloud in the US or EU, hybrid or self-hosted on Enterprise.
Arize Phoenix
Arize's open-source AI observability platform: OpenTelemetry tracing through OpenInference for OpenAI, Anthropic, LangGraph, the Vercel AI SDK and more, with LLM evals, datasets, experiments and a prompt playground; runs locally, in Docker or on Kubernetes.
What it traces, where it runs, what it costs.
Straight from each vendor's own pages. Plans move often, so the check date is below.
| Tool | What it traces | Self-host | Free option | Price |
|---|---|---|---|---|
| Cekura | Live voice and chat agent calls, checked for instruction-following and tool-call errors. Simulated conversations before launch. | VPC or on-prem on Enterprise. | Pay as you go with the first seat free, no card. 300 free credits to start. | $0.05 per monitored call, $0.25 per voice testing minute. Startup $500/mo. Enterprise custom. |
| Promptic | OpenTelemetry-native LLM traces, tied to benchmarks and prompt and agent tuning scored on your own quality and cost metrics. | No. Cloud in the EU. Private cloud or BYOC on Enterprise. | Free: 1 user, 1 AI application, 200K credits a month, 14-day retention. | Team €149/mo with 1M credits, then €10 per 100K; extra applications €100/mo each. Business €599/mo. Enterprise custom. |
| Latitude | Production agent runs, turned into tracked issues with evals attached. | Yes, open source. Custom on-prem deployment on Enterprise. | Starter: 20K credits a month, 30-day retention, unlimited seats. | Pro $99/mo with 100K credits and 90-day retention, then $20 per 10K credits. Enterprise custom. |
| Langfuse | Traces, sessions and users for LLM apps and agents, with token and cost tracking. Prompts, datasets and evals in the same app. | Yes, free. Core is MIT; enterprise features are licensed separately. | Hobby: 50K units a month, 30 days of data, 2 users. No card. | Core $29/mo, Pro $199/mo, each with 100K units, then $8 per 100K (lower with volume). Teams add-on $300/mo. Enterprise $2,499/mo. |
| LangSmith | Agent runs and LLM calls from any framework, with online and offline evals. | Enterprise only: hybrid or fully self-hosted. | Developer: 1 seat, 5K base traces a month. | Plus $39 per seat a month with 10K base traces, then pay as you go. Base traces keep 14 days, extended 180. Enterprise custom. |
| Arize Phoenix | OpenTelemetry traces through OpenInference, plus evals, datasets, experiments and a prompt playground. | Yes, under the Elastic License 2.0: locally, in Docker or on Kubernetes. | Phoenix is free to self-host. Arize AX Free: 25K spans a month, 15-day retention. | Arize AX Pro $50/mo: 50K spans, 10 GB, 30-day retention. Enterprise custom, SaaS or self-hosted. |
Checked 2026-10-30. Sources: Langfuse pricing, Langfuse license, LangSmith pricing, Arize pricing, Phoenix on GitHub, Latitude pricing, Promptic pricing, Cekura pricing.
Pick it if.
- LangfuseYou want open source you can self-host for free, or a cloud plan billed on what you send it.
- LangSmithYou build on LangChain or LangGraph, or want tracing, deployment and an LLM gateway from one vendor.
- Arize PhoenixYou want tracing on your laptop today and a managed platform later, on the same instrumentation.
- LatitudeYou want production failures to land as issues with evals attached, not as traces to dig through.
- PrompticYou need EU hosting, or want traces wired straight into prompt tuning.
- CekuraYour agent talks to customers by voice or chat and every live call needs checking.
Each listing.
The directory page, alternatives in the same category, and a head-to-head where we have one.
- CekuraAlternatives
- PrompticAlternatives
- LatitudeAlternatives
- LangfuseAlternatives
- LangSmithAlternatives
- Arize PhoenixAlternatives
Category pages list every tool in that category, including ones left off this page. Built one of these? Get it featured in its category.
The 2 biggest AI stories, every morning.
One short email: the two stories that matter, and why.
You're in.
The Daily AI Brief will land in your inbox soon. If you don't see it, check Promotions or Spam and move it to your main inbox.