LLM observability and eval tools
Traces tell you what your LLM app did. Evals tell you whether it was any good. You need both: traces to debug a bad answer in production, datasets and scorers so the same mistake doesn't come back after the next prompt change. Every tool here does both. Langfuse, Helicone, Opik, Laminar and Latitude ship under open licenses you can self-host, Phoenix is source-available, Weave opens its SDK but not its server, and Braintrust and LangSmith self-host only on Enterprise. Each one meters something different (units, traces, spans, requests, GB, credits), so price it on your own volume. 9 tools, in the order they appear in the directory. This is not a ranking.
Grouped by job: AI Agent Infrastructure
Latitude
Open-source (MIT) observability and quality platform for AI agents: traces production runs, groups failures into tracked issues with evals attached, dispatches your coding agent to fix them and replays the fix against the failing traces. MCP server and CLI. Free Starter plan with 20K credits a month; Pro $99 a month.
Langfuse
Open-source LLM engineering platform: traces and sessions for LLM apps and agents with token and cost tracking, prompt management, datasets, LLM-as-judge and human evals; OpenTelemetry-based, self-host free or use Langfuse Cloud in the US, EU or Japan.
LangSmith
LangChain's platform for agents: tracing and online and offline evals for any framework, plus managed agent deployment, sandboxes and an LLM gateway; cloud in the US or EU, hybrid or self-hosted on Enterprise.
Arize Phoenix
Arize's AI observability platform, source-available under the Elastic License 2.0: OpenTelemetry tracing through OpenInference for OpenAI, Anthropic, LangGraph, the Vercel AI SDK and more, with LLM evals, datasets, experiments and a prompt playground; runs locally, in Docker or on Kubernetes.
Braintrust
Observability and evals platform for AI agents: trace production calls from OpenAI, Anthropic, Gemini, the Vercel AI SDK, LangChain and more, run experiments scored by LLM judges, Autoevals or your own code, review in playgrounds and human annotation queues, and let the built-in Loop agent generate test cases and build evals. MCP server and a bt CLI. Free Starter plan with unlimited users; Pro $249 a month.
Helicone
Open-source (Apache-2.0) AI gateway and LLM observability platform: point an OpenAI-compatible client at one URL to reach 100+ models with routing and automatic fallbacks, and every request is logged with cost, latency, sessions and agent traces. Prompt versioning, a playground, and scores and datasets for evaluating outputs. Self-host with Docker Compose or use the cloud, free for 10,000 requests a month.
W&B Weave
Weights & Biases' toolkit for tracing and evaluating AI agents: sessions, turns, tool calls and sub-agents as first-class traces, an evaluation framework with comparisons and leaderboards, built-in signals with Slack alerts, guardrail scorers for toxicity, PII and hallucinations, a playground, and an MCP server so coding agents can read traces and run evals. SDK is open source (Apache-2.0); free plan with 1 GB of trace data a month, Pro from $60 a month.
Opik
Comet's open-source (Apache-2.0) LLM observability and evaluation platform: traces for agents and multi-turn conversations, datasets and experiments with 30+ LLM-as-a-judge and heuristic metrics, pass/fail test suites, a PyTest integration for CI, online evaluation rules on production traces, guardrails and an agent optimizer. The full platform, backend included, self-hosts free; Opik Cloud is free for 25K spans a month.
Laminar
Open-source (Apache-2.0) observability for AI agents, written in Rust: one-line OpenTelemetry tracing for the Vercel AI SDK, Browser Use, Stagehand, LangChain, OpenAI, Anthropic and more, Signals that flag behaviour you describe in plain English, evals from an SDK and CLI, browser session recordings, SQL over traces and MCP access for coding agents. Self-host with Docker Compose, or use the cloud, free for 1 GB of data.
What it is, where it runs, what it costs.
Straight from each vendor's own pages. Plans move often, so the check date is below.
| Tool | What it is | Runs on | Open source | Price |
|---|---|---|---|---|
| Latitude | Observability and quality for agents: traces production runs, groups failures into tracked issues with evals attached and dispatches your coding agent to fix them. | Latitude's cloud, or self-hosted from the repo. Custom on-prem or cloud deployment on Enterprise. | Yes, MIT. | Starter free: 20K credits a month, 30-day retention, unlimited seats. Pro $99/mo: 100K credits, 90 days, then $20 per 10K credits. Enterprise custom. |
| Langfuse | Open-source LLM engineering platform: traces, sessions and cost tracking, prompt management, datasets, LLM-as-a-judge and human evals. | Langfuse Cloud in the US, EU or Japan, or self-hosted with Docker Compose or Kubernetes. | Yes, MIT core. Enterprise directories carry a separate license. | Hobby free: 50K units a month, 30 days of data, 2 users. Core $29/mo and Pro $199/mo with 100K units, then $8 per 100K (less at volume). Enterprise $2,499/mo. Self-hosting is free. |
| LangSmith | LangChain's tracing plus online and offline evals for agents on any framework, with deployment, sandboxes and an LLM gateway alongside. | LangChain's cloud in the US or EU. Hybrid or fully self-hosted on Enterprise. | No. | Developer free: 1 seat, 5K base traces a month. Plus $39 per seat a month with 10K base traces, then pay as you go. Base traces keep 14 days, extended 180. Enterprise custom. |
| Arize Phoenix | Tracing on OpenTelemetry through OpenInference, with evals, datasets, experiments and a prompt playground. Arize AX is the managed platform on top. | Your laptop, Docker or Kubernetes. Arize AX in Arize's cloud (US, EU or Canada), or self-hosted on Enterprise. | Source-available under the Elastic License 2.0, not an OSI license. | Phoenix is free to self-host. AX Free: 25K spans and 1 GB a month, 15-day retention. AX Pro $50/mo: 50K spans, 10 GB, 30 days. Enterprise custom. |
| Braintrust | Tracing and evals for agents: experiments scored by LLM judges, Autoevals or your own code, playgrounds, human review and a Loop agent that builds evals. | Braintrust's cloud. On-prem or hosted deployment on Enterprise. | No. Its Autoevals scorer library is MIT. | Starter free: unlimited users, 1 GB processed data, 10K scores, 14-day retention, $10 model credit. Pro $249/mo: 5 GB and 50K scores, then $3/GB and $1.50 per 1K scores, 30 days. Enterprise custom. |
| Helicone | AI gateway and observability in one: 100+ models behind one OpenAI-compatible API, with every request logged for cost, latency, sessions and traces. Prompts, scores and datasets. | Helicone's cloud, or self-hosted with Docker Compose. Helm chart and on-prem through Enterprise. | Yes, Apache-2.0. | Hobby free: 10K requests, 1 GB, 1 seat, 7-day retention. Pro $79/mo and Team $799/mo, plus usage-based pricing past 10K requests and 1 GB. Enterprise: contact sales. |
| W&B Weave | Weights & Biases' tracing and evals for agents: sessions, turns and sub-agents as traces, eval comparisons, leaderboards, signals, guardrail scorers and a playground. | W&B's cloud, now sold as CoreWeave Forge plans. Self-managed on your Kubernetes with a Weave-enabled W&B license. | SDK yes, Apache-2.0. Self-managed server needs a commercial license. | Free: 1 GB of Weave data a month. Pro from $60/mo for teams under 50 people: 1.5 GB, then $0.10/MB. Enterprise custom. |
| Opik | Comet's observability and evals platform: agent and conversation traces, datasets, experiments, 30+ built-in metrics, test suites, online evals, guardrails and prompt optimization. | Self-hosted from the repo, or Opik Cloud in the US. | Yes, Apache-2.0, the full platform including the backend. | Self-hosting free. Free Cloud: 25K spans a month, 60-day retention. Pro Cloud $19/mo: 100K spans, then $5 per 100K. Enterprise custom. |
| Laminar | Agent observability written in Rust: OpenTelemetry traces, Signals that flag behaviour you describe in plain English, evals from SDK or CLI, browser session recordings, SQL over traces. | Laminar's cloud, or self-hosted with Docker Compose. On-prem on Enterprise. | Yes, Apache-2.0. | Free: 1 GB, 7 days of data, 1 project, no Signals. Starter $30/mo: 3 GB, then $2/GB. Pro $150/mo: 10 GB, then $1.50/GB, 6 months. Signals billed per token. Enterprise custom. |
Checked 2026-11-01. Sources: Langfuse pricing, Langfuse on GitHub, LangSmith pricing, Braintrust pricing, Braintrust docs, Helicone pricing, Helicone on GitHub, Arize pricing, Phoenix on GitHub, W&B pricing, Weave on GitHub, Weave self-managed docs, Opik pricing, Opik on GitHub, Laminar pricing, Laminar on GitHub, Latitude pricing, Latitude on GitHub.
Pick it if.
- LangfuseYou want open source you can run yourself for free, with prompts and evals in the same app as your traces.
- LangSmithYou build on LangChain or LangGraph and want tracing, evals and deployment from one vendor.
- BraintrustEvals are the main job and you want the whole team scoring experiments without paying per seat.
- HeliconeYou need a gateway anyway and want logging from changing one base URL.
- Arize PhoenixYou want to start tracing on your laptop and move to a managed platform later on the same OpenTelemetry setup.
- W&B WeaveYour team already tracks models in Weights & Biases and wants agent traces in the same place.
- OpikYou want the whole platform, backend included, under Apache-2.0, with evals that run in CI.
- LaminarYour agent drives a browser, or you want plain-English alerts on what it does in production.
- LatitudeYou want failures to show up as tracked issues, with a coding agent sent to fix them.
Each listing.
The directory page, alternatives in the same category, and a head-to-head where we have one.
- LatitudeAlternatives
- LangfuseAlternatives
- LangSmithAlternatives
- Arize PhoenixAlternatives
- BraintrustAlternatives
- HeliconeAlternatives
- W&B WeaveAlternatives
- OpikAlternatives
- LaminarAlternatives
Category pages list every tool in that category, including ones left off this page. Built one of these? Get it featured in its category.
The 2 biggest AI stories, every morning.
One short email: the two stories that matter, and why.
You're in.
The Daily AI Brief will land in your inbox soon. If you don't see it, check Promotions or Spam and move it to your main inbox.