# LLM observability and eval tools

> Traces tell you what your LLM app did. Evals tell you whether it was any good. You need both: traces to debug a bad answer in production, datasets and scorers so the same mistake doesn't come back after the next prompt change. Every tool here does both. Langfuse, Helicone, Opik, Laminar and Latitude ship under open licenses you can self-host, Phoenix is source-available, Weave opens its SDK but not its server, and Braintrust and LangSmith self-host only on Enterprise. Each one meters something different (units, traces, spans, requests, GB, credits), so price it on your own volume.

9 tools, in the order they appear in the Digidle Tools directory. This is not a ranking.

- Page: https://digidle.com/best/llm-observability-evals/
- All best-of lists: https://digidle.com/best/

## Tools

- [Latitude](https://digidle.com/tools/latitude/): Open-source (MIT) observability and quality platform for AI agents: traces production runs, groups failures into tracked issues with evals attached, dispatches your coding agent to fix them and replays the fix against the failing traces. MCP server and CLI. Free Starter plan with 20K credits a month; Pro $99 a month. Official site: https://latitude.so. Alternatives: https://digidle.com/alternatives/latitude/.
- [Langfuse](https://digidle.com/tools/langfuse/): Open-source LLM engineering platform: traces and sessions for LLM apps and agents with token and cost tracking, prompt management, datasets, LLM-as-judge and human evals; OpenTelemetry-based, self-host free or use Langfuse Cloud in the US, EU or Japan. Official site: https://langfuse.com. Alternatives: https://digidle.com/alternatives/langfuse/.
- [LangSmith](https://digidle.com/tools/langsmith/): LangChain's platform for agents: tracing and online and offline evals for any framework, plus managed agent deployment, sandboxes and an LLM gateway; cloud in the US or EU, hybrid or self-hosted on Enterprise. Official site: https://www.langchain.com/langsmith/observability. Alternatives: https://digidle.com/alternatives/langsmith/.
- [Arize Phoenix](https://digidle.com/tools/arize-phoenix/): Arize's AI observability platform, source-available under the Elastic License 2.0: OpenTelemetry tracing through OpenInference for OpenAI, Anthropic, LangGraph, the Vercel AI SDK and more, with LLM evals, datasets, experiments and a prompt playground; runs locally, in Docker or on Kubernetes. Official site: https://arize.com/phoenix/. Alternatives: https://digidle.com/alternatives/arize-phoenix/.
- [Braintrust](https://digidle.com/tools/braintrust/): Observability and evals platform for AI agents: trace production calls from OpenAI, Anthropic, Gemini, the Vercel AI SDK, LangChain and more, run experiments scored by LLM judges, Autoevals or your own code, review in playgrounds and human annotation queues, and let the built-in Loop agent generate test cases and build evals. MCP server and a bt CLI. Free Starter plan with unlimited users; Pro $249 a month. Official site: https://www.braintrust.dev. Alternatives: https://digidle.com/alternatives/braintrust/.
- [Helicone](https://digidle.com/tools/helicone/): Open-source (Apache-2.0) AI gateway and LLM observability platform: point an OpenAI-compatible client at one URL to reach 100+ models with routing and automatic fallbacks, and every request is logged with cost, latency, sessions and agent traces. Prompt versioning, a playground, and scores and datasets for evaluating outputs. Self-host with Docker Compose or use the cloud, free for 10,000 requests a month. Official site: https://www.helicone.ai. Alternatives: https://digidle.com/alternatives/helicone/.
- [W&B Weave](https://digidle.com/tools/wandb-weave/): Weights & Biases' toolkit for tracing and evaluating AI agents: sessions, turns, tool calls and sub-agents as first-class traces, an evaluation framework with comparisons and leaderboards, built-in signals with Slack alerts, guardrail scorers for toxicity, PII and hallucinations, a playground, and an MCP server so coding agents can read traces and run evals. SDK is open source (Apache-2.0); free plan with 1 GB of trace data a month, Pro from $60 a month. Official site: https://wandb.ai/site/weave/. Alternatives: https://digidle.com/alternatives/wandb-weave/.
- [Opik](https://digidle.com/tools/opik/): Comet's open-source (Apache-2.0) LLM observability and evaluation platform: traces for agents and multi-turn conversations, datasets and experiments with 30+ LLM-as-a-judge and heuristic metrics, pass/fail test suites, a PyTest integration for CI, online evaluation rules on production traces, guardrails and an agent optimizer. The full platform, backend included, self-hosts free; Opik Cloud is free for 25K spans a month. Official site: https://www.comet.com/site/products/opik/. Alternatives: https://digidle.com/alternatives/opik/.
- [Laminar](https://digidle.com/tools/laminar/): Open-source (Apache-2.0) observability for AI agents, written in Rust: one-line OpenTelemetry tracing for the Vercel AI SDK, Browser Use, Stagehand, LangChain, OpenAI, Anthropic and more, Signals that flag behaviour you describe in plain English, evals from an SDK and CLI, browser session recordings, SQL over traces and MCP access for coding agents. Self-host with Docker Compose, or use the cloud, free for 1 GB of data. Official site: https://laminar.sh. Alternatives: https://digidle.com/alternatives/laminar/.

## At a glance

| Tool | What it is | Runs on | Open source | Price |
| --- | --- | --- | --- | --- |
| [Latitude](https://digidle.com/tools/latitude/) | Observability and quality for agents: traces production runs, groups failures into tracked issues with evals attached and dispatches your coding agent to fix them. | Latitude's cloud, or self-hosted from the repo. Custom on-prem or cloud deployment on Enterprise. | Yes, MIT. | Starter free: 20K credits a month, 30-day retention, unlimited seats. Pro $99/mo: 100K credits, 90 days, then $20 per 10K credits. Enterprise custom. |
| [Langfuse](https://digidle.com/tools/langfuse/) | Open-source LLM engineering platform: traces, sessions and cost tracking, prompt management, datasets, LLM-as-a-judge and human evals. | Langfuse Cloud in the US, EU or Japan, or self-hosted with Docker Compose or Kubernetes. | Yes, MIT core. Enterprise directories carry a separate license. | Hobby free: 50K units a month, 30 days of data, 2 users. Core $29/mo and Pro $199/mo with 100K units, then $8 per 100K (less at volume). Enterprise $2,499/mo. Self-hosting is free. |
| [LangSmith](https://digidle.com/tools/langsmith/) | LangChain's tracing plus online and offline evals for agents on any framework, with deployment, sandboxes and an LLM gateway alongside. | LangChain's cloud in the US or EU. Hybrid or fully self-hosted on Enterprise. | No. | Developer free: 1 seat, 5K base traces a month. Plus $39 per seat a month with 10K base traces, then pay as you go. Base traces keep 14 days, extended 180. Enterprise custom. |
| [Arize Phoenix](https://digidle.com/tools/arize-phoenix/) | Tracing on OpenTelemetry through OpenInference, with evals, datasets, experiments and a prompt playground. Arize AX is the managed platform on top. | Your laptop, Docker or Kubernetes. Arize AX in Arize's cloud (US, EU or Canada), or self-hosted on Enterprise. | Source-available under the Elastic License 2.0, not an OSI license. | Phoenix is free to self-host. AX Free: 25K spans and 1 GB a month, 15-day retention. AX Pro $50/mo: 50K spans, 10 GB, 30 days. Enterprise custom. |
| [Braintrust](https://digidle.com/tools/braintrust/) | Tracing and evals for agents: experiments scored by LLM judges, Autoevals or your own code, playgrounds, human review and a Loop agent that builds evals. | Braintrust's cloud. On-prem or hosted deployment on Enterprise. | No. Its Autoevals scorer library is MIT. | Starter free: unlimited users, 1 GB processed data, 10K scores, 14-day retention, $10 model credit. Pro $249/mo: 5 GB and 50K scores, then $3/GB and $1.50 per 1K scores, 30 days. Enterprise custom. |
| [Helicone](https://digidle.com/tools/helicone/) | AI gateway and observability in one: 100+ models behind one OpenAI-compatible API, with every request logged for cost, latency, sessions and traces. Prompts, scores and datasets. | Helicone's cloud, or self-hosted with Docker Compose. Helm chart and on-prem through Enterprise. | Yes, Apache-2.0. | Hobby free: 10K requests, 1 GB, 1 seat, 7-day retention. Pro $79/mo and Team $799/mo, plus usage-based pricing past 10K requests and 1 GB. Enterprise: contact sales. |
| [W&B Weave](https://digidle.com/tools/wandb-weave/) | Weights & Biases' tracing and evals for agents: sessions, turns and sub-agents as traces, eval comparisons, leaderboards, signals, guardrail scorers and a playground. | W&B's cloud, now sold as CoreWeave Forge plans. Self-managed on your Kubernetes with a Weave-enabled W&B license. | SDK yes, Apache-2.0. Self-managed server needs a commercial license. | Free: 1 GB of Weave data a month. Pro from $60/mo for teams under 50 people: 1.5 GB, then $0.10/MB. Enterprise custom. |
| [Opik](https://digidle.com/tools/opik/) | Comet's observability and evals platform: agent and conversation traces, datasets, experiments, 30+ built-in metrics, test suites, online evals, guardrails and prompt optimization. | Self-hosted from the repo, or Opik Cloud in the US. | Yes, Apache-2.0, the full platform including the backend. | Self-hosting free. Free Cloud: 25K spans a month, 60-day retention. Pro Cloud $19/mo: 100K spans, then $5 per 100K. Enterprise custom. |
| [Laminar](https://digidle.com/tools/laminar/) | Agent observability written in Rust: OpenTelemetry traces, Signals that flag behaviour you describe in plain English, evals from SDK or CLI, browser session recordings, SQL over traces. | Laminar's cloud, or self-hosted with Docker Compose. On-prem on Enterprise. | Yes, Apache-2.0. | Free: 1 GB, 7 days of data, 1 project, no Signals. Starter $30/mo: 3 GB, then $2/GB. Pro $150/mo: 10 GB, then $1.50/GB, 6 months. Signals billed per token. Enterprise custom. |

Checked 2026-11-01 on vendor pages: [Langfuse pricing](https://langfuse.com/pricing), [Langfuse on GitHub](https://github.com/langfuse/langfuse), [LangSmith pricing](https://www.langchain.com/pricing), [Braintrust pricing](https://www.braintrust.dev/pricing), [Braintrust docs](https://www.braintrust.dev/docs), [Helicone pricing](https://www.helicone.ai/pricing), [Helicone on GitHub](https://github.com/Helicone/helicone), [Arize pricing](https://arize.com/pricing/), [Phoenix on GitHub](https://github.com/Arize-ai/phoenix), [W&B pricing](https://wandb.ai/site/pricing/), [Weave on GitHub](https://github.com/wandb/weave), [Weave self-managed docs](https://docs.wandb.ai/weave/guides/platform/weave-self-managed), [Opik pricing](https://www.comet.com/site/pricing/), [Opik on GitHub](https://github.com/comet-ml/opik), [Laminar pricing](https://laminar.sh/pricing), [Laminar on GitHub](https://github.com/lmnr-ai/lmnr), [Latitude pricing](https://latitude.so/pricing), [Latitude on GitHub](https://github.com/latitude-dev/latitude-llm).

## Pick it if

- [Langfuse](https://digidle.com/tools/langfuse/): You want open source you can run yourself for free, with prompts and evals in the same app as your traces.
- [LangSmith](https://digidle.com/tools/langsmith/): You build on LangChain or LangGraph and want tracing, evals and deployment from one vendor.
- [Braintrust](https://digidle.com/tools/braintrust/): Evals are the main job and you want the whole team scoring experiments without paying per seat.
- [Helicone](https://digidle.com/tools/helicone/): You need a gateway anyway and want logging from changing one base URL.
- [Arize Phoenix](https://digidle.com/tools/arize-phoenix/): You want to start tracing on your laptop and move to a managed platform later on the same OpenTelemetry setup.
- [W&B Weave](https://digidle.com/tools/wandb-weave/): Your team already tracks models in Weights & Biases and wants agent traces in the same place.
- [Opik](https://digidle.com/tools/opik/): You want the whole platform, backend included, under Apache-2.0, with evals that run in CI.
- [Laminar](https://digidle.com/tools/laminar/): Your agent drives a browser, or you want plain-English alerts on what it does in production.
- [Latitude](https://digidle.com/tools/latitude/): You want failures to show up as tracked issues, with a coding agent sent to fix them.

---
Source: Digidle Tools (https://digidle.com/). Full index: https://digidle.com/llms.txt