What It Costs to Monitor an AI Agent

An AI agent’s monitoring bill depends on what its platform counts. One run can create a trace, several recorded steps, and multiple evaluation scores. For an illustrative 100,000-run month with five observations and two scores per evaluated run, the Langfuse Cloud Core meter would count 800,000 units if everything were recorded and scored. At its published Core rates, that is $85 for the platform. Recording one in ten runs and scoring one in ten recorded runs would produce an expected 62,000 units and a $29 Core fee. Neither figure includes model calls, judge-model tokens, taxes, or other services. They are calculations, not invoices.

Compare what each platform counts

The same application traffic cannot be priced by applying one vendor’s unit count to another vendor’s rate. A trace can hold many steps. Some services count traces, some count spans or individual data points, and others count bytes. The table uses each maker’s published plan page, read on November 9, 2026. An allowance is a starting point, not proof that a workload fits it.

Platform and official source	Published entry meter and allowance	Cost and history detail to check
Langfuse Cloud	Hobby includes 50,000 units/month; Core includes 100,000. A trace, observation, or score is one unit.	Core is $29/month plus graduated usage, beginning at $8 per 100,000 units above the allowance. Core’s 90-day data access is not automatic deletion.
LangSmith	Developer includes 5,000 base traces/month with one seat; Plus includes 10,000. One trace may contain many steps.	Plus is $39 per seat/month, then usage charges. Base traces have 14-day retention; extended traces have 180 days and an additional fee.
Braintrust	Starter includes 1 GB of processed data and 10,000 scores.	Starter is $0/month, then $4/GB and $2.50 per 1,000 scores on the published page. It lists 14-day retention; model credits and token rates are a separate line.
Arize AX	AX Free includes 25,000 spans and 1 GB ingestion/month; AX Pro includes 50,000 spans and 10 GB.	AX Pro is $50/month with 30-day retention and unlimited users and evals. A span limit and a byte limit both need checking.
Comet Opik Cloud	Free Cloud includes 25,000 spans/month; Pro Cloud includes 100,000.	Pro Cloud is $19/month, up to 50 members, with 60-day data retention; its page describes customizable span limits. Confirm the applicable limit before assuming automatic overage.
Datadog Agent Observability	Bills LLM spans: one model request can be one span, and an agent run can make several requests.	The published list shows $160/month for the first 100,000 LLM spans on annual terms with 15-day retention, plus rates for further spans and longer retention. Contract terms change the price.
W&B Weave on CoreWeave Forge	Free includes 1 GB/month of Weave ingestion; Pro includes 1.5 GB/month.	Pro starts at $60/month and lists $0.10/MB of additional ingestion. Logged inputs, outputs, and metadata affect the byte count; a fixed trace size is not published.

Langfuse is a useful example of why a single run is not a single billable event: its unit definition counts the trace, each observation, and each score. Its Hobby allowance may cover small workloads at no platform fee, but the Core calculation below uses the Core plan’s allowance and rate. Check the plan that actually fits your users, history needs, and throughput before comparing totals.

LangSmith’s pricing page describes a trace as one application execution that can contain multiple steps. Its base-trace allowance is therefore not interchangeable with Arize’s or Opik’s span allowance. Plus also adds a per-seat charge, so the number of people who need access changes its fixed cost. LangSmith documents SDK trace sampling, including global and per-client rates; that guide says the control does not cover direct API logging in the same way.

Braintrust’s pricing page makes two volume lines explicit: processed GB and scores. A team that sends verbose prompts and outputs may hit the data line before its score allowance; a team that runs many evaluators could hit the score line first. Its model credits and token rates should be budgeted separately from those two meters.

Arize AX’s pricing page publishes both monthly spans and ingestion GB. Count the emitted spans and measure payload size, then check both limits. The listed retention window and unlimited users/evals address different questions; an evaluation can still call an external judge model with its own token bill.

Opik’s pricing page gives a straightforward hosted span allowance and a 60-day retention period. It also offers open-source self-hosting, but a $0 software license is not a $0 infrastructure budget. For a hosted purchase, ask what happens at the configured span limit rather than treating a plan allowance as a published per-span overage rate.

Datadog’s Agent Observability documentation says billing follows ingested LLM spans, where a span represents an LLM-provider request. The price list has separate 15-, 30-, 60-, and 90-day retention rates and contract columns. Count model calls in each agent workflow, then use the correct retention and contract column. A general APM span price would be the wrong meter for this product.

Weave’s Forge pricing page defines ingestion from the bytes it receives and stores, including trace metadata and logged inputs and outputs. Export or measure real payload bytes. Multiplying a trace count by a guessed constant size can miss the effect of long prompts, attachments, or added metadata.

Turn agent runs into a monitoring budget

Start with a monthly ledger, not a vendor price table. Let N be agent runs, p the fraction of runs recorded as traces, o the average observations inside each recorded trace, q the fraction of recorded traces that receive evaluations, and k the average scores per evaluated trace. For a Langfuse-style unit definition, expected units are:

N × p × (1 + o + q × k)

The 1 is the trace itself. The next term counts recorded observations, such as model generations, tool spans, and events. The final term counts scores. This is an expectation using averages, not a guaranteed invoice. Inspect how your SDK emits root and nested spans, retries, failed sends, and scores; otherwise o and k are guesses. For a GB-based service such as Braintrust or Weave, replace the unit formula with measured processed or ingested bytes, following that vendor’s definition.

Here is a fully hypothetical month with 100,000 runs, five observations per recorded run, and two scores per evaluated run. The published Langfuse Core plan is $29/month, includes 100,000 units, and charges $8 per 100,000 units from 100,000 through 1 million. The arithmetic assumes the example emits exactly the stated events and remains in that first overage band.

Trace sample p	Eval sample q of recorded traces	Expected units: N × p × (1 + 5 + q × 2)	Illustrative Core platform fee
100%	100%	800,000	$29 + (700,000 / 100,000 × $8) = $85.00
100%	10%	620,000	$29 + (520,000 / 100,000 × $8) = $70.60
10%	10%	62,000	$29.00, below Core’s included units

Changing q cuts score events but leaves traces and observations. Changing p cuts everything sent for unsampled runs. Neither is a hard spend cap: actual traffic, observations per run, and what the SDK sends can change. These figures price only Langfuse Cloud Core platform units. A judge model called for an evaluation may add inference tokens; our AI API cost calculator helps with that separate model bill. Our API price comparison explains why model choice and token mix affect it.

Sample each workflow deliberately

Different sampling rates across workflows can make the recorded mix differ from production traffic. Suppose, only as an example, 80,000 monthly runs use a simple workflow and 20,000 use a complex workflow. Recording 10% of simple runs and 50% of complex runs yields 8,000 simple traces and 10,000 complex traces. Complex work is 20% of production runs but about 56% of recorded traces. A pooled failure percentage from those traces would describe the sampled mix, not the production mix. A lower pooled failure rate after changing sampling is not evidence that the agent improved.

Keep counts by workflow, record each sampling probability, and compare failure rates within the same workflow before combining them. If you need a production-wide estimate, weight each workflow result by its production run count and account for the sampling design. Preserve enough coverage of rare or costly failure paths to investigate them. LangSmith’s sampling guide documents per-client rates for its SDK and distinguishes conditional tracing; other vendors’ controls need their own documentation check. Our rate-limit guide shows why traffic shape matters even when a monthly average appears safe.

Separate history access, deletion, and evaluation cost

The word retention can hide a material difference. Langfuse says Hobby permits 30 days of data access and Core 90 days. Older data remains stored and becomes accessible after an upgrade. Permanent deletion after a custom period is a separate opt-in feature on Pro and Enterprise. Do not promise deletion after a Core access window. By contrast, LangSmith’s pricing page calls its 14-day and 180-day periods trace retention. Check each vendor’s deletion policy and contract before using these periods for a privacy requirement.

An evaluation has at least two possible bills: a platform score or evaluation meter, and the model or service that performs the judgment. Langfuse counts scores as units; Braintrust lists scores separately; Arize’s plan says evaluations are unlimited within its plan description. None of those lines proves that external judge-model tokens are free. If you select a judge model, consult its maker’s token price and record the prompt and output token mix. The model tracker can help identify models to compare, while the maker’s current price remains the billing source.

Check the real usage export before choosing

Run a small representative workload and reconcile the counts before committing to a plan. Record production runs by workflow, traces accepted by the platform, observations or spans per trace, scores emitted, and processed or ingested bytes where relevant. Check SDK flush and export errors so a low bill is not mistaken for complete monitoring. Then inspect the vendor’s billing dashboard for allowance, overage, plan cap, seat count, and history behavior. These are steps to perform; this article does not claim we tested any of the seven products.

If the agent also runs paid sandboxes, budget their execution and saved state separately. The monitoring ledger should stay beside the inference, evaluation-model, and execution ledgers. That separation makes a plan comparison meaningful when each provider meters a different part of the same run.

Sources

• Langfuse pricing: plans, billable units, graduated rates, data access, and opt-in deletion.
• LangSmith pricing and trace sampling documentation: traces, seats, retention, and SDK sampling scope.
• Braintrust pricing: processed data, scores, model credits, and retention.
• Arize AX pricing: spans, ingestion volume, users, evaluations, and retention.
• Comet Opik pricing: hosted span allowances and retention.
• Datadog price list and Agent Observability documentation: LLM-span meter and contract/retention rates.
• CoreWeave Forge pricing: Weave ingestion allowances and byte definition.

All pages read on November 9, 2026.

Free toolWhat will 10,000 chats a month cost?Estimate my bill