When Batch AI APIs Actually Save Money

Batch AI APIs can cut the token price of work that can wait, but a lower rate is only useful if the job fits the provider’s model, size and deadline rules. OpenAI, Anthropic, Google, Mistral and Groq advertise a 50% batch discount; Together says up to 50% for most serverless models. A batch is asynchronous: submit independent requests, collect results later, and budget separately for anything that fails or becomes urgent. A 10,000-request example below costs $1.80 at standard Claude Haiku 5.5 rates, $0.90 if every request finishes in a batch, or $0.945 if 5% must instead use standard processing. Those figures are illustrative calculations from Anthropic’s current rates, not a measured production bill.

What batch processing changes in the bill

With a normal API call, your application waits for a response. With batch processing, it submits a file or group of independent calls and retrieves results after the service processes them. That makes batch suitable for overnight classification, document tagging and offline evaluations. It is a poor fit for a chat turn that a person is waiting on, or for a sequence where request B needs request A’s answer. OpenAI’s Batch guide describes the separate queue, 50% discount and 24-hour turnaround; Google’s guide calls 24 hours a target, not a guaranteed finish time.

The advertised percentage applies to eligible work on the specified API. It does not mean every model and modality has the same dollar rate. Input and output tokens can have different prices, long prompts can move into a higher tier, and cached input may follow another rule. Start with a particular model and endpoint, then compare its standard and batch prices on the maker’s page. Our standard API price comparison covers model selection; this guide is about whether the same workload should enter a batch queue.

Compare six batch APIs on the same terms

The table describes direct platform APIs, not reseller prices. A model is an example explicitly named on a maker’s page, not a promise that your account can use it. “Not stated” means the cited public pages did not establish a general rule; check the current account limits and model page before submitting. No inference jobs were run for this article.

API and documented model example	Batch price	Submission limit	Completion rule	Failed or unfinished work; cache rule
OpenAI Batch, GPT-6 Luna on its pricing page	50% below standard for eligible endpoints	50,000 requests and 200 MB per file; model queue quota varies	24-hour turnaround	Completed requests consume tokens and are billed; unfinished requests expire into an error file. Check the selected model’s cached-input rate separately.
Anthropic Message Batches, Claude Haiku 5.5	50% below standard input and output rates	100,000 requests or 256 MB, whichever comes first	Batches expire after 24 hours; most complete within an hour, but that is not a guarantee	Results identify succeeded, errored, cancelled and expired requests. Prompt caching can stack, though hits are best effort in asynchronous work.
Google Gemini Developer API Batch, Gemini 4 Argon in pricing	50% of standard equivalent-model cost	2 GB input file; per-model queued-token limits and concurrent-job limits in rate limits	24-hour target, with runtime varying by load and size	Inspect failed-request counts and per-line errors. Cache support and rates depend on the model: Gemini 4 Argon supports implicit caching only, and its pricing page lists a separate Batch cache rate.
Mistral Batch Processing, mistral-small-latest in its example	50% discount	Its known-limitations page says 100,000 requests and 512 MB; its batch guide also describes larger uploads, so confirm the active API limit	No fixed completion guarantee stated in the cited guide	Job results expose success and failure counts; a general charge rule for failed requests and a batch-cache stacking rule were not established here.
Groq Batch, llama-3.1-8b-instant	50% below synchronous price	50,000 JSONL lines, 200 MB in its batch guide	Choose a processing window from 24 hours to seven days; longer windows reduce expiry risk	On expiry, only successfully completed requests are charged. The batch and prompt-cache discounts do not stack.
Together AI Batch, DeepSeek family among its named examples	Up to 50% off for most serverless models; check the exact model	Its current public page states up to 30 billion enqueued tokens per model; it does not specify a universal file-request cap	The current page advertises a sub-24-hour processing SLA; check the contract that applies to your account	Its Batch API announcement describes billing successful completions; the current page does not specify a cache-stacking rule.

These are six operational choices, not six equivalent model-quality choices. Compare answer quality on your own permitted evaluation set before changing models. Also check regional availability, data handling and output format if those matter to your workflow. The price table gives a procurement starting point, not an account-specific quote.

Price one offline workload before submitting

Illustrative scenario: 10,000 independent text-classification requests, each with 800 input tokens and 200 output tokens. That is 8 million input tokens and 2 million output tokens. The reusable formula is:

Cost = (input tokens ÷ 1,000,000 × input rate) + (output tokens ÷ 1,000,000 × output rate).

For Claude Haiku 5.5 prompts below 100,000 tokens, Anthropic lists $0.10 per million standard input tokens and $0.50 per million standard output tokens. Its batch table lists $0.05 and $0.25. Standard cost is 8 × $0.10 + 2 × $0.50 = $1.80. If all 10,000 requests finish through Batch, the comparable batch cost is 8 × $0.05 + 2 × $0.25 = $0.90. The $0.90 difference is the simple all-success case; it excludes other account charges and any change in actual token usage.

To reproduce the worksheet for another model, replace all four rates with its current standard and batch figures. Do not multiply a standard quote by 0.5 when the maker publishes a different applicable tier, a promotion, a long-context surcharge or model exclusions. The AI API cost calculator can help estimate a base workload; apply the batch rates and fallback calculation yourself because this article has not verified that the tool models batch jobs automatically.

Budget for failures and a deadline fallback

Suppose, only as an example, that the batch has ended after completion, expiry or finalized cancellation: 9,500 requests succeeded and 500 have a terminal non-success result with no billable usage. The completed portion has 7.6 million input and 1.9 million output tokens, costing 7.6 × $0.05 + 1.9 × $0.25 = $0.855 at the Haiku batch rates. Running the 500 non-success requests once at the standard rate adds 0.4 million input and 0.1 million output tokens: 0.4 × $0.10 + 0.1 × $0.50 = $0.09. The combined example is $0.945, a $0.855 saving versus sending the original 10,000 requests synchronously. This assumes the non-success requests consumed no billable input or output usage and that the same token counts apply on fallback; verify the actual result and usage records before treating it as a bill. If you send fallback requests while the original batch is still running or canceling, both copies may finish and be billed; the $0.945 example does not cover that overlap. Finalize cancellation and reconcile all results before retrying, or budget for duplicate work when the deadline cannot wait.

The result file matters more than the batch status label. OpenAI says an expired job retains completed responses and charges tokens consumed by completed requests. Groq says only successfully completed requests are charged after expiration. Anthropic’s guide distinguishes success, error, cancellation and expiry for each request; it is safer to reconcile usage than to assume a whole job either succeeded or failed. For Google, inspect batchStats and individual errors, as its guide instructs. For providers whose public billing language is less explicit, confirm it in the account billing record before automating retries.

Give each request a stable identifier using the provider’s schema: custom_id for OpenAI and Anthropic, or key for a Gemini JSONL request. Store the submitted input securely and match results by that identifier rather than file order. Mark each ID as complete, failed with a fixable input problem, or eligible for retry. Retry only the missing IDs. A blind resend can create a second billable job: Google says batch creation is not idempotent. If results are needed at 9 a.m., your internal cutoff should be earlier than 9 a.m. so that retrieval, validation and a standard-rate fallback still fit.

Check limits and cache rules before choosing

A 10,000-request file fits the published per-file request caps above, but count bytes and queued tokens too. Attachments and long prompts can hit a file limit before a request limit. Model-specific queue allowances can reject or delay a job even when the JSONL is valid. Break large workloads into independently identifiable chunks, then measure how many results each chunk actually returned. Mistral’s current documentation gives different batch-size signals, so an account’s live limit should settle that choice rather than a copied maximum.

Caching can change the comparison. Anthropic says its batch and prompt-cache discounts can stack, but cache hits are best effort because batch requests run asynchronously. Google’s general batch guide describes cached_content for supported models, but Gemini 4 Argon does not support explicit caching; use its Batch pricing table instead of applying the general guide’s standard-cache-rate wording. Groq explicitly excludes stacking. Do not assume one cache rule works on all six services. Our prompt-caching guide explains when repeated input earns a separate discount.

When standard processing is worth the extra cost

Use standard processing when a user needs the answer now, when a later call depends on an earlier one, or when you cannot tolerate the provider’s batch window. Batch is strongest when jobs are independent, the output can wait, and you have a result-reconciliation step. A hybrid schedule can send routine work to Batch and keep the last urgent slice on the standard endpoint. The break-even question is then concrete: do the batch savings on completed work exceed the standard-rate fallback and the operational work needed to manage results?

Recheck current model availability and prices immediately before a large run. Models and promotional rates change. The AI model tracker can help identify new releases, but the provider’s model and pricing pages decide whether a particular batch job is eligible and what it costs.

Sources

• OpenAI Batch guide and pricing: endpoints, limits, expiry and model rates.
• Anthropic Message Batches and pricing: batch rules and Haiku 5.5 example rates.
• Google Gemini Batch, pricing and rate limits: target, cache and quotas.
• Mistral Batch Processing and known limitations: discount and platform limits.
• Groq Batch: supported models, processing windows, expiry billing and cache rule.
• Together AI Batch Inference and Batch API announcement: current platform claims and the earlier billing description.

All pages read on October 29, 2026.

Free toolWhat will 10,000 chats a month cost?Estimate my bill