What Serverless GPUs Cost Between Requests

Serverless GPU cost depends on how long each worker exists, not just how long it produces an answer. A worker can incur charges while loading a model and waiting between requests. For example, Modal lists an L4 at $0.000222 per GPU second, before its separate CPU and memory charges; RunPod lists its L4 serverless class at $0.75 per worker hour and bills from worker start to stop. Those figures buy different configurations, so the useful question is how many billable worker seconds your own request pattern creates.

This guide compares six ways to run your own GPU workload, then prices the same twelve hypothetical jobs when they arrive close together or far apart. These are calculations from published rules, not a deployment test or a latency promise. Public model APIs that charge per image or token are a different product.

Compare the whole worker bill

Prices below are public list rates on November 8, 2026, in USD. GPU types and included resources differ. A raw rate does not tell you which platform completes your model fastest, and no identical-workload benchmark was run. Follow each linked maker page before committing spend.

Platform and product	Published GPU example	Other bill components	Startup, idle and warm behavior
Modal Functions	L4: $0.000222 per GPU second	CPU: $0.0000131 per physical core second; memory: $0.00000222 per GiB second	Application load and the post-input idle window are billable. A warm minimum can keep containers running; the configured scale-down window is a maximum, not a guaranteed lifespan.
RunPod Serverless	L4/A5000/3090/MIG 24GB class: $0.75 per worker hour	Container disk and network volumes have separate charges	Billing starts when a worker starts and ends when it fully stops, rounded up to a second. Flex workers scale to zero; active workers run continuously.
Replicate private models and deployments	H100: $0.001525 per instance second	The listed hardware configuration includes its stated CPUs and system RAM; check model-specific terms	Setup, active and idle instance time are billed. A deployment defaults to a minimum of zero instances, but can keep a floor. Public models and fast-booting fine-tunes use different rules.
fal Serverless	H100: $4.50 per runner hour at list price	Check the app and account terms for any other charges	Setup, idle, running, draining and teardown are billed. min_concurrency creates a warm floor; keep_alive controls idle life.
Cerebrium	L4: $0.000222 per GPU second	The page separately lists memory at $0.00000222 per GB second and storage at $0.05 per GB month after its stated allowance	Compute is metered in seconds. A definitive public idle-window or minimum-worker rule was not established from the pricing page; verify the deployment setting before estimating those seconds.
Google Cloud Run services with GPU	L4 without zonal redundancy: $0.0001867 per GPU second in the displayed Iowa instance-based table	In that table, CPU is $0.000018 per vCPU second and memory $0.000002 per GiB second	GPU services require instance-based billing: the GPU is charged for the entire instance lifecycle. Minimum instances cost the full rate while idle.

The RunPod row is a worker class that can assign several 24 GB card types, not a claim that they deliver identical speed. Modal, Cerebrium and Cloud Run expose separate resource lines; Replicate’s H100 hardware row is an instance package. fal’s “as low as” H100 rate requires a sales arrangement, so the table uses its $4.50 list rate. These distinctions matter more than a spurious cheapest-to-most-expensive ranking.

Storage, egress, regional multipliers, plan fees, credits, discounts and taxes may change the invoice. For example, RunPod’s billing documentation lists container disk in five-minute intervals and network-volume rates separately. A stopped worker can still leave persistent storage to pay for.

Count worker time without counting it twice

Build a timeline for each worker, from its billed start to its billed stop. Mark setup, request processing, idle and shutdown periods. The billable duration is the length of that worker’s alive interval. Add durations across separate workers, including workers that overlap in clock time. Do not attach a full idle timeout to every request on the same worker: when the next request arrives during an existing idle window, it reuses that window.

Actual start and stop records beat a model. If a provider ends a worker early, starts a second one, rounds seconds, or holds a warm minimum, the real bill can differ. Modal says its autoscaler may stop a container before the configured scaledown_window; RunPod rounds each worker lifetime up to the next second. The formulas below describe a deliberately simple policy, not either platform’s precise scheduler.

For one sequential worker with a fixed idle timeout, count the first setup once. After each completed job, the next gap contributes at most the idle timeout. If the gap exceeds that timeout, the worker stops and a later job starts a new session with another setup. Add one final idle window after the last job. A useful ledger has one row per worker session: start | ready | requests completed | last completion | stop | rounded billable seconds.

The model assumes one GPU per worker, one request processed at a time, a fixed ten-second service time, no parallel input, no failed job, no additional worker spun up by a queue and the same hardware throughout. Those assumptions keep the arithmetic inspectable. Your queue and scaling settings may produce more than one worker; then make a separate timeline for each.

Same requests, different bill

Suppose twelve jobs arrive at times 0, 15, 30 and so on, or at 60-second or 600-second intervals. Each job requires ten seconds of processing. A cold worker takes 20 seconds from billed start to ready and stops 60 seconds after its last completion unless another job arrives. A job arriving during startup or processing queues for this single worker. The 20 and 60 seconds are illustrative choices, not measured provider startup or idle behavior.

Arrival spacing	Cold starts	Productive seconds	Setup seconds	Idle seconds, counted once per worker	Total billable seconds
15 seconds	1	120	20	95	235
60 seconds	1	120	20	590	730
600 seconds	12	120	240	720	1,080

Here is the check behind the first row. The first job arrives at time 0, the worker is ready at 20 and finishes it at 30. The job that arrived at 15 is queued, then finishes at 40. Later arrivals fill or partly fill the gaps; the twelfth arrives at 165, finishes at 175, and the worker stops at 235. Its single alive interval is 235 seconds: 20 setup + 120 processing + 95 idle. Assigning 60 idle seconds to every one of twelve jobs would count overlapping time repeatedly.

At 60-second spacing, the first worker stays alive: the last arrival is at 660, the last completion at 670 and shutdown at 730. At 600-second spacing, every gap exceeds the timeout, so twelve separate 90-second lifetimes yield 1,080 seconds. Clustering reduces the hypothetical worker bill, although real queues can worsen response time or trigger extra workers.

To turn those seconds into a transparent price example, assume a Modal L4 Function requests one physical CPU core and 4 GiB of memory for all billed seconds. Its published component rates give $0.000222 + $0.0000131 + 4 × $0.00000222 = $0.00024398 per worker second. Multiplying that assumed full rate by the three totals produces about $0.0573, $0.1781 and $0.2635 respectively. These figures exclude credits, storage, egress and any actual variation in CPU or memory metering. They are a worked pricing model, not an observed Modal invoice.

Find your break-even and reconcile the invoice

An always-on alternative has a simple daily compute baseline: 24 × hourly full-worker price. For a comparable serverless configuration, divide that daily amount by the full serverless rate per billable second. The result is a threshold of billed seconds, not useful processing seconds. If the 12 jobs above were repeated, cold setup and idle time would move that threshold even though the productive 120 seconds never change.

For a starting estimate, use your measured distribution of arrival gaps, startup time, service time and peak concurrency. Specify a latency target before shortening the idle timeout: shorter windows can save idle charges but cause more cold starts and longer waits. A warm floor buys responsiveness at a continuous cost. The VRAM sizing guide and VRAM calculator help select a card that can hold the model; neither predicts cloud billing. The model tracker helps identify the model before sizing it.

For an invoice check, export worker start, ready, request-start, request-end and stop timestamps, with a worker ID and endpoint ID. Merge events by worker, round according to that provider’s billing rule, then compare your computed seconds with the usage export. Separately account for storage, traffic, minimum workers and any CPU or RAM lines. Failed or repeated requests can consume billed worker time; do not divide by attempted requests if your business metric is cost per successful response. Replicate explicitly describes different treatment for public models versus private deployments, including failed runs, in its billing documentation. Check the rule for the product you actually bought.

This worker calculation differs from AI API token pricing and batch API discounts. Those products charge for provider-hosted model usage under their own terms; a deployed GPU worker can bill for time between requests. Compare them only after fixing the model, output quality, useful throughput and deadline you need.

Choose a worker policy before a provider

First fit the model and its runtime memory into a GPU. Then measure the busiest period, not only the daily average. Decide how long a user may wait for a cold start and whether a warm minimum is worth its daily cost. Finally, run the same useful workload on candidate configurations and compare cost per successful output, including every billed worker and resource. No published GPU-second price proves a universal cheapest provider or equivalent throughput.

For intermittent work, scale-to-zero plus a short idle window may be attractive if the added startup wait is acceptable. For frequent arrivals, one reused worker may avoid repeated setup, but long idle windows can dominate the bill. For steady demand, compare an always-on option with the full measured serverless invoice. The right setting follows the traffic timeline, not the headline GPU rate.

Sources

• Modal pricing and autoscaling guide: GPU, CPU and memory rates; billable idle time and scale-down controls.
• RunPod pricing and Serverless billing: worker class rates, lifetime, rounding and storage.
• Replicate pricing and billing: private deployment hardware and instance lifecycle.
• fal pricing and production Serverless guide: H100 list rate and runner states.
• Cerebrium pricing: GPU, memory and storage rates.
• Cloud Run pricing and GPU service rules: regional components and instance-based billing.

All pages read on November 8, 2026.

Free toolCan my GPU run this model?Check my GPU