GPU cloud pricing is wider than most buyers expect. PetaPrice's November 14, 2026 report showed on-demand H100 SXM rates from $1.95 to $12.30 per GPU-hour across 32 providers, with a $3.70 median. The gap between the cheapest and the dearest listing is more than 6x for the same chip. The sticker rate is only the start: billing model, idle time, storage and data transfer decide what lands on your invoice.
This guide shows what the main GPU models cost, how each pricing model works, and how to turn a rate into a monthly budget for your workload.
TL;DR
- On-demand H100 SXM ranged from $1.95 to $12.30 per GPU-hour across 32 providers in PetaPrice's November 14, 2026 report, with a $3.70 median.
- Pick the billing model by workload shape: serverless for bursty traffic, on-demand for development and training, reserved for steady always-on demand, spot for fault-tolerant batch jobs.
- Reserved capacity only beats on-demand when you actually use more than reserved price divided by on-demand price of the hours you pay for.
- The hourly GPU rate excludes storage, egress and taxes. Divide multi-GPU instance prices by GPU count before comparing.
- Estimate from your own usage data (hours, utilization, data moved), not from a headline rate.
Table of Contents
- What a GPU-hour price actually covers
- Cloud GPU prices by model, 2026
- On-demand, reserved, spot and serverless: how each is billed
- The break-even rule for reserved capacity
- Costs outside the hourly rate
- How to estimate your own GPU cloud bill
- Comparing providers fairly
- Our view: price the result, not the hour
- Choosing the right model for your workload
- Where to go next
- FAQ
- Sources
What a GPU-hour price actually covers
A per-GPU-hour price is the cost of one GPU for one hour. That sounds simple, but GPU Ledger's glossary points out that the figure usually excludes storage, network transfer, taxes and negotiated discounts. It also notes that an instance price has to be divided by its GPU count to get a per-GPU figure, while some providers quote the GPU component alone.
Billing increments matter too. Per-second billing charges you for the time used. Hourly billing can round a partial hour up. In the same glossary, twelve of 29 providers published per-second billing. If you run many short jobs, such as test runs or small fine-tunes, rounding can add a meaningful share to your bill.
Three things to check on any pricing page before you compare it with another:
- Is the price per GPU or per whole instance?
- What is the billing increment (second, minute, hour)?
- Which costs sit outside the rate: disk, egress, support, managed networking?
Cloud GPU prices by model, 2026
Prices move often, so read these as published snapshots for their dates, not as today's rate. Each provider and aggregator updates on its own schedule, and supply, new chip generations and demand shift them.
| GPU | Published figure | Source and period |
|---|---|---|
| H100 SXM, on-demand | $1.95 to $12.30 per GPU-hour, $3.70 median, 32 providers | PetaPrice, November 14, 2026 |
| H100, on-demand and spot | $1.99 on-demand, spot from $1.25 (interruptible) | GlobeGPU index, November 14, 2026 |
| H200, on-demand | $2.09 low, $4.44 median, $10.60 high, 46 providers | American Compute, October 2026 |
| B200, on-demand | $3.69 low, $6.82 median, $14.24 high, 31 providers | American Compute, October 2026 |
| L40S, on-demand | $0.70 to $3.50, $1.50 median, 23 providers | PetaPrice, November 14, 2026 |
Three patterns stand out.
Hyperscalers sit at the top of the range. Compute Suppliers' November 2026 break-even table lists Azure ND H100 v5 at $12.29 per GPU on-demand, against Together AI's H100 at $3.99. Hyperscalers bundle ecosystems, compliance tooling and regional coverage, which may justify a premium if you already depend on them.
The median is a better anchor than the low. The lowest listing often comes with tighter availability, fewer regions or interruption risk. The median tells you what a typical offer costs.
Newer chips cost more per hour but not always per result. A faster GPU that finishes a job in half the time can cost less overall. That is why cost per finished job beats cost per hour as a comparison.

On-demand, reserved, spot and serverless: how each is billed
Providers sell the same hardware under four billing models. GMICloud's pricing-models guide and Spheron's billing guide describe the trade-offs the same way.
On-demand
No term commitment, billed for runtime, usually at the highest per-hour rate. The premium pays for flexibility: the provider carries the risk of unsold capacity. Use it for development bursts and experiments where you do not know your usage yet.
Reserved
You commit to a term, from weeks to years, in exchange for a lower rate, and you pay for the whole term whether the GPU is busy or idle. Examples from Compute Suppliers' table: Together AI H100 at $3.69 to $3.19 depending on a 3 to 30, 31 to 90 or 91 to 180 day term, and Azure H100 at $7.87 for a one-year reservation. VendorBenchmark's January 2026 benchmark reports one-year reserved savings of about 40% on Azure ND H100 v5 and Google Cloud a3-highgpu, and 32% on AWS p5.48xlarge. CoreWeave's pricing page advertises up to 60% off on-demand for committed usage.
Spot or interruptible
Discounted capacity that the provider can take back. Vast.ai's pricing documentation says interruptible instances are often more than 50% cheaper than on-demand and may be paused. In Compute Suppliers' table, Together AI's H100 is $3.99 on-demand but $1.99 spot. Spot suits batch training, evaluation sweeps and anything that checkpoints and restarts cleanly.
Serverless
You pay per call or per second of execution and the endpoint scales to zero. Compute Suppliers lists serverless H100s at $3.95 per busy hour on Modal and $5.39 on RunPod. The hourly figure looks higher than on-demand, but you pay only while work runs. That wins on bursty traffic and loses once an endpoint is nearly always busy.
The break-even rule for reserved capacity
Reserving is a bet on utilization. Compute Suppliers states the rule: reserved beats on-demand only if the share of hours you actually use exceeds reserved price divided by on-demand price.
A worked example from its table: DigitalOcean H100 at $3.26 reserved against $4.41 on-demand gives 74%. That is roughly 540 hours of a 730-hour month. Together AI's 91 to 180 day H100 term comes to 80% (584 hours a month), and Hyperstack's H100 PCIe to 70% (511 hours). The same source notes six months of one Together AI H100 at $3.19 is a $13,972 commitment, so the downside of overestimating usage is concrete.
A practical rule of thumb from GPUaaS.com's July 2026 framework: spot for fault-tolerant batch work, on-demand for development bursts, and reserved for predictable inference once utilization is roughly 60 to 70% or higher.
How to apply it:
- Log your GPU hours for four weeks on on-demand or serverless.
- Divide hours used by 730 to get utilization.
- Compare it with the break-even percentage for the reserved offer you are considering.
- Reserve only the steady baseline and keep the peaks on-demand.
Costs outside the hourly rate
The GPU line is rarely the whole bill. Four additions show up most often.
Storage. Vast.ai's documentation notes that storage keeps billing on stopped instances until you delete it. A parked instance with a large disk is not free.
Egress. GPU Ledger says outbound data is generally charged per gigabyte, sometimes after a free allowance, and is not in the hourly figure. For inference serving large outputs, such as video, this can rival the compute cost. A Brazilian engineer's worked example priced one always-on GPU (730 hours) with 30 TB of monthly outbound traffic across four clouds, and showed how much the comparison hinges on egress.
Idle and warm time. BudgetForge's Modal analysis says a container is billed during cold-start warmup and any keep-warm window, not only while serving requests. It calculates a keep-warm H100 held for 30 days at zero traffic at $2,843. Scale-to-zero only saves money if your keep-warm settings let it scale to zero.
Operations. SemiAnalysis defines monthly cluster cost to include compute, storage, networking, control plane, support, goodput, setup and debugging. If your team spends a week debugging a flaky cluster, that time belongs in the cost. When the bill points to a staffing gap, for example nobody owns GPU utilization, hiring in AI and machine learning is a separate budget line to plan for.
How to estimate your own GPU cloud bill

Start from the workload, not from a price list. Allwo's infrastructure guide recommends separating training, fine-tuning and inference budgets and using real telemetry: tokens, latency, GPU utilization, concurrency and data-transfer logs.
A simple template:
- Define the job. Model size, GPU type and VRAM needed, and whether you are training, fine-tuning or serving.
- Count GPU-hours. Hours per run times runs per month for training; for serving, hours the endpoint must be warm.
- Pick the billing model from the section above and take the matching rate.
- Add storage and egress in gigabytes, at the provider's per-GB rates.
- Add a buffer for failed runs and reruns.
- Convert to cost per outcome: dollars per training run, per thousand requests, or per generated minute of video.
Example: a team serves a model on one H100 for 730 hours a month. At a $3.70 on-demand rate (the PetaPrice median) that is $2,701 before storage and egress. If the endpoint is really busy only 200 hours, a serverless rate of $3.95 per busy hour comes to $790. If it is busy 600 hours, serverless is $2,370 and starts to look much like dedicated capacity. Use your own utilization and quotes, as every figure here is a snapshot.
Comparing providers fairly
TransAtlas's live comparison offers a clean checklist for matching offers:
- Same GPU model and VRAM (an H100 SXM is not an H100 PCIe).
- Instance prices divided by GPU count.
- On-demand, interruptible and committed prices compared separately.
- Region availability, egress, storage and disk charges.
- Cold starts, interruption risk, SLAs and commitment terms.
For large pretraining, GPUaaS.com adds that the lowest sticker rate need not be the cheapest outcome: interconnect quality can set training time and therefore total cost.
One more caution on regional pricing. NerdLevelTech's February 2026 comparison lists AWS H100 at about $6.50 to $7 per GPU-hour in us-east-1, Google Cloud A3 High at about $10.98 in us-central1, and Azure ND H100 v5 at $12.29 in East US, so region and instance family change the number even within one provider.
Our view: price the result, not the hour
We test AI tools hands-on for a minimum of 14 days, measuring real-world performance, latency and cost-effectiveness, and that habit shapes how we read GPU price lists.
An hourly rate answers one question: what does the hardware cost? Buyers need a different one: what does a finished job cost? A cheaper GPU that is slower, interrupted or short on memory can lose to a pricier one. A reserved discount that goes unused is no discount at all.
So we recommend the same discipline we apply to tools: define the workload, run it for long enough to see real utilization, and record latency, throughput and total spend together. Our performance-benchmark approach is the model: numbers tied to a specific use case, with hidden costs and tier limits written down. If you run AI through hosted tools rather than your own GPUs, the same logic applies, and our reviews show verified pricing next to measured results.
AIMaestrohub
Choosing the right model for your workload
| Workload | Best-fit billing | Why |
|---|---|---|
| Prototype, unknown usage | On-demand | No commitment, quick to stop |
| Batch training with checkpoints | Spot | Lowest rate, tolerates interruptions |
| Spiky inference API | Serverless | No idle charge if warm time is controlled |
| Steady 24/7 inference | Reserved | Lowest rate once utilization clears break-even |
| Large multi-node pretraining | Reserved or committed cluster | Interconnect and availability matter beyond price |
Non-technical founders can shortcut this: if you do not run your own models, you may not need raw GPUs at all. Hosted AI tools bundle the compute into a subscription, and the question becomes which tool is worth its price. That is where independent benchmarks help.
Where to go next
If your AI spend is about tools rather than raw compute, our AI Tools Directory covers 870+ tools with hands-on reviews, performance benchmarks and verified pricing, and the head-to-head comparisons put tools side by side with hidden costs and tier limits spelled out.

Developers building on top of models can start with our best AI tools for developers, and our guide on how to read AI performance benchmarks explains how to judge latency and throughput claims before you commit budget.
FAQ
Why do GPU cloud prices differ so much for the same chip?
Providers differ in scale, networking, support, region, reliability and what the rate includes. PetaPrice's November 14, 2026 report showed H100 SXM from $1.95 to $12.30 per GPU-hour. Always match the exact model, VRAM and billing terms before comparing.
Is spot or interruptible GPU capacity safe to use?
It is safe for work that can stop and restart, such as batch training with checkpoints and evaluation runs. Providers can reclaim the capacity, so avoid it for latency-sensitive production endpoints. Vast.ai documents interruptible instances as often more than 50% cheaper than on-demand.
When does reserved GPU capacity pay off?
When your utilization of the reserved hours exceeds the reserved price divided by the on-demand price. Compute Suppliers' examples land between 70% and 80% for H100 offers. Measure a month of real usage before you sign a term.
Does serverless GPU billing really cost nothing when idle?
Only when the endpoint scales fully to zero. Warmup and keep-warm windows are billed, so a long keep-warm setting can turn a serverless endpoint into a full-time rental. Check the idle timeout before estimating a bill.
Sources
- PetaPrice GPU cloud price report
- GlobeGPU Cloud GPU Price Index
- American Compute GPU cloud price comparison
- Compute Suppliers spot vs reserved break-even
- VendorBenchmark AI infrastructure costs benchmark
- CoreWeave GPU cloud pricing
- GPU Ledger cloud GPU pricing glossary
- Vast.ai pricing documentation
- SemiAnalysis: How much do GPU clusters really cost
- BudgetForge Modal pricing analysis
- GMICloud cloud GPU pricing models
- Spheron serverless vs on-demand vs reserved
- GPUaaS.com provider comparison framework
- Allwo AI infrastructure cost models
- TransAtlas cloud GPU providers compared
- NerdLevelTech AWS vs GCP vs Azure GPU pricing
Recommended
- Claude vs Gemini (2026): Which AI Fits Your Work? Benchmark & Pricing
- Candy.ai Pricing, Free Trial & Features: 2026 In-Depth Buyer Guide
Editorial Disclosure: AI Maestro Hub is an independent software evaluation publication. When you buy through our links, we may earn an affiliate commission at zero extra cost to you. Our ratings remain 100% editorially independent. Read our full policy. ⚡ Content distribution and programmatic SEO architecture powered by [BabyLoveGrowth.ai] (https://www.babylovegrowth.ai/?red=aimaestrohub).
