NVIDIA H200 GPU Guide 2026: Specs, Benchmarks, and Pricing

By Sandeep Narahari, Contributor. Last updated: August 2026.

Note: All hourly rates in this guide were fetched directly from provider pricing APIs and public pricing pages on August 20, 2026 at 20:05 UTC. GPU rental prices move weekly, so treat every figure as a point-in-time reading and check the linked sources before you budget.

The NVIDIA H200 is a Hopper-architecture data center GPU with 141GB of HBM3e memory and 4.8TB/s of memory bandwidth. Renting one costs between $4.45 and $10.60 per GPU-hour on demand, depending on the provider.

TL;DR

• Specs: 141GB HBM3e, 4.8TB/s bandwidth, 3,958 TFLOPS FP8 with sparsity, up to 700W TDP in SXM form factor. Same compute silicon as the H100, with 1.76x the memory and 1.43x the bandwidth (NVIDIA).
• Cheapest on-demand rate found: $4.45 per GPU-hour, on the Akash Network marketplace, fetched from its public API on August 20, 2026. Nebius at $4.50 and RunPod at $4.59 sit within 3% of it, so the sub-$5 tier is competitive rather than a single outlier.
• Most expensive: $10.60 per GPU-hour on Azure ND96isr H200 v5 in East US 2, fetched from the Azure Retail Prices API on the same date. Azure costs 2.4x the marketplace low for identical hardware.
• Cheapest interruptible rate: $2.33 per GPU-hour on Azure low priority in South Central US 2, which is 78% below Azure list and suits checkpointed or batch work only.
• Real inference gain over H100: roughly 1.4x on memory-bound decode, not the 1.9x headline figure, which came from a specific TensorRT-LLM offline throughput test.
• Buying instead: an H200 SXM module runs $32,000 to $40,000, and an 8-GPU HGX H200 server runs about $370,000, which is roughly 14 months of continuous rental at $4.45/hr.

What are the NVIDIA H200’s full specs?

The H200 pairs the H100’s Hopper compute die with 141GB of HBM3e memory at 4.8TB/s, which is 1.76x the capacity and 1.43x the bandwidth of the H100 SXM. Peak math throughput is unchanged from the H100, so every H200 gain traces back to memory.

Source: NVIDIA H200 product page. The practical takeaway is that the SXM variant is the one nearly every cloud rents, and the NVL variant trades about 16% of peak throughput and 100W of power for a PCIe form factor that fits standard servers.

How much faster is the H200 than the H100?

The H200 delivers roughly 1.4x the H100’s LLM decode throughput in independent testing, and up to 1.9x in NVIDIA’s own offline throughput benchmarks. The gap between those two numbers is the difference between a bandwidth-bound benchmark and a marketing-facing best case.

Sources: General Compute benchmark, June 14, 2026, NVIDIA TensorRT-LLM H200 launch post, NVIDIA H200 product page.

The 1.4x figure tracks the 1.43x bandwidth increase almost exactly, which is the expected result for autoregressive decoding. For what the older part costs today, see H100 rental price in August 2026, which tracks H100 cost per hour across eleven providers. NVIDIA’s own TensorRT-LLM numbers carry a disclaimer that they were produced on TensorRT-LLM v0.5, so newer serving stacks will shift them.

The larger H200 benefit is topology, not throughput. A 70B model in BF16 fits on one H200 but needs two H100s, and Llama 405B in FP8 fits on two H200s versus four to six H100s. Removing a GPU from the serving topology removes tensor-parallel communication overhead, which often beats the raw 1.4x.

How much does an H200 cost per hour in 2026?

On-demand H200 rental costs $4.45 to $10.60 per GPU-hour, a 2.4x spread across providers for the same 141GB SXM part. Every rate below was fetched on August 20, 2026 at 20:05 UTC from the source listed in the final column.

Two things stand out. First, hyperscaler H200 capacity sells only in 8-GPU nodes, so the minimum AWS commitment is $63.296 per hour and the minimum Azure commitment is $84.80 per hour, while the marketplaces and neoclouds rent single GPUs. Second, the entire sub-$5 tier is 44% to 58% below the hyperscaler rates for identical silicon, and the gap is structural rather than promotional: AWS and Azure bundle enterprise support, compliance certifications, and integrated networking into the hourly rate.

A third-party aggregator listing 36 providers put the full H200 range at $1.41 to $13.78 per GPU-hour across all billing models on the same day (GetDeploying).

How much does it cost to buy an H200 instead of renting one?

An H200 SXM module costs $32,000 to $40,000, and a complete 8-GPU HGX H200 server costs about $370,000, or roughly $46,250 per GPU once NVSwitch fabric, CPUs, and chassis are included (Mercatus, last verified 2026-05-04).

These break-even points ignore power, cooling, colocation, networking, staff, and depreciation, all of which push the real crossover later. A 700W GPU at industrial power rates plus datacenter overhead adds meaningful monthly cost per GPU, so a purchase that looks break-even at 14 months on hardware alone typically lands closer to 20 months or more in practice. Buying makes sense at sustained 90%-plus utilization over multiple years. Renting makes sense for everything spikier than that.

Why did H200 supply and pricing shift in August 2026?

H200 supply loosened in August 2026 because the first meaningful shipments of the chip reached Chinese buyers, releasing inventory that NVIDIA had stockpiled for that market. ByteDance and Tencent each took delivery of roughly 10,000 H200 accelerators in the weeks before August 19, 2026, the first substantial movement into the region since export approval in December 2025 (Tom’s Hardware, August 19, 2026).

Those 10,000-unit deliveries are about 2.5% of the 400,000-plus units the approved Chinese firms are licensed to buy, and NVIDIA had built up roughly 500,000 H200s for the market. Beijing has routed most licensed chips to Hong Kong, whose 47 datacenters total 581MW, well short of what a single company’s full allocation would draw. The practical effect for renters outside China is that a large H200 inventory remains in the western market for now.

Pricing has moved in both directions this year. AWS raised P5e and P5en capacity block pricing by about 15% in January 2026, taking one US West configuration from $43.26 to $49.75 per hour while stating that its commitment not to raise on-demand and Savings Plan pricing was unchanged (Data Center Dynamics, January 6, 2026). Marketplace and neocloud rates, meanwhile, have drifted down as Blackwell capacity absorbs the demand for frontier training.

FAQ