Provisioned throughput for AI workloads, priced.

Reserved capacity that sits idle is the most common and least visible waste in an AI estate, because the meter charges the same whether tokens flow or not.

What does provisioned throughput actually buy you?

On Azure OpenAI you can call a model two ways. Pay as you go bills per input and output token at a published rate, with no commitment and no capacity guarantee, so cost scales perfectly with usage but latency can drift under load and there is no protection against rate limits at peak. Provisioned throughput instead reserves dedicated capacity measured in PTUs. Each PTU represents a guaranteed slice of model processing, and you pay a fixed hourly rate for it whether you send one request or saturate it. That fixed rate is the whole point: it converts a variable per token bill into a predictable capacity bill and gives you stable latency and headroom you control.

The catch is symmetry. The same property that makes provisioned throughput predictable, a fixed price decoupled from usage, makes it punishing when usage is low. An idle PTU costs exactly as much as a busy one. So the question is never whether provisioned throughput is cheaper in the abstract; it is whether your workload keeps the units you bought busy enough to clear the break even.

Where is the break even between PTUs and pay as you go?

The break even is the utilization at which the fixed hourly cost of a PTU equals the pay as you go cost of the tokens that PTU could serve in that hour. Below it, you are paying for capacity you did not use and pay as you go would have been cheaper. Above it, the fixed cost is spread across enough tokens that the effective per token rate drops below the pay as you go rate, and the advantage grows the busier the unit runs. Because list prices change, treat the exact crossover as a number to compute from current Azure pricing, not to memorise, and label any figure you quote as indicative until it is checked against the provider page.

Two variables move the break even. The first is sustained utilization: a unit that runs near saturation for most of the hour clears the bar easily, while one that serves a trickle of traffic never will. The second is commitment term. Buying PTUs hourly carries the highest rate and the most flexibility. Committing to a monthly or annual reservation lowers the effective hourly rate in exchange for utilization risk you now carry, which pulls the break even down and makes provisioned throughput pay off at a lower utilization, but only if the demand to fill it is real.

How should you size provisioned throughput capacity?

Size to a defensible peak forecast, not to your busiest minute ever. Start by measuring real production demand in tokens per minute at the exact model version and prompt shape you run, because input length, output length, and model all change how much a single PTU can serve. Azure exposes a capacity calculator that maps a target throughput to a PTU count for that model, so feed it your measured demand rather than a guess. Then split the workload. The steady base load that is present most of the day is the part worth covering with provisioned throughput or a reservation, because it will keep the units busy. The peaks above that base are cheaper to serve on pay as you go, because reserving for them strands capacity every hour the peak is absent.

This base plus burst pattern is the same risk adjusted logic that governs compute commitments. You commit to the demand you are confident will exist and you leave the uncertain top of the curve on the flexible meter. The failure mode we see most often is a team that reserved for its launch day spike and now pays around the clock for capacity that is used for one hour in twenty.

What else inflates the AI serving bill?

Capacity choice is the largest lever but not the only one. Prompt design moves real money: trimming system prompts, capping output length, and removing redundant context cut tokens on every single call, and on pay as you go that flows straight to the bill while on provisioned throughput it raises how much each PTU can serve and lowers the units you need. Caching repeated or near identical requests avoids paying twice for the same answer. Routing simpler requests to a smaller, cheaper model instead of sending everything to the largest one often halves cost with no quality loss for the easy cases. And right sizing the reserved portion every month, as traffic patterns shift, keeps utilization above the break even instead of letting a once correct reservation drift into waste.

Where this sits in the wider Azure picture

Provisioned throughput is one discipline inside a larger AI and PaaS cost program. The governance side of model spend, who can deploy, how usage is attributed, and how budgets are enforced, is covered in Azure OpenAI spend governance, and the broader question of which managed services earn their place is in PaaS choices that cut the Azure bill. The full estate playbook lives in the Azure cost optimization guide, and the cross cloud view, including how AWS and GCP price reserved AI capacity differently, is in the cross cloud cost optimization guide.

Put a number on your AI capacity

We help enterprises measure real model demand, compute the break even against current pricing, and size the reserved portion so capacity stays busy instead of idling. Our guarantee: we reduce your cloud spend or we reimburse our service fee. Pricing is either a Fixed Fee scoped up front or Gainshare, a share of verified savings with no retainer and no risk.