Model choice as a cost decision.

Here is how to put a price on each model, route requests by difficulty, and build a decision framework your engineers and your finance team both trust.

Why model choice dominates AI cost

Most enterprise AI bills are inference, not training, and inference cost is set by two numbers: the price per token of the model you call and the number of tokens each request consumes. The model term dominates because the spread is so wide. A capable mid tier model can serve a request for a small fraction of what a frontier model charges for the same tokens, and the cheapest hosted models cost less again. The trap is the default. Teams wire an application to the best model during a demo, ship it, and never revisit the choice. Every request then pays frontier prices, including the large share of traffic that is classification, extraction, short summarisation, or routing, where a much cheaper model produces an answer a user cannot distinguish. The bill grows linearly with adoption, and nobody decided that it should.

How do you put a price on a model?

Price per token is the headline, but the number that matters is cost per successful outcome. Build it from four inputs you can pull from billing and application logs:

• Input tokens per request, including the system prompt, retrieved context, and few shot examples that ride along on every call.
• Output tokens per request, which you control through response length limits.
• Retry and verification overhead, because a cheap model that fails a quality check and triggers a second call has effectively paid twice.
• The published price per million tokens for input and output, which differ and should be modelled separately.

Multiply through and you get a defensible cost per request for each candidate model. Run the same evaluation set across two or three models and the table almost always shows a mid tier model clearing the quality bar at a fraction of the frontier cost, which is the decision the team should have been making all along.

What is model routing and when does it pay?

Routing means a request is sent to a model chosen by task type or difficulty rather than a single default. A lightweight classifier, or even a set of rules keyed on the request type, decides which model handles each call. Simple intents go to a small fast model, ambiguous or high stakes ones escalate to a frontier model, and the expensive tier only ever sees the traffic that needs it. Routing pays whenever your traffic mix is uneven, which is almost always. If eight in ten requests are routine and clear the bar on a model costing a tenth of the frontier price, routing collapses the blended cost per request toward the cheap tier while preserving frontier quality on the hard minority. The cost of the classifier itself is tiny next to the saving it unlocks, because the classifier runs on a small model and decides in a few tokens.

Does this differ across AWS, Azure, GCP, and OCI?

The principle is identical on every cloud; the instruments differ. On AWS the Cost and Usage Report attributes model spend when you tag inference workloads, and a model gateway lets you switch the backing model without touching application code. On Azure, OpenAI usage is metered per model and per deployment, so the deployment you point traffic at is the cost decision, and provisioned throughput changes the math for steady high volume. On GCP the same routing logic applies across the model garden, and capacity pricing rewards predictable traffic. On OCI, generative AI is billed against Universal Credits, so model choice draws down the same committed pool and should be governed with the same discipline. Across all four, the buyer side rule holds: never let a single default model quietly bill at frontier rates for routine work. The cloud only sells you what you ask for, so the saving comes from asking for less expensive capacity where it suffices.

Make model choice a governed decision with us

We help enterprises instrument model level cost across AWS, Azure, GCP, and OCI, then design routing that holds quality while cutting inference spend. Our guarantee is plain: we reduce your cloud spend or we reimburse our service fee, on either a Fixed Fee or a no risk Gainshare basis.

For the wider picture, our cross cloud guide sets model choice in the context of every other AI cost lever, and we send new analysis through The Cloud Spend Navigator.