When organizations start evaluating enterprise AI on Azure, one of the first questions IT finance and technology leaders ask is straightforward: what will this actually cost? The answer depends on which Azure OpenAI pricing model you choose, how much throughput you need, and whether you’re running a single pilot or multiple AI workloads across teams.
This post explains how Azure AI pricing works, what the main cost drivers are, and how to plan for costs before you’ve committed to a build.
Key takeaways
- Azure AI costs on Microsoft Foundry fall into two main billing models: pay-as-you-go (serverless, billed by token) and provisioned throughput units (PTU), which offer a reserved, hourly capacity model.
- Token-based pricing varies by model. More capable models cost more per token. Fine-tuned models add separate charges for training, hosting, and inference.
- Provisioned throughput (PTU) provides predictable performance and bills hourly for deployed capacity, whether or not it’s used. Billing starts when a deployment is created and stops only when it’s deleted.
- Cost monitoring in Microsoft Foundry requires looking at multiple meters. Foundry costs are only part of a full AI solution’s cost.
- Fine-tuned model hosting charges accumulate even when the model is idle. Unneeded deployments should be scaled down or removed.
What are the billing models for Azure AI on Microsoft Foundry?
There are two main options.
Pay-as-you-go (serverless API) bills per token consumed: input tokens and output tokens are metered separately, and you pay for what you use with no upfront commitment. This is the right starting point for pilots and early-stage workloads where usage is unpredictable, since there’s no minimum spend and no reserved capacity to manage.
Provisioned throughput units (PTU) work differently. Instead of paying per token, you reserve a defined amount of model capacity for a region and deployment type, and billing runs by the hour from the moment the deployment is created. PTU is appropriate when you have consistent, high-volume workloads where latency predictability matters and where the economics of reserved capacity make sense versus per-token pricing.
For organizations running both exploratory and production AI workloads, using both models at the same time is common: pay-as-you-go for new or variable-use cases, PTU for workloads that have matured to a consistent throughput level.
Microsoft’s documentation on managing costs in Microsoft Foundry covers how to monitor both billing models, view usage by meter and resource, and set budget alerts. See the Microsoft Learn documentation on managing costs in Microsoft Foundry for that guidance.
What drives AI costs beyond the model itself
Model inference is the most visible cost, but it’s not the only one. A complete Azure AI solution typically includes several cost components that operate on different meters.
Fine-tuned models add three separate cost categories: a one-time training cost when the model is trained, a per-hour hosting cost once the deployment exists (whether or not the model is actively processing requests), and a per-token inference cost when it is. The hosting cost is the one that surprises organizations most. A fine-tuned deployment that runs 24/7 at low utilization can accumulate significant charges. Microsoft’s guidance on cost management explicitly recommends removing or scaling down fine-tuned deployments that are no longer needed.
Storage, compute, and retrieval costs apply when the AI solution includes Azure AI Search, Azure Blob Storage for document grounding, or other data services. A retrieval-augmented generation (RAG) setup that queries a large document corpus will have AI Search costs that run independently of model inference.
PTU deployments create an important billing nuance. Provisioned throughput billing starts at deployment creation and stops at deletion. A development environment with a PTU deployment left running over a weekend accumulates full charges for that time. For sustained production workloads, a 1-month or 1-year reservation lowers the effective hourly PTU rate, but canceling or exchanging a reservation might incur fees.
Before you commit to provisioned throughput, read Microsoft’s provisioned throughput billing documentation. It explains hourly billing, reservations, and how resizing a deployment changes the charge.
Our AI on Your Data pilot includes a written cost model at your projected monthly question volume, including how your existing Azure credits apply.
How do you monitor and control Azure AI spending?
Microsoft Foundry provides cost visibility at the portal level: the agent and model monitoring tabs show usage data, and the Azure portal’s Cost Management view lets you group charges by meter and resource to understand what’s driving spend. That said, Foundry costs appear alongside other Azure costs, and a complete picture of what an AI solution costs requires looking across multiple resource types.
Project-level chargeback is available in preview for supported Azure-sold models. Usage is automatically tagged with a project identifier, which lets organizations split shared AI spend by team or workload without requiring separate subscriptions. Marketplace models and some third-party model integrations don’t support this tagging.
Budget alerts can be set on subscriptions and resource groups. One important limitation: Azure OpenAI doesn’t support hard spending limits, so an alert notifies you when a threshold is reached but doesn’t stop the spend. Stopping spend automatically requires custom automation through Azure action groups.
For organizations managing AI costs across multiple teams, exporting cost data to a storage account for analysis in Excel or Power BI is a common practice. The lag in cost data visibility is worth accounting for: Microsoft’s documentation notes that cost data can lag, so compare trend windows rather than minute-by-minute figures before you assume a billing discrepancy.
Our post on FinOps for Azure AI covers cost governance for AI workloads in more depth. Our cloud engineering page covers how AI costs fit into broader Azure cost control.
Frequently asked questions
How does Azure OpenAI pricing work?
Azure OpenAI uses token-based pricing for pay-as-you-go deployments. Input and output tokens are billed separately, and the rate varies by model. More capable models cost more per token. The exact rates are listed on Microsoft’s Azure pricing pages and are subject to change as models are updated.
What are provisioned throughput units (PTU)?
PTU is a reserved capacity model for Azure OpenAI. You purchase a defined amount of model throughput for a region and deployment type, and billing runs hourly from deployment creation to deletion. PTU provides predictable performance for high-volume workloads. You pay for the deployed capacity whether or not it’s used, so it needs consistent utilization to justify the cost.
Do fine-tuned models cost more than standard models?
Yes. Fine-tuned models incur three types of charges: a one-time training cost, a per-hour hosting cost while the deployment exists, and a per-token inference cost when it’s in use. The hosting cost runs continuously, even when the model isn’t processing requests, so unused fine-tuned deployments should be removed.
How do you set a budget for Azure AI?
You can create budgets and alerts in the Azure portal for subscriptions and resource groups. Azure OpenAI doesn’t support hard spending limits, so alerts notify you but don’t stop spend. For automated cost controls, you’d need to configure automation through Azure action groups to respond to budget threshold alerts.
What’s the difference between serverless and provisioned throughput for Azure OpenAI?
Serverless (pay-as-you-go) bills per token with no minimum commitment. It’s appropriate for variable or early-stage workloads. Provisioned throughput (PTU) reserves capacity and bills hourly. It’s appropriate for consistent, high-volume workloads where predictable latency matters. Many organizations use both: serverless for new use cases, PTU for production workloads that have stabilized.
Keep reading
Model Context Protocol Explained for Enterprise Teams
If you've been evaluating AI agents for enterprise use, you've likely started hearing about Model Context Protocol, usually shortened to MCP.
AI Agent Security: Identity, Permissions, and Blast Radius
When an AI agent runs in a production environment, it doesn't just answer questions.
Enterprise AI Agents on Azure: Where to Start
The term "AI agent" has moved from research papers to enterprise roadmaps faster than most technology concepts do.