Most enterprise teams building AI agents on Azure start by asking the wrong question. They ask which framework to use before they’ve decided who owns the infrastructure underneath it. Foundry Agent Service answers that second question: it’s Microsoft’s managed runtime for agent deployments, and it takes the infrastructure off your team’s plate.
This post covers what Foundry Agent Service actually does, when it’s the right choice over Azure Container Apps, and what you need in place before you deploy your first agent.
Key takeaways
- Foundry Agent Service is the managed runtime within Microsoft Foundry for building and deploying AI agents. It handles conversation state, tool execution, and model calls without custom infrastructure.
- An agent in Foundry has three parts: instructions that define its role and constraints, a model deployment from your Foundry project, and a set of tools it’s allowed to call.
- The choice between Foundry Agent Service and Azure Container Apps comes down to orchestration: Foundry works for agents that fit the managed model; Container Apps gives you control when they don’t.
- Before you deploy, you need a Foundry project with a model deployed, defined tool connections, a managed identity with scoped permissions, and a written scope of what the agent can and can’t do.
- Human oversight rules, such as what the agent escalates and what it can’t decide without review, belong in the design, not the retrospective.
What is Foundry Agent Service?
Foundry Agent Service is the runtime layer inside Microsoft Foundry where agents run. When a user sends a message to an agent, Foundry creates or continues a conversation, adds the message, calls the model with the conversation so far, executes any tool calls the model makes, and returns a response. Your team writes the agent’s instructions and configures its tools. Foundry runs the loop.
An agent has three parts. The instructions define what the agent does and what it doesn’t do: role, constraints, escalation behavior. The model is a deployment within your Foundry project; Foundry works with many models from its catalog, such as GPT-4o, Llama, and DeepSeek, and you can swap models without changing agent code. The tools define what the agent can act on: built-in tools like file search and code interpreter, or connections to external APIs and services through OpenAPI specs and MCP servers.
Conversation history across turns lives in Foundry, so the agent doesn’t have to manage it. That’s what “managed runtime” means in practice.
For a full technical reference, see the Microsoft Learn overview of Foundry Agent Service.
When should an agent run in Foundry versus Azure Container Apps?
Foundry Agent Service is the right choice when your agent fits what the managed runtime handles: built-in tool calls, persistent conversation state, and content safety controls applied to inputs and outputs. You set up the agent, configure its tools, and Foundry handles the execution environment.
Container Apps is the right choice when your orchestration requirements go beyond that. Multi-agent systems where agents hand off tasks between themselves, agentic loops with custom retry logic, or agents that need to integrate tightly with existing application infrastructure usually need the control that a containerized runtime gives you.
The two aren’t mutually exclusive. A common pattern is an orchestrator running in Container Apps that delegates specific tasks to agents in Foundry Agent Service. Each layer does what it’s suited for. Foundry can also run your own agent code as a hosted agent, so the real decision is how much of the runtime your team wants to own. If Container Apps is already part of your stack, our guide to migrating from AKS to Azure Container Apps covers how the platform behaves in production.
Winmill designs, builds, and runs data and AI solutions on Azure for organizations across North America.
What do you need in place before deploying an agent?
A Foundry project with a model deployment. Agents run within a project and call a model deployment inside it. You need the project configured, a model deployed, and enough quota for your expected usage.
Defined tool connections. If your agent calls external tools to query an API, retrieve from a file index, or run code, those connections need to be in place before the agent does anything. Built-in tools like file search and code interpreter are enabled per agent. External connections through OpenAPI specs or MCP servers require the connection to be configured and the agent’s identity to have the right permissions to call it.
A managed identity with scoped permissions. The agent should authenticate to its tools through a managed identity with the minimum required permissions, not through credentials in the agent instructions or stored as plain text anywhere.
A written scope. Before an agent goes to production, its instructions should say not just what it does but what it doesn’t do. What can it decide without a human in the loop? What does it escalate? What happens when a tool call fails? Scope constraints in the instructions are a production requirement. They’re much harder to add after something goes wrong. Our post on responsible AI guardrails on Azure covers the controls that sit around them.
Frequently asked questions
What is Microsoft Foundry Agent Service?
It’s the managed runtime within Microsoft Foundry for building and deploying AI agents on Azure. It handles conversation state across turns, tool execution, and content safety controls. Your team writes the instructions and configures the tools; Foundry runs the agent.
How is Foundry Agent Service different from running an agent in Azure Container Apps?
Foundry Agent Service is a managed runtime: conversation state, tool execution, and content filters come built in. Container Apps is a flexible container host where your code controls the orchestration logic. Foundry is faster to deploy for agents that fit the managed model. Container Apps gives you more control when your requirements go beyond what Foundry handles natively.
What tools can a Foundry Agent Service agent call?
Built-in tools include web search, file search, and code interpreter. External connections work through OpenAPI specs and MCP servers. The agent can only call tools you explicitly configure; it has no access to anything else.
What identity should an agent use to call external tools?
Managed identity, with the minimum required RBAC role for each tool it calls. No embedded credentials, no API keys in the instructions. For connections to Azure services like Azure AI Search or Blob Storage, a managed identity with appropriate roles is the right model.
Who should be accountable for a Foundry Agent Service agent in production?
A named owner who’s responsible for monitoring agent behavior, reviewing escalations, and approving changes to the agent’s instructions or tool access. An agent that takes action on behalf of users by writing to a database, calling an external API, or sending messages needs clear accountability before it goes live.
Keep reading
FinOps for Azure AI: Controlling Token and Compute Costs
Azure AI spending doesn't behave like other cloud costs. You're not just paying for compute and storage. You're paying per token, per API call, per model.
Detecting Data Drift and Model Drift in Production ML Models
A model that performed well at deployment can quietly degrade for months before anyone notices. The inputs change. The real-world patterns your model learned shift.