Microsoft Foundry: Agents via AI Gateway

Key insights

• Summary: This note summarizes a demo video that walks through building and deploying a Microsoft Foundry agent using the new AI Gateway tier of Azure API Management.
  I am not the video's author and I present the key points and practical steps shown in the demo.
• Core purpose: The AI Gateway tier acts as a control plane to secure and manage agent-to-tool traffic, enforcing policies and adding observability so enterprises can govern agent behavior centrally.
• Platform integration: Microsoft Foundry provides a managed runtime with Hosted agents in the Foundry Agent Service, plus managed tool endpoints called Toolboxes, memory support, and voice capabilities for richer agent experiences.
• Developer flow shown: The demo uses a single azd up flow to clone, configure, and deploy a hosted agent wired through the AI Gateway, giving governed access to MCP tools and frontier models like gpt-5.6-sol.
• Enterprise benefits: Expect centralized security, shared policy-based controls, full distributed tracing for production monitoring, and smoother publishing paths into Microsoft 365 and Teams.
• Next steps: Try the public preview quickstarts: create an agent, connect it through the AI Gateway, apply policies, test in the playground, and monitor traces before publishing to production.

Overview: Microsoft showcases the AI Gateway in action

The Microsoft Azure Developers channel published a demonstration video that walks through the new AI Gateway tier of Azure API Management and how it integrates with Microsoft Foundry. In the clip, presenter Paul Yuknewicz shows a full path from cloning a sample to deploying a hosted agent and wiring it through the gateway with a single azd up command. The video emphasizes that this setup gives agents governed access to frontier models like gpt-5.6-sol, managed tools, and policy-based controls. Overall, the demo positions the AI Gateway as a central control plane for agent traffic and tool access.

Demo walkthrough: from clone to governed agent

The walkthrough begins with the Foundry sample repository and proceeds through the Foundry Toolkit. Viewers see how a developer can create a hosted agent, select a model, and deploy to the Foundry Agent Service in minutes. Then the demo wires the agent through the AI Gateway so that all tool calls and external requests flow through API Management policies and tracing. The clear takeaway is simplicity: the demo shows a compact workflow that reduces manual wiring and adds governance out of the box.

The video also highlights runtime observability and distributed tracing in Foundry, which helps teams diagnose interactions between agents and tools. By routing traffic through the gateway, teams gain consistent logs and metrics without instrumenting each agent separately. This centralized tracing is useful for production monitoring and debugging, particularly when multiple agents share the same MCP tools. Consequently, organizations can scale agent deployments while keeping a single place to inspect behavior.

Governance and security: centralized control with tradeoffs

The presenters argue that the AI Gateway adds meaningful governance by enforcing policies for tool access and applying security controls at the gateway layer. Central policies can block risky calls, rate-limit tool usage, and require authentication so enterprise rules apply uniformly across agents. This centralization reduces the burden on individual agent developers, who no longer must implement custom connectors or security checks for each tool they call.

However, centralizing control also brings tradeoffs. Routing all traffic through API Management can add latency and increase operational cost, especially at high volume. Moreover, it introduces a single control plane that must itself scale and be highly available. Teams should therefore weigh the benefits of unified governance against the potential impact on performance and vendor lock-in when a critical part of the toolchain depends on the gateway.

Observability and tooling: benefits and limitations

The video demonstrates how the combined Foundry and gateway stack surfaces telemetry for both agent behavior and tool interactions, making it easier to trace complex flows across services. In addition to logs, the demo shows distributed tracing that ties model calls to downstream tool usage, which can speed up incident response and root-cause analysis. Furthermore, policies in API Management let administrators control access and collect usage metrics without changing agent code.

At the same time, the approach introduces complexity in toolchain management, as teams must coordinate policy updates and tracing schemas across organizations. Integrating third-party tools behind the gateway may require additional adapters or configuration. Therefore, while observability improves, teams must invest in operational processes and testing to avoid breaking agent behavior when policies change.

Practical guidance: when to adopt and what to plan for

For teams building production agent systems, the AI Gateway tier offers a compelling way to centralize governance, secure tool access, and maintain observability. The public preview status means teams can experiment and validate integration patterns today, but should be cautious about moving critical workloads to preview features without a migration plan. Consequently, pilot projects and staged rollouts are a sensible approach to validate performance, policy behavior, and cost implications.

Teams should also plan for tradeoffs: balancing governance with latency, handling the operational overhead of a centralized control plane, and defining clear policies for tool sharing across units. In addition, organizations should assess multi-tenant needs, tooling adapters, and disaster recovery for the gateway itself. Ultimately, the demo shows a fast path to a governed agent deployment, but successful production use will require careful testing, monitoring, and a governance strategy aligned with business risk tolerance.

Bottom line

The Microsoft Azure Developers video presents the AI Gateway as an important piece in the emerging Foundry agent ecosystem, linking hosted agents, managed tools, and enterprise governance. It demonstrates a streamlined developer experience that gets an agent from clone to governed deployment in minutes, while also highlighting the operational gains from centralized policies and tracing. Nevertheless, organizations should weigh the tradeoffs around latency, cost, and operational complexity before adopting the gateway broadly in production.

Keywords

Microsoft Foundry agents, AI Gateway API Management, API Management AI Gateway, Build Foundry agents, Azure API Management AI Gateway, Deploy Foundry agents, Foundry agents tutorial, AI Gateway tier features