Microsoft Previews Dedicated Azure AI Gateway Tier

Microsoft has released a dedicated AI Gateway tier of Azure API Management in public preview, currently available for free in the East US 2 and Sweden Central regions. Unlike previous iterations that layered policies onto existing gateways, this new tier features a control plane designed specifically around models, Model Context Protocol (MCP) servers, and tools. It provisions in approximately one minute with no scale units to plan, running directly within the customer's subscription and Entra tenant.

The gateway allows organizations to publish Foundry-hosted models from OpenAI, Anthropic, and Mistral, alongside external models from AWS Bedrock, Google Vertex AI, and OpenAI directly. It routes OpenAI-compatible providers through a single endpoint path based on exact model name matches, while Anthropic is managed via a custom provider with Messages API passthrough. For tools, the gateway federates backends from remote MCP servers by URL, OpenAPI specifications, or a built-in connector supporting more than 1,000 SaaS applications.

Administrators can configure policies like token limits, Content Safety, and model fallbacks using portal cards instead of XML. Telemetry is exported as OpenTelemetry token metrics to platforms like Datadog, Grafana, and Application Insights. While industry figures like newsletter writer Paolo Perrone praised the centralized cost governance, others have raised questions. Enterprise architect Adolph White Jr. questioned how the gateway handles incomplete agent runs, while Sreenivasulu Kandakuru of Aer Lingus suggested Azure still trails competitors like Databricks and its Unity AI Gateway.

Security and migration also remain key considerations for practitioners. A runtime access key is currently gateway-scoped, meaning a single leaked key compromises every model and tool on that gateway. Furthermore, Microsoft has not yet provided guidance on how this preview tier will coexist with or migrate from existing Premium or Standard v2 tiers, leaving early adopters to navigate integration strategies on their own.