Category: Azure AI

Deploying, evaluating, and calling models in Microsoft Foundry: a production guide for architects

I have spent the last few programmes wiring Microsoft Foundry models into real workloads, and the same confusions keep surfacing in design reviews. People conflate the model with the deployment, pick a deployment type by habit, and discover the cost implications only when the first invoice lands. This post is the guide I wish my teams had read first.

I have deliberately left out resource and project creation, RBAC setup, and quota-increase mechanics, because those sit in the getting-started post. Here I focus on the four decisions that actually move cost and behaviour: what a deployment is, which deployment type to choose, how to evaluate before you commit, and how to call the model cleanly from the SDK.

One caveat up front. Foundry naming and feature status move quickly, and pricing on Azure renders dynamically. Treat every number below as something to verify on the Azure pricing calculator before you budget, and check the (preview) tag on the specific Learn page before you depend on a feature.

Resource, project, model, deployment. The deployment name, not the model name, is the addressable unit for inference.

What “deploying a model” actually means

The mental model that saved my teams the most time is to separate four things. The resource is the Microsoft.CognitiveServices account of kind AIServices: it is the governance, quota, and networking boundary. The project is the RBAC and isolation boundary inside it, and it owns agents, connections, evaluations, and threads.

The model is an item in the catalogue, whether an Azure OpenAI model, a Microsoft model, or a partner model from Anthropic, Meta, Mistral, Cohere, DeepSeek, xAI, and others. The deployment is what you get when you deploy a model into the resource with a chosen deployment type (the SKU) and quota. It has a deployment name that you choose, and that name, not the underlying model name, is the addressable unit for inference.

This matters in code. Microsoft’s quickstart is explicit: the model parameter requires the model deployment name, and if your deployment name differs from the underlying model name you adjust your code accordingly. An agent definition references the same deployment name through its model field.

There is a useful exception since early 2026. Instant models (preview) let you call a supported model by name with no deployment at all, drawing on a separate global quota pool. They route to the latest evergreen version by default (pin a version by appending a date suffix), and during preview they are available only in West US 3 projects. Microsoft frames deployments as something you level up to, not a gate you must pass first. I reach for a deployment when I need reserved throughput, custom content filters, data residency, or enterprise configuration.

Deployment types: the decision that fixes everything else

The deployment type is the single biggest choice. It fixes data residency, latency variance, the quota model, and how you pay. Microsoft groups the types into standard (pay-per-token), provisioned (reserved PTU), and batch (async, 50% off), each available at global, data-zone, or regional scope. Data stored at rest always remains in the designated Azure geography: the differences below are about where inference is processed and how throughput is guaranteed.

Deployment type	SKU code	Data processing scope	Billing model	SLA / latency	Best for
Instant (preview)	N/A (no deployment)	Any Azure region	Pay-per-token (global quota pool)	Best-effort, no SLA	Getting started, prototyping
Global Standard	GlobalStandard	Any Azure region	Pay-per-token	Best-effort, highest default quota	General workloads, highest quota
Data Zone Standard	DataZoneStandard	Within US or EU data zone	Pay-per-token	Best-effort, higher quota than regional	EU/US data-zone compliance
Standard (regional)	Standard	Single deployment region	Pay-per-token	Best-effort, limited regional capacity	Regional compliance, low to medium volume
Global Provisioned	GlobalProvisionedManaged	Any Azure region	Reserved PTU (hourly or reservation)	Guaranteed throughput, low latency variance	Predictable high throughput
Data Zone Provisioned	DataZoneProvisionedManaged	Within US or EU data zone	Reserved PTU	Guaranteed throughput plus data-zone	Data-zone plus predictable throughput
Regional Provisioned	ProvisionedManaged	Single deployment region	Reserved PTU	Guaranteed throughput, strict residency	Regional compliance plus throughput
Global Batch	GlobalBatch	Any Azure region	50% off Global Standard	No real-time SLA, 24-hour target	Large async jobs
Data Zone Batch	DataZoneBatch	Within US or EU data zone	50% off	No real-time SLA, 24-hour target	Large async jobs with data-zone
Developer	DeveloperTier	Any Azure region	Pay-per-token	No SLA, no residency guarantee, 24-hour lifetime then auto-deleted	Fine-tuned model evaluation only

A few notes from Learn that bite teams in production. Not all models support all types, so check “Foundry Models sold by Azure” for availability. With Global Standard and Data Zone Standard, a primary-region interruption affects all traffic initially routed there. Developer deployments self-delete after 24 hours, so they are for evaluating fine-tuned models, not for anything that needs to persist.

Choosing a deployment type: residency requirement, then traffic pattern, then volume.

Choosing by requirement is usually faster than reading the full matrix.

If you need	Use
No residency restriction	Global Standard or Global Provisioned
EU or US data-zone compliance	Data Zone Standard / Data Zone Provisioned
Single-region residency	Standard or Regional Provisioned
Quick start or prototype	Instant models (preview)
Variable, bursty traffic	Standard or Global Standard (pay-per-token)
Consistent high volume	Provisioned types
Large, non-time-sensitive jobs	Global Batch or Data Zone Batch
Low latency variance	Provisioned types
Fine-tuned model evaluation	Developer

Three platform features are worth knowing before you commit. Spillover (GA, 2026) routes overflow from a provisioned deployment (a 429 when PTUs are exhausted, for example) to a matching Standard deployment in the same resource, billed at the standard per-token rate. The data-processing level must match (global provisioned to global standard), and it works with the Foundry Agent Service but not the Responses API, so plan gateway-level fallback there. Priority processing (GA, 2026) is a pay-per-call fast lane for latency-sensitive Standard workloads at a premium over Standard. Model router is a deployable chat model that picks an underlying model per prompt, and it now supports the GPT-5 series.

Spillover: a provisioned deployment hits a 429, overflow routes to a matching Standard deployment in the same resource, billed per token. Does not cover the Responses API.

Cost: the three levers, and the free money teams forget

Per-token rates on Azure match OpenAI’s direct API. The premium you pay buys compliance, private networking, Entra authentication, support, and a single invoice. The deployment type changes how you pay, not the underlying token rate for a given scope.

Indicative pay-as-you-go rates follow, for Global Standard, per 1M tokens. Verify these on the Azure pricing calculator: they are correct as of June 2026 per third-party aggregators (PricePerToken.com, last updated 14 June 2026) consistent with OpenAI list prices, not guaranteed Microsoft figures.

Model	Input / 1M	Output / 1M	Notes
GPT-4.1	\$2.00	\$8.00	1M-token context
GPT-4.1-mini	\$0.40	\$1.60	strong cost/quality for routers
GPT-5	\$1.25	\$10.00	flagship reasoning, ~272,000-token context
GPT-5-mini	\$0.25	\$2.00
GPT-5-nano	\$0.05	\$0.40	cheapest

Pay-as-you-go vs PTU vs Batch across volume, with the 150 to 200M tokens/month break-even band marked. Verify on the Azure pricing calculator.

The three cost levers are pay-as-you-go, Provisioned Throughput Units (PTU), and Batch. Pay-as-you-go wins for variable traffic. Batch runs at 50% of Global Standard with a 24-hour target turnaround and a separate enqueued-token quota, so async jobs do not disrupt online traffic. Input is JSONL, one request per line with a unique custom_id, and you pay only for completed work.

PTU is reserved capacity, billed hourly per deployed unit regardless of tokens consumed. The GPT-4o-class Global provisioned rate is roughly \$1/hour per PTU. Reservations give large term discounts: per Microsoft Learn’s onboarding page, a 1-month reservation is around 64% off and a 1-year around 70% off for GPT-4o-class, with example rates stamped “Azure pricing as of January 1, 2025”. Minimums matter too: Global and Data Zone Provisioned require 15 PTU, Regional Provisioned requires 25 PTU for mini/nano-class and 50 PTU for larger models.

Two scope and discount rules round this out. For a given model, Data Zone is roughly +10% over Global and Regional is roughly +10% to +25%. Cached input tokens receive an automatic discount (roughly 50% to 90% off the input rate on repeated prefixes), so keep system prompts byte-identical across requests to trigger it.

On the PTU break-even, do not commit on a calculator estimate. Third-party analysis (AZ365.ai, 2026) puts break-even at roughly 150 to 200 million tokens per month for GPT-5, but that assumes 100% sustained utilisation. I run pay-as-you-go for 30 to 60 days, measure P95 hourly throughput, then size PTU against real telemetry. One more trap: rate limiting estimates max processed tokens at request time including max_tokens, so an over-large max_tokens can self-throttle you.

Evaluating a model before you commit

Before deployment I shortlist with the Foundry model leaderboard (preview), which ranks catalogue models on quality, safety, cost, and throughput with trade-off charts and side-by-side comparison of up to three models. Cost benchmarks assume a 3:1 input-to-output ratio. This narrows the field cheaply before I spend on my own evaluation.

Then I evaluate on my own data. Model and dataset evaluation is GA (agent evaluation remains preview), runnable from the portal or via the azure-ai-evaluation SDK. AI-assisted evaluators need an Azure OpenAI deployment as the judge, and Microsoft recommends gpt-5-mini for a good cost/quality balance.

For a RAG system, GroundednessEvaluator is the first one I set up, because it is the leading indicator of hallucination risk. I pair it with RelevanceEvaluator and the safety evaluators (ViolenceEvaluator, SelfHarmEvaluator, HateUnfairnessEvaluator, and similar). Quality evaluators such as CoherenceEvaluator and FluencyEvaluator use a 1 to 5 Likert scale with a default pass threshold of 3. Different evaluators have different data needs: groundedness needs the source context, ROUGE-style evaluators need ground-truth references, and tool-call accuracy needs the full agent message trace. Results publish to Azure Monitor and Application Insights, so I alert on groundedness regressions. The decision rule is simple: shortlist with the leaderboard, evaluate the top two or three on your own data, and pick the cheapest model that clears your thresholds.

Using a deployed model in an agent

An agent references the deployment by name through its model field. In Azure AI Projects 2.x:

			
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
project = AIProjectClient(
    endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)
agent = project.agents.create_version(
    agent_name="my-agent",
    definition=PromptAgentDefinition(
        model="gpt-5-mini",   # the DEPLOYMENT name (or an instant-model name)
        instructions="You are a helpful assistant that answers general questions",
    ),
)

		

To converse, create an OpenAI-compatible client and use the Responses API with an agent reference:

			
openai = project.get_openai_client()
conversation = openai.conversations.create()
response = openai.responses.create(
    conversation=conversation.id,
    extra_body={"agent_reference": {"name": "my-agent"}},
    input="...",
)

		

The Foundry Agent Service went GA in March 2026. The teaching point worth repeating in reviews: the agent’s model is the deployment name, which you find under Models + Endpoints in the portal.

Calling the model via the Foundry SDK

The SDK consolidated. azure-ai-projects 2.x is now the single Foundry SDK, covering agents, inference, evaluations, and memory, with the standalone azure-ai-agents dependency folded in. The 2.0.0 stable release shipped on 6 March 2026 and current PyPI is 2.2.0. Code written for 2.x is incompatible with 1.x: the old from_connection_string and .inference.get_chat_completions_client() patterns were removed, so budget a refactor sprint if you built against the beta.

Authenticate with DefaultAzureCredential from azure-identity. The recommended current pattern is to get an OpenAI-compatible client from the project:

			
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
project = AIProjectClient(
    endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)
with project.get_openai_client() as client:
    response = client.responses.create(
        model="gpt-5-mini",            # deployment name
        input="What is the size of France in square miles?",
    )
    print(response.output_text)

		

For OpenAI-style chat completions specifically:

			
with project.get_openai_client() as client:
    resp = client.chat.completions.create(
        model="gpt-4.1",               # deployment name
        messages=[{"role": "user", "content": "How many feet are in a mile?"}],
        temperature=0.7,
        max_tokens=500,
    )
    print(resp.choices[0].message.content)

		

Use the project endpoint and Responses API for Foundry features (agents, evaluations, tracing, content filters). Use the direct /openai/v1 endpoint for maximum OpenAI compatibility, lowest latency, or embeddings, which the project endpoint does not currently route. On .NET the equivalents are Azure.AI.Projects, Azure.AI.Extensions.OpenAI, and Azure.Identity, with the same pattern: construct AIProjectClient, get a chat or responses client, and pass the deployment name. One gotcha: do not install the preview Azure.AI.Projects.OpenAI alongside the GA Azure.AI.Extensions.OpenAI, because duplicate types cause ambiguous references.

Inference parameters, and why reasoning models break the rules

These are standard OpenAI parameters on the chat-completions and responses calls.

Parameter	What it does	Range / default	Notes
`temperature`	Scales the whole distribution	0 to 2, default 1.0	Tune this or `top_p`, not both
`top_p`	Nucleus sampling	0 to 1, default 1.0	0.9 a common safety net; no `top_k` exposed
`max_tokens` / `max_completion_tokens`	Caps output tokens	set conservatively	Reasoning models require `max_completion_tokens`
`frequency_penalty`	Penalises repeated tokens	-2.0 to 2.0, default 0	Leave at 0 for code/JSON
`presence_penalty`	Encourages new topics	-2.0 to 2.0, default 0	Harmful for structured output
`stop`	Stop sequences	list of strings
`seed`	Best-effort reproducibility	integer	Not guaranteed, pin model version too
`response_format`	text, json_object, or JSON schema
`reasoning_effort`	Reasoning models only	low / medium / high	Higher means more tokens, latency, cost

The guidance on temperature versus top_p is consistent across Microsoft and OpenAI: alter one or the other, not both. My default is to leave top_p at its default and tune temperature, reaching for top_p only when I want to keep temperature fixed for style but trim the occasional weird token. A common enterprise default for GPT-style chat is temperature 0.2 to 0.3 with top_p 0.8 to 0.95.

Reasoning models behave differently and this catches teams out. The GPT-5 series and o-series (o1, o3, o3-mini, o4-mini) reject temperature, top_p, presence_penalty, frequency_penalty, logprobs, logit_bias, and max_tokens. Sending temperature typically returns a 400 “Unsupported parameter”. Instead you use reasoning_effort (low/medium/high, with newer models adding none/minimal/xhigh) and max_completion_tokens on chat completions or max_output_tokens on the Responses API. A wrinkle to watch: gpt-5.1 defaults reasoning_effort to none, so migrating from an earlier reasoning model may require you to pass an effort level explicitly to get any reasoning at all. System messages are treated as developer messages on the o-series.

Reasoning models reject temperature, top_p, and the penalties, and substitute reasoning_effort and max_completion_tokens.

The practical fix is a shared wrapper that branches on model family and strips unsupported parameters before the call. That one piece of plumbing prevents most of the 400 errors that bite teams moving to GPT-5.

What I would actually do

Default to Global Standard, then narrow only for a reason: Data Zone when EU or US residency is required (accept roughly +10%), Regional only for strict single-region residency (accept +10% to +25% and smaller capacity). Do not buy PTU on a guess: run pay-as-you-go for 30 to 60 days, measure P95 hourly throughput, and commit to a 1-year reservation only once sustained volume sits in the 150 to 200M tokens/month range for GPT-5-class.

Turn on the free discounts. Route anything async to Batch, make system prompts byte-identical for cached-input savings, and send routing, extraction, and classification to a mini or nano model while reserving flagships for the hard cases. Add spillover to any provisioned customer-facing deployment, but remember it does not cover the Responses API.

Gate model choice on evaluation, not vibes, and make parameter handling model-aware. Those two habits, plus standardising on azure-ai-projects 2.x with DefaultAzureCredential, have removed most of the surprises my teams used to hit in production.

References

Image credits

All diagrams in this post are my own. They illustrate concepts documented on Microsoft Learn (linked in the References above); pricing figures shown are indicative and should be verified on the Azure pricing calculator.

June 21, 2026

Microsoft IQ: the intelligence layer your agents inherit — and what it actually changes for enterprise AI builders
For a few years now, every enterprise agent I built started the same way: from scratch. A new connector here, a handcrafted retrieval pipeline there, a fresh attempt to teach the agent what the business already knew. I built agent grounding the hard way more than once — custom RAG, bespoke chunking, my own identity plumbing — and watched two agents in the same tenant give two different answers to the same question because they had been grounded against two different copies of reality.

That is the production problem nobody puts on a slide. Context gets rebuilt per agent. Connectors sprawl. Answers drift. And the cost of all that plumbing lands on the same small group of engineers every time a new use case appears.

Microsoft IQ is Microsoft’s answer to that problem, and it reached general availability at Build 2026. This is the pillar post for the wider Microsoft IQ cluster on this blog: what each layer does, how they compose, and — just as important — where the GA reality differs from the keynote framing.

What Microsoft IQ actually is

Microsoft IQ is a shared, permission-aware intelligence layer that agents inherit, rather than rebuild. The pitch is simple: stop stitching connectors and pipelines into every agent, and ground them all against one governed view of how people work, how the business operates, and how to reuse knowledge.

It is worth being precise about what it is not. Microsoft IQ is not a model, and it is not a chatbot feature. It sits underneath the agents you build in Copilot Studio, Microsoft Foundry, or GitHub Copilot, and feeds them context. It is composed of four layers — Work IQ, Fabric IQ, Foundry IQ, and Web IQ — and any agent across those three build surfaces can consume them.

The “GA” label, though, is an umbrella, not a uniform guarantee. Each of the four layers shipped at a different stage of maturity, and anyone scheduling a deployment this quarter — picking what to commit to in the next planning cycle — needs that breakdown up front, not buried in the small print. So I will give each layer its own section, with the state it actually shipped in.

Work IQ — the workplace context layer

Work IQ is the contextual intelligence layer for Microsoft 365. It captures the signals that describe how people actually work — emails, meetings, documents, Teams messages, people relationships, and collaboration patterns — and exposes them so an agent can reason over them in natural language.

The capability ships as a CLI and a Model Context Protocol server today, which is how AI assistants such as GitHub Copilot reach into a user’s M365 context. The broader public Work IQ APIs — REST, A2A, and MCP — are slated to reach GA on 16 June 2026.

The one thing an architect should know: as of writing, Microsoft Learn still labels Work IQ “public preview”, and accessing organisation data requires admin-consented permissions and tenant billing activation. Treat the GA date as imminent rather than banked, and plan the admin-consent step into your rollout — it is not a developer-self-serve switch.

Fabric IQ — the business-data semantic layer

Fabric IQ is the semantic layer over your business data. It elevates raw analytical, real-time, and operational data in OneLake into the language of the business — entities, relationships, rules, and actions — so that agents reason in terms of Customer, Shipment, or Breach rather than table columns.

It delivers this through two core items: semantic models and an ontology. And here is the GA nuance Microsoft’s umbrella headline glosses over — the Fabric IQ ontology is in preview. Learn marks it “ontology (preview)” consistently. The semantic-model side is more mature, and ontologies can be generated directly from Power BI semantic models already running in production, which is the realistic on-ramp for most estates that have years of Power BI behind them.

The one thing an architect should know: you can bootstrap a Fabric IQ ontology from an existing Power BI semantic model, keeping business terminology consistent across reports, agents, and apps — but treat the ontology itself as preview-grade until Microsoft says otherwise.

Foundry IQ — the managed knowledge layer

Foundry IQ is the layer that replaces the most plumbing, so it earns the most depth here. It turns fragmented enterprise content into governed, reusable knowledge that multiple agents can share.

The model has three concepts worth learning precisely. A knowledge base is the top-level resource that orchestrates retrieval and carries a retrieval reasoning-effort setting of minimal, low, or medium. It is composed of knowledge sources — connections to indexed or remote content such as Azure Blob Storage, SharePoint, OneLake, the web, MCP, Azure SQL, and File Search. And it is queried through agentic retrieval: an LLM plans the query, decomposes a complex question into parallel subqueries across sources, semantically reranks the results, and returns extractive answers with citations the agent can trace.

Azure AI Search provides the underlying infrastructure. Crucially, a knowledge base is shareable: one knowledge base can ground many agents, and those agents can run in Foundry Agent Service, the Microsoft Agent Framework, or any custom app via the Azure AI Search knowledge base APIs. At Build 2026, Microsoft positioned Foundry IQ knowledge bases as the unifying point — bringing Work IQ, Fabric IQ, File Search, Azure SQL, and MCP behind a single, SLA-backed retrieval endpoint.

Now the candour. Foundry IQ’s GA is uneven, and Microsoft says so on the concept page itself: some features are generally available while others remain in preview, and which is which depends on the Search Service REST API version you call. The same page notes that the Foundry portal and Azure portal still expose all agentic retrieval features as preview-only. So “Foundry IQ is GA” is true and incomplete at the same time — the answer depends on how you call it.

The one thing an architect should know: before you plan a production rollout, pin down which Search Service REST API version your code targets, because that single choice determines whether you are on a GA or a preview surface.

Web IQ — the live web-grounding layer

Web IQ is the newest layer, announced at Build 2026. It is web grounding rebuilt for LLMs and multi-step agents: a suite of AI-native APIs returning ranked, citation-ready context across web pages, news, images, and video, built on two decades of Bing infrastructure rather than SERP scraping.

The engineering numbers are the headline. Microsoft claims roughly 164ms P95 latency — close to 2.5x faster than the best alternative — with fewer tokens per query. It is model-agnostic and MCP-native over JSON-RPC 2.0, so there is no inference lock-in, and it is benchmarked against suites including DeepSearchQA.

The one thing an architect should know: Web IQ is limited access and waitlist-only today, prioritised for enterprise customers working with Microsoft account teams. If live web grounding matters to your roadmap, the action this month is to join the waitlist, not to design around guaranteed availability.

How the layers compose

The architecture is cleaner than the four-product naming suggests. The four IQ layers feed a shared intelligence layer that any agent — GitHub Copilot, a Foundry agent, or a Copilot Studio agent — consumes. Foundry IQ knowledge bases act as the retrieval hub that the others can flow through. Cutting across all of it is governance: queries run under the caller’s Microsoft Entra identity, ACLs synchronise for supported sources, and Microsoft Purview sensitivity labels are enforced end to end.

Microsoft IQ architecture. Work IQ, Fabric IQ, Foundry IQ, and Web IQ feed a shared intelligence layer that agents inherit, with Entra identity and Purview labels as a cross-cutting governance band. (Diagram placeholder — Mermaid source in the image plan.)

The retrieval path inside Foundry IQ is the part most worth internalising, because it is where the “no complex RAG” claim meets reality: a query is planned by an LLM, decomposed into parallel subqueries across sources, semantically reranked, and returned as a cited answer.

Foundry IQ agentic retrieval. Query, LLM query planning, parallel subqueries across knowledge sources, semantic rerank, cited answer. (Diagram placeholder — Mermaid source in the image plan.)

Where this falls short today

“No complex RAG” is a marketing claim, not an engineering fact. Foundry IQ genuinely removes connector and pipeline plumbing, and that is real value. It does not remove the need to understand chunking behaviour, to evaluate retrieval quality, or to model cost. You are not maintaining the pipeline; you are still responsible for whether it returns the right thing at a price you can defend.

The GA label is uneven. Microsoft IQ is GA as an umbrella, but Fabric IQ’s ontology is preview, Web IQ is waitlist-only, and Foundry IQ’s GA depends on which Search REST API version you call. If you are signing off a production plan, build that table honestly: per layer, what is GA, what is preview, what is gated.

Microsoft has promised programmable enterprise context before. Microsoft Graph was supposed to be this a decade ago. Semantic Kernel memory abstractions and earlier Copilot extensibility models each hit walls — latency, schema drift, identity resolution. What is different this time is a retrieval-planning layer doing the grounding and GA APIs rather than perpetual preview. That difference is meaningful. It is also unproven at production scale, and I would hold both thoughts at once.

Lock-in is the trade. An intelligence layer this deep couples your agent estate to the Microsoft stack. For organisations already on M365, Fabric, and Dynamics, that is leverage of investment you have already made. For hybrid estates, it is a strategic decision to make deliberately, not a default to drift into.

Pricing and billing clarity is thin at GA. Work IQ requires tenant billing activation, and the consumption models across the layers are not yet fully documented. Before you commit a budget, get your Microsoft account team to put the per-layer billing model in writing — that is the gap most likely to surprise you later.

What I would do this month

Concrete, role-aware next steps if you own enterprise agent architecture:
- Pin the Search Service REST API version your Foundry IQ code targets, and confirm whether that puts you on a GA or a preview surface before anything reaches production.
- Plan for the Work IQ API GA on 16 June 2026 — including the admin-consent and tenant-billing steps — rather than assuming developer self-serve.
- Bootstrap a Fabric IQ ontology from an existing Power BI semantic model in a sandbox, so you learn the preview behaviour without betting a production workload on it.
- Join the Web IQ waitlist now if live web grounding is on your roadmap, because availability is gated and lead time is unknown.
- Ask your account team for the per-layer consumption and billing model in writing before you size a budget.
Where this cluster goes next

This pillar is deliberately broad. Each layer deserves its own engineering deep-dive, and those are coming — starting with Foundry IQ versus a do-it-yourself RAG pipeline, the comparison I get asked about most. Fabric IQ ontologies, the Work IQ API surface, and Web IQ’s grounding economics will each get their own treatment. When those land, this post will link out to them from the sections above.

The short version: Microsoft IQ is the first time the “shared enterprise context” promise has arrived with a retrieval-planning layer and real GA APIs behind it. That is worth taking seriously. It is also worth reading the GA label one layer at a time.

References
Image credits

The layer diagrams in this post are reused from the Microsoft IQ product page with attribution to Microsoft:
- Banner and the four layer illustrations (Work IQ, Fabric IQ, Foundry IQ, Web IQ): Microsoft IQ
The two architecture diagrams are my own. All other commentary, code, and opinions in this post are my own and reflect lessons from building enterprise agent grounding the hard way.
June 10, 2026
Authentication patterns for Microsoft Foundry — beyond DefaultAzureCredential
DefaultAzureCredential is the right default, and I said as much in the getting-started guide that this post follows. It walks an ordered chain — environment variables, managed identity, Azure CLI, VS Code, interactive browser — and the same line of code works on a laptop, in CI, and on production compute. That is exactly why it earns its place on day one.

The trouble starts by the time you hit production, when the questions get more specific. Your production workload needs to authenticate as something stronger than “whichever managed identity the host happens to provide.” Your CI/CD pipeline has to deploy agents, model deployments, and role assignments without a client secret sitting on the build agent. Your app calls Foundry on behalf of a signed-in user, and the user’s own identity has to reach Foundry — both for RBAC and for audit. And a security review asks for a complete inventory of who can call what, and “DefaultAzureCredential” is not an answer to that question.

What follows is the auth pattern catalogue I wish I had when I went from prototype to production on Foundry. Five patterns, a per-environment role assignment model, the multi-environment story, and the four things that will bite you.

The big picture — one diagram

Before the catalogue, the one diagram that summarises the relationships. Every identity — a developer’s laptop, a signed-in end user, a workload on Azure compute, a CI/CD pipeline — reaches Foundry by way of an Entra-issued access token. The pattern you pick determines how that token is minted, not whether Entra is in the loop.

Authentication architecture for Microsoft Foundry. Every calling identity reaches Foundry via an Entra-issued access token.

1. The auth pattern catalogue

1.1 System-assigned managed identity for single-resource workloads

When to use it. A single App Service, Function, or Container App that calls one Foundry resource, has no shared identity needs with anything else, and never has to outlive its host.

When not. Anything where two compute resources need the same identity, or where the identity must persist across redeploys.

Trade-off. System-assigned managed identities are created and deleted with their host. Zero lifecycle work, zero secrets, and zero portability. If you delete the App Service, the identity is gone — along with every role assignment that ever referenced it.
resource app 'Microsoft.Web/sites@2023-12-01' = { name: 'app-foundry-prod' location: location identity: { type: 'SystemAssigned' } properties: { serverFarmId: plan.id } } // Assign Foundry User on the project (not the resource) resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(project.id, app.id, foundryUserRoleId) scope: project properties: { principalId: app.identity.principalId principalType: 'ServicePrincipal' // Foundry User role ID — stable across the rename roleDefinitionId: subscriptionResourceId( 'Microsoft.Authorization/roleDefinitions', '53ca6127-db72-4b80-b1b0-d745d6d5456d' ) } }
System-assigned managed identity lifecycle. The identity is created with the host and deleted with it — taking every role assignment with it.

1.2 User-assigned managed identity for shared and durable workloads

When to use it. Multiple compute resources sharing one identity (App Service plus a Function, two AKS workloads, a Container App plus a Logic App). Or anywhere the identity must survive a redeploy of the compute.

When not. A single transient workload — system-assigned is simpler, and you do not have an identity hanging around with no host.

Trade-off. Durable and shareable, but you own the lifecycle. Think of it as identity-as-a-resource: it gets its own Bicep module, its own naming convention, and its own teardown plan.
resource uami 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: 'id-foundry-app-prod' location: location } resource app 'Microsoft.Web/sites@2023-12-01' = { name: 'app-foundry-prod' location: location identity: { type: 'UserAssigned' userAssignedIdentities: { '${uami.id}': {} } } properties: { serverFarmId: plan.id } } resource projectRole 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(project.id, uami.id, foundryUserRoleId) scope: project properties: { principalId: uami.properties.principalId principalType: 'ServicePrincipal' roleDefinitionId: subscriptionResourceId( 'Microsoft.Authorization/roleDefinitions', '53ca6127-db72-4b80-b1b0-d745d6d5456d' ) } }
User-assigned managed identity shared across App Service, Function, AKS, and Container Apps — one identity, one role assignment, multiple workloads.

For anything in production, my default is user-assigned. The first time you redeploy a Container App and discover every role assignment has gone with it, you will thank yourself.

1.3 Workload identity federation for GitHub Actions and other federated CI/CD

When to use it. Any pipeline that deploys Foundry agents, model deployments, role assignments, or any other RBAC-protected operation. GitHub Actions, Azure DevOps with OIDC, Terraform Cloud, AKS workload identity — all federated subjects.

When not. There is not a good “when not.” If your GitHub Actions workflow still has AZURE_CLIENT_SECRET in its repository secrets, you should be migrating off it.

Trade-off. A bit of configuration up front — a federated credential on the app registration with the right subject claim and audience. Zero credential rotation forever after. The external identity provider (GitHub, Kubernetes, etc.) is trusted to assert the workload’s identity, and Entra exchanges that assertion for a token. No client secret ever crosses the wire.
# Create the federated credential on an app registration az ad app federated-credential create \ --id $APP_ID \ --parameters '{ "name": "github-main-prod", "issuer": "https://token.actions.githubusercontent.com", "subject": "repo:my-org/my-repo:ref:refs/heads/main", "audiences": ["api://AzureADTokenExchange"] }'
# .github/workflows/deploy.yml permissions: id-token: write # required to mint the OIDC token contents: read jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: azure/login@v2 with: client-id: ${{ secrets.AZURE_CLIENT_ID }} tenant-id: ${{ secrets.AZURE_TENANT_ID }} subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} enable-AzPSSession: false - run: az deployment group create ...
Workload identity federation trust between GitHub Actions and Microsoft Entra ID. The runner sends an OIDC token, Entra validates it against the federated credential, and returns a Foundry-scoped access token.

The pattern generalises. AKS workload identity uses the same federation primitive with the cluster’s OIDC issuer as the subject. Terraform Cloud has its own. The configuration changes; the model does not.

1.4 On-Behalf-Of flow for apps that call Foundry as the signed-in user

When to use it. A web app or API where the end user’s identity must reach Foundry — because the user’s own RBAC determines what they can see, because audit logs need the user not the app, or because a compliance regime requires per-user attribution all the way to the model call.

When not. Pure machine-to-machine workloads. If there is no signed-in human in the loop, you want a managed identity, not OBO.

Trade-off. More moving parts. The user signs into the front end, the front end calls your API with their access token, the API exchanges that token for a downstream token scoped to Foundry, and only then does the call go through. It is the only correct answer for user-scoped operations.
# Middle-tier API: exchange the incoming user token for a Foundry-scoped token import msal app = msal.ConfidentialClientApplication( client_id=API_CLIENT_ID, client_credential=API_CLIENT_SECRET, # or a certificate / federated credential authority=f"https://login.microsoftonline.com/{TENANT_ID}", ) # incoming_user_token comes from the Authorization header on the request result = app.acquire_token_on_behalf_of( user_assertion=incoming_user_token, scopes=["https://ai.azure.com/.default"], ) foundry_access_token = result["access_token"]
On-Behalf-Of flow. The middle-tier API exchanges the user token for a Foundry-scoped token, and the call runs under the user identity with their RBAC and Conditional Access applied.

One implication worth calling out: any Conditional Access policy on the user’s original sign-in propagates through the OBO exchange. If your CA policy says “no Foundry access from non-compliant devices,” the downstream Foundry call inherits that. That is almost always what you want.

1.5 Application registrations with client secrets — when (rarely) still appropriate

When to use it. Local developer machines that are not on a corporate-managed laptop with Entra-joined credentials. Genuinely headless scripts that cannot use a managed identity or federated workload identity. Third-party integrations that do not yet support OIDC federation. That is it.

When not. Anything in production on Azure compute — use a managed identity. Anything in CI/CD on a platform that supports federation — use workload identity federation. Anything an auditor will ever look at.

Trade-off. Simplest to set up, hardest to govern. Secrets rotate, they leak, they accumulate. If you have more than a handful, you have a secret-sprawl problem and you do not yet know it.

If you must use one: short expiry (90 days), stored in Key Vault, never in a .env checked into a repo, and the role assigned to the app’s service principal is the minimum it needs — Foundry User scoped to the project, never Contributor scoped to the subscription.

The hard line: if you are putting a client secret on a production workload, you have taken a wrong turn. Go back and use one of the four patterns above.

Client secrets in production are an anti-pattern. Replace with managed identity for Azure compute, workload identity federation for CI/CD, or On-Behalf-Of for signed-in user apps.

2. The role assignment model — least privilege without the spreadsheet

Two principles. Roles are assigned to principals — managed identities, user accounts, Entra groups — at a scope. The scope can be project, Foundry resource, resource group, or subscription. Get the scope right and least privilege follows naturally. Get it wrong and you will be re-assigning Contributor every six months because somebody got blocked at a demo.

In prose, here is the model I deploy:

Application principals — the managed identity that the production app authenticates as, the federated workload identity the AKS pod assumes — get the Foundry User role, scoped to the project, not the resource. Project-scoped assignments mean a misconfigured app cannot accidentally see another project’s agents, threads, or connections.

Build and deploy principals — the federated CI/CD identity that runs your GitHub Actions workflow — get Foundry Project Manager scoped to the project. If the same pipeline also creates projects, then it needs a resource-level role for that one operation; keep it as narrow as you can get away with.

Human developers get Foundry Project Manager on the dev project, Foundry User on staging, and read-only on prod. Production changes go through the pipeline; they do not go through individual developer accounts.

Resource-level roles — Foundry Account Owner and Foundry Owner — are platform-team territory, and even there they should be PIM-eligible rather than standing assignments. These are the roles that can create new projects, configure guardrails, and conditionally hand out other roles. Treat them accordingly.

A few practical notes the docs are explicit about. Do not assign built-in roles that start with Cognitive Services for Foundry work — Microsoft’s RBAC documentation calls this out directly. Those roles are for accessing AI Services resources directly and do not apply to Foundry scenarios, even though Foundry sits on the Microsoft.CognitiveServices resource provider. Also avoid the Azure AI Developer role for Foundry — despite the name, it is scoped to Azure Machine Learning workspaces and Foundry hubs, not to Foundry projects or resources.

One more practical note: reference role definition GUIDs in Bicep and Azure CLI, not display names. The Foundry roles were recently renamed from their Azure AI predecessors (Azure AI User → Foundry User, Azure AI Project Manager → Foundry Project Manager, Azure AI Account Owner → Foundry Account Owner). The GUIDs are stable; the display names are still mid-rollout across the portal and tooling.

Role assignment model. Application principals get Foundry User on the project, CI/CD and developer principals get Foundry Project Manager, and resource-level Foundry Account Owner / Foundry Owner stay with the platform team. Avoid Cognitive Services * roles and Azure AI Developer for Foundry work.

3. The multi-environment story

Dev, staging, and prod each get their own Foundry resource — not just their own project. Quotas are resource-scoped. Network configuration is resource-scoped. The blast radius of a misconfigured role assignment is resource-scoped. All of those argue for full resource separation between non-prod and prod, even if it means three sets of Bicep modules and three Application Insights workspaces. The cost of running an under-utilised dev resource is far less than the cost of an intern accidentally pointing a load test at a prod deployment.

Each environment gets its own user-assigned managed identity for the application principal, its own federated credential on the CI/CD app registration (one per environment, with a distinct subject claim — environment:dev, environment:prod — so prod deploys only run from protected branches and reviewed environments), and its own Entra group for human access. Group membership rather than direct user assignment, always — that is how you get clean joiner/mover/leaver flows without a quarterly spreadsheet review.

Secrets that genuinely have to exist — third-party API keys, database connection strings — live in a per-environment Key Vault, accessed by the per-environment managed identity. Foundry credentials themselves are never in Key Vault. They are token exchanges via the patterns in Section 1.

Elevated roles on the prod resource go through Privileged Identity Management. The platform team holds Foundry Owner on prod as PIM-eligible, not as a standing assignment. Activation requires justification, a time window, and an audit trail. If your auditor asks “who could have changed the prod guardrails on this date,” you want PIM logs to answer that, not Azure Activity Log archaeology.

Per-environment isolation. Dev, staging, and production each get their own Foundry resource, user-assigned managed identity, federated credential, and Key Vault. Elevated roles on prod are PIM-eligible only.

4. The four things that will bite you

Token caching. The Azure SDK clients cache tokens for the lifetime of the credential object. Long-lived processes — anything stateful, anything that processes a queue, anything with a connection pool — need to handle credential refresh correctly. The right pattern is usually to reuse a single credential instance across all clients in the process, not to recreate DefaultAzureCredential() (or its successor) per call. Recreating it per call defeats the cache and, on a busy worker, will get you rate-limited at the IMDS endpoint before you have shipped a single completion.

Cross-tenant scenarios. Foundry resources live in a single tenant. If you have a partner tenant whose users need to call your Foundry workload, you are in B2B territory and the patterns above need adapting. Managed identities do not cross tenants without explicit federation, and OBO has its own constraints when the user is a guest. Do not discover this two weeks before a launch — design for the tenant model on day one.

Private endpoints and DNS. Authentication works, the call still fails. If you have put Foundry behind a private endpoint, the DNS for the resource FQDN must resolve to the private IP from the calling network. Public DNS will look correct, your nslookup from a different network will look correct, and the call from inside the VNet will time out with no useful error. Always check resolution from the calling subnet, not from your laptop.

Role propagation latency. New role assignments take up to ten minutes to propagate. Pipelines that create a user-assigned managed identity and immediately use it against Foundry will hit 403s on the first run. Options: insert a wait step after role assignment, retry with exponential backoff in the calling code, or assign roles ahead of provisioning the compute they are attached to. I prefer the third — the assignment is declarative and the compute picks it up when it comes online.

Four things that will bite you in production: stale tokens in long-lived processes, cross-tenant scenarios needing multi-tenant app registrations, private-endpoint DNS failures, and the up-to-ten-minute delay before new role assignments take effect.

5. When NOT to add another auth pattern

Counterweight, briefly. If your workload is one App Service calling one Foundry resource for one tenant’s users, deployed by one GitHub Actions workflow, you do not need four patterns. You need a user-assigned managed identity on the App Service and a federated workload identity for the pipeline. Stop there. Adding OBO, custom token exchange, or a second managed identity because “we might need it later” is the kind of architecture work that looks responsible in a design doc and creates three years of operational debt.

And if you find yourself building a custom token-exchange layer — your own service that sits in front of Foundry and stamps tokens on requests — you are almost certainly reinventing something Entra already does. Read the workload identity federation and OBO docs again before you write more code. The thing you are about to build is probably a federated credential with the wrong subject claim.

6. Closing

DefaultAzureCredential is how you start. The patterns in this post are how you scale. Pick the right managed identity flavour for the workload’s lifecycle. Federate your CI/CD so no client secret ever lives on a build agent. Use OBO where the user’s identity has to reach Foundry, and do not use it where it does not. Get the role scope right at the project level. Separate environments by resource, not just by project.

References
May 22, 2026
Starting an Azure Foundry project — the getting-started guide nobody wrote
Banner image: Microsoft Foundry. Source: Microsoft Tech Community — Introducing Microsoft Foundry.

Most “getting started with Foundry” content is a screenshot tour of the portal. You watch someone click “Create resource,” pick a region from a dropdown, and end the post with a chat playground saying “Hello, world.” None of that helps you on Monday morning when you have to commit to a region, an auth pattern, and a project topology that you’ll be living with for the next year.

This is the post I wish I’d had open in another tab when I started TrafficIQ, our multi-agent supply-chain transport intelligence build on Foundry Agent Service. Five decisions you make before you click Create, the auth pattern you should adopt from day one, a first-sprint checklist, and the three things that will bite you.

1. The naming maze — what Foundry actually is in 2026

Eighteen months ago you had four products: Azure OpenAI, Azure AI Studio, Azure AI Services, and a sprawling Cognitive Services back catalogue. Today you have one Azure resource type — kind: AIServices with allowProjectManagement: true — and Microsoft calls it Microsoft Foundry (formerly Azure AI Foundry). Single resource, single ARM object, and three FQDNs hanging off it: the Azure OpenAI-compatible inference endpoint, the cognitive-services endpoint, and the Foundry project endpoint your agents and Responses API code talks to.

There are also two portals. Foundry (classic) is the hub-based experience that grew out of Azure AI Studio. Foundry (new) is the project-first experience built around the consolidated resource. Both still work. Classic is in maintenance mode. If you are starting a new project in 2026, start in the new portal and create a Foundry project — not a hub project. Hub projects still exist for backwards compatibility, but everything Microsoft is investing in — agent service, evaluations, the new model catalogue, observability — is wired up around Foundry projects first.

One more piece of context before you create anything: the Assistants API retirement deadline of 26 August 2026 is real. If you are building anything new today, do not start on Assistants — go directly to Foundry Agent Service and the Responses API. I’ll cover the migration path in a dedicated post; for now, treat Assistants as legacy.

Microsoft Foundry resource and project architecture. Source: Microsoft Learn — Microsoft Foundry architecture.

2. The five decisions you make before you click Create

2.1. Foundry resource vs upgrading an existing Azure OpenAI resource

Decision: create a brand-new Foundry resource, or upgrade an existing Azure OpenAI resource in place. Trade-off: the in-place upgrade keeps your existing endpoint, deployments, network config, and RBAC bindings — but it requires a system-assigned managed identity on the source resource and is one-way once you commit (rollback exists but is a support operation, not a button).

For TrafficIQ: new resource. The repo was greenfield, I wanted a clean project boundary, and I didn’t want to inherit eighteen months of ad-hoc role assignments from the old Azure OpenAI resource.

2.2. Region

Decision: which Azure region hosts the resource. Trade-off: model availability is not uniform. Sweden Central, East US 2, and France Central each have meaningfully different model catalogues, and frontier models often land in one region weeks before the others. Pick the wrong region and you’ll either rewrite code against a different deployment or pay cross-region latency. For TrafficIQ: Sweden Central. TrafficIQ shipped on gpt-4.1 and gpt-4.1-mini, and Sweden Central was the region that aligned with both the model availability I needed and my EU data-residency obligations. Starting fresh today, I’d still default to Sweden Central but I’d evaluate gpt-5-mini for the router/orchestrator.

2.3. New portal vs classic portal

Decision: which portal you do your work in. Trade-off: classic gives you hub projects (good if you have an existing hub and shared compute), new gives you Foundry projects (better isolation, simpler RBAC, where all the new features land first).

For TrafficIQ: new portal, Foundry project. No hub.

2.4. Single project vs multiple projects per resource

Decision: how many projects to carve out of one Foundry resource. Trade-off: projects are the isolation and RBAC boundary in Foundry — a project owns its agents, threads, evaluations, connections, and the people who can see them. One project is simpler; multiple projects are how you separate prod from dev, or two workloads that should never see each other’s data.

For TrafficIQ: I started with a single project and split as soon as evaluations grew enough to need their own connections and quotas. The pattern I’d recommend day one: two projects per environment — one for the agent runtime, one for evaluations and offline experiments — and prod in a separate Foundry resource entirely from non-prod, so a misconfigured RBAC binding can never reach production data.

2.5. Direct Foundry-billed models vs Azure Marketplace third-party models

Decision: how you procure non-OpenAI models — Anthropic, Cohere, Mistral, Meta, and the rest. Trade-off: direct (first-party in the Foundry catalogue, billed on your Azure invoice, full enterprise SLA, no separate contract) versus Azure Marketplace (third-party publisher, often the only way to get the very latest version of a partner model, but it’s a separate offer you have to accept and the billing line lands differently).

For TrafficIQ: direct for everything I could, marketplace only where a specific model version wasn’t available first-party. One Azure invoice is worth real money in procurement time.

3. Authentication and authorisation — the day-one setup

If you take one thing from this post, take this: don’t use API keys. Foundry resources support Entra ID (Azure AD) authentication everywhere, and DefaultAzureCredential from azure-identity is the right pattern from day one. Keys feel quick on day one and become a rotation, secrets-sprawl, and audit nightmare by month three.

The pattern I use in TrafficIQ, lifted down to its essentials:
from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient # DefaultAzureCredential walks an ordered chain: # env vars -> managed identity -> Azure CLI -> VS Code -> interactive # Same line of code works locally, in CI, and in production. credential = DefaultAzureCredential() project = AIProjectClient( endpoint="https://<your-foundry-resource>.services.ai.azure.com/api/projects/<project-name>", credential=credential, ) # Now you can use Agents, Responses, evaluations, connections — # all authenticated as the principal the host environment provides. agents = project.agents
There are three roles you’ll actually find yourself assigning in the first week. Microsoft renamed these in the last release wave; both old and new names still appear across the portal and docs during the rollout, but the new names are what you should write into runbooks.
- Foundry User (formerly Azure AI User) — read/use existing agents, run inference, call the Responses API. This is the role for your application’s managed identity in production, and for engineers who consume but don’t author. Role ID: 53ca6127-db72-4b80-b1b0-d745d6d5456d.
- Foundry Project Manager (formerly Azure AI Project Manager) — create and modify agents, manage connections, deploy models into the project. The role for developers actually building. Role ID: eadc314b-1a2d-4efa-be10-5d325db5065e.
- Foundry Account Owner (formerly Azure AI Account Owner) — resource-level operations like creating new Foundry resources and configuring guardrails. The elevated tier. Don’t grant casually.
Two practical notes. In Azure CLI and Bicep, use the role definition GUIDs, not the names — names are still mid-rename and the GUIDs are stable. And don’t grant any role that starts with “Cognitive Services” for Foundry work. The Microsoft Learn RBAC doc explicitly calls these out as not applicable to Foundry, even though Foundry sits on the Microsoft.CognitiveServices provider under the hood.

Foundry User role (formerly Azure AI User), scoped at the Foundry resource. Source: Microsoft Learn — RBAC for Microsoft Foundry.

In production, the application principal is a managed identity — a user-assigned managed identity attached to your App Service, Container App, AKS workload identity, or Function. App registrations with client secrets are for local development and headless CI/CD only. If you find yourself putting an app registration secret on a production workload, you’ve taken a wrong turn — go back and attach a managed identity instead.

Secrets that genuinely have to exist — third-party API keys, database connection strings, anything that isn’t a Foundry credential — live in Azure Key Vault and are injected at build time, not runtime where possible. TrafficIQ uses a Vite Key Vault plugin pattern for the frontend so that the bundle never contains a literal secret and the build agent’s managed identity is the only thing that ever touches the vault.

One last thing the docs bury and I wish someone had said louder: private endpoints are the most-forgotten production step, and you have to recreate them after an in-place upgrade from Azure OpenAI to Foundry. The upgrade preserves most of your network configuration, but private endpoints targeting the new Foundry sub-resources need to be re-provisioned, and DNS will be wrong until you do. Put it on the upgrade runbook.

Network isolation plan for Microsoft Foundry. Source: Microsoft Learn — Configure network isolation for Microsoft Foundry.

4. The first sprint — a working checklist

In order. One line on what to do, one line on the trap.
1. Create the Foundry resource. Use kind: AIServices, allowProjectManagement: true, system-assigned managed identity on. Trap: if you let someone create it as a vanilla Azure OpenAI resource “for now,” you’ll be doing an upgrade migration in week three.
2. Create the first Foundry project. Give it a name that survives renames — <workload-<env works. Trap: project name is in the endpoint URL, so renaming later means client config changes everywhere.
3. Assign roles, not keys. Azure AI Project Manager for builders, Azure AI User for the app’s managed identity. Trap: don’t grant subscription-level Contributor “just to unblock the demo” — it never gets revoked.
4. Set up Key Vault and managed identity. One vault per environment, user-assigned managed identity attached to your compute. Trap: system-assigned MIs disappear when you delete the compute resource; use user-assigned for anything you care about.
5. Deploy a model. A reasonable default in 2026: gpt-5-mini for router/orchestrator agents and gpt-4.1 for specialists with heavier tool-calling. Trap: model availability is regional — check the catalogue in your target region before you write code against a specific deployment name.
6. Wire a connection for any external data source. Foundry “connections” are the project-scoped credential store for storage accounts, search indexes, and tools. Trap: connections live inside the project — copy them when you split prod from dev, don’t share.
7. Call the Responses API from a smoke-test script. AIProjectClient → get inference client → responses.create. Trap: if you copy a sample using the legacy chat-completions endpoint, you’ll miss the new tool-calling and reasoning surface entirely.
8. Stand up your first agent in Foundry Agent Service. Tools, instructions, model — keep it boring. Trap: don’t start with a mega-agent; start with one narrow agent and add a second before you make the first one cleverer.
9. Turn on Guardrails and review the defaults. They are on by default at “medium” across categories. Trap: defaults block legitimate enterprise content — see Section 5.
10. Wire up observability before you ship. Application Insights connection on the project, distributed tracing through opentelemetry, Foundry’s built-in run/thread tracing on. Trap: adding observability after the fact is two orders of magnitude harder than turning it on now.
5. The three things that will bite you in the first sprint

Quota. Tokens-per-minute (TPM) and requests-per-minute (RPM) limits are per-deployment and per-region, and the default quota you get on a fresh subscription is sized for demos, not production. The day you flip a real workload on, you will hit 429s. Mitigations: request quota increases early (the form is slow), spread deployments across multiple regions if your latency budget allows, and put Provisioned Throughput Units (PTU) under anything customer-facing where you cannot tolerate rate-limit jitter.

Guardrails (formerly content filters). Foundry’s Guardrails system is on by default with sensible consumer settings — and it will block legitimate enterprise content. Customer-complaint emails trip the harm filter. Security logs trip the violence filter. Code review of an exploit-handling library trips multiple. You can tune controls per-model and per-agent under Guardrails in the portal, define custom guardrails with their own controls, and apply them at four intervention points: user input, tool call, tool response, and output (the final completion returned to the user). Audit the defaults the day you deploy your first model, not the day a business user shows you a screenshot of a blocked legitimate prompt.

Observability. Foundry exposes distributed traces, per-run token accounting, evaluation hooks, and a thread/run viewer in the portal — but only if you wire it up. Wire it up on day one. The cost of adding tracing to a quiet new system is an afternoon. The cost of adding tracing to a live multi-agent system with real users is a sprint and a half, plus the customer trust you spend debugging the bug you can’t see.

6. When NOT to use Foundry

I’m bullish on Foundry, but it isn’t the answer to every question.

If you have exactly one OpenAI model in production and a stable PTU reservation on it, defer the upgrade. The in-place upgrade is non-trivial, and you get nothing from it if you aren’t using agents, evaluations, or the broader catalogue. Revisit when one of those becomes a “yes.”

If you need offline or on-device inference — air-gapped environments, edge devices, sub-10ms latency budgets — you want Foundry Local, not cloud Foundry. Same model story, very different deployment shape, and trying to make cloud Foundry pretend to be local will end badly.

If you have a price-sensitive, non-enterprise workload with no Entra or Azure compliance requirement — a side project, a hobby tool, a community OSS app — going direct to OpenAI’s or Anthropic’s API is still cheaper and operationally simpler. Foundry’s value is enterprise: SSO, RBAC, private networking, compliance attestations, one invoice. If you don’t need those, you’re paying for them anyway.

7. Closing — and what’s next

Foundry rewards a small amount of up-front thinking. Pick the region for the models you actually need. Use Entra and managed identities from line one of code. Multi-project from the start if you’re going to run more than one environment. Turn on observability before the first user hits the first endpoint. Re-do your private endpoints after any upgrade. Most of the pain I see on Foundry projects is pain that comes from skipping one of those.

Two follow-ups coming next on this blog: Foundry Agent Service migration from the Assistants API (with code from TrafficIQ) and an authentication-patterns deep-dive that goes well past DefaultAzureCredential into workload identity federation, on-behalf-of flows, and the per-environment role assignments I actually deploy. Subscribe if that’s useful — I’ll link them here as they go live.

Image credits

Diagrams in this post are reused from Microsoft Learn with attribution to Microsoft:
- Section 1 — Foundry resource architecture: Microsoft Foundry architecture
- Section 3 — Azure AI User role scope: RBAC for Microsoft Foundry
- Section 3 — Network isolation plan: Configure private link for Foundry
- Section 4 — Agent components: What is Microsoft Foundry Agent Service?
All other commentary, code, and opinions in this post are my own and reflect lessons from building TrafficIQ.
May 20, 2026
Why I built 6 agents instead of 1 mega-agent — lessons from TrafficIQ
I had two design choices for TrafficIQ: one super-agent holding 56 tools, or six specialist agents sharing them. I picked six. Here is what the one-agent path gets right, where it breaks, and the six lessons I took into production.

TrafficIQ went on to win Best Use of Microsoft Foundry at the AI Dev Days Hackathon — chosen from 401 projects and 2,041 registrants. The architecture choices below are what made that possible, and what I would actually defend in front of an enterprise architecture review board.

Why one-agent is genuinely tempting

The one-agent design is the simpler mental model. One assistant. One system prompt. One thread. One place to debug.

When you are sketching the first prototype, this is almost always the right move. Orchestration is not free — you have to write a router, define handoff contracts, manage cross-agent state. Skipping all of that gets you to a working demo in an afternoon. Most enterprise teams default here, and for a 10-tool assistant, they are right to.

The trouble starts later. It starts when the surface area grows past what a single model can hold in its head.

Where one-agent breaks

In my experience tool-selection accuracy degrades non-linearly past around 15 to 20 tools. The model does not fail loudly. It fails subtly. It picks get_shipment_status when the user clearly needed check_shipment_status, because the names overlap and the descriptions rhyme. It calls track_shipment when the right answer was get_proof_of_delivery.

The system prompt becomes the second symptom. To compensate for the confusion, you add disambiguation rules. “Use tool X only when the user mentions Y.” The prompt grows. By the time you have 40 tools, you are nursing a 4,000-token monolith that nobody on the team wants to touch.

And then there is context-window pressure. Every tool’s JSON schema, every parameter description, every example — it all lives in the agent’s context on every turn. With 56 tools, that alone is enough to crowd out the actual conversation.

A super-agent does not just get slower. It gets less correct. The failure mode is “looks plausible, called the wrong tool.”

The architecture I chose

Six specialist agents, each with a tight tool set scoped to its domain. One orchestrator on top. One router inside the orchestrator. GPT-4.1 under each agent. The whole orchestration layer is built on the Microsoft Foundry SDK — the MultiAgentOrchestrator, the specialists, and the RouterAgent are all SDK-native, using the Foundry Assistants pattern (agent, thread, message, run) end to end.

TrafficIQ multi-agent architecture — 6 specialist agents and the orchestrator.

The split is the part most people skip past, so it is worth being concrete:
- Traffic Agent — 17 tools. Routing, journeys, incidents, reroutes, weather, POI, isochrone, snap-to-road.
- Supply Chain Agent — 11 tools. Shipments, deliveries, inventory, ETAs, KPIs, proof of delivery. Backed by D365 F&O via the MCP Server.
- Fleet Agent — 7 tools. Vehicle positions, driver performance, health, maintenance.
- Operations Agent — 7 tools. Work orders, technician availability, schedule optimisation, returns.
- Field Service Agent — 7 tools. Service requests, customer assets, SLAs, dispatch, parts.
- IoT & Logistics Agent — 7 tools. Device health, geofences, driving behaviour, connectivity, batch route alternatives.
Plus 2 shared tools (navigate_to_page, show_input_form) that every agent can call. That is 56 tools total, none of which any single agent actually has to reason over.

Coordination sits in a MultiAgentOrchestrator. It runs a three-tier router: sticky → keyword → LLM classifier (the RouterAgent). Each specialist holds its own Foundry thread so its context stays clean. The orchestrator handles handoff when the user pivots from one domain to another.

Broader TrafficIQ architecture — agents, MCP, Azure services, Dataverse.

The rest of this post is the six lessons that fell out of building it.

Lesson 1 — route in tiers, not in one LLM call

The naive multi-agent router is “ask GPT which agent should handle this.” It works. It is also slow and expensive on every single turn, including the easy ones.

I run three tiers in order. First, sticky: if the user is mid-thread with the Supply Chain Agent and the next message is “and the one after that?”, stay put. Conversations are usually continuous. The default should be continuity, not re-evaluation.

Second, keyword. Each agent registers a small set of high-signal terms — “shipment”, “warehouse”, “geofence”, “technician”. A keyword match is effectively free. For roughly the queries you would expect — the obvious ones — this resolves the routing decision in microseconds with no token spend.

Only when both tiers miss do I fall back to the LLM classifier. That is the RouterAgent, and it is the only model call dedicated to routing. The result is a router that is fast on the common path, accurate on the ambiguous one, and cheap in aggregate. Putting the cheap checks first is the entire trick.

Lesson 2 — each agent owns its own thread

This one took me a while to land on, and I think it is the most underrated decision in the whole architecture.

The obvious approach is to share a single conversation thread across all agents, and have the orchestrator switch which agent reads from it. Do not do this. It is the worst of both worlds. Each agent now sees every tool’s history, including tools it does not own. The tool-set bleed contaminates selection. You also get token bloat: every agent re-reads the entire shared history on every run.

In TrafficIQ each specialist owns its own thread via the Microsoft Foundry SDK. The Supply Chain Agent’s thread only ever contains Supply Chain turns. Its tool schemas, its system prompt, its prior tool calls — none of it touches the Fleet Agent’s context. Each agent is, effectively, a tightly scoped assistant that does not know the others exist. The SDK’s thread primitive is what makes that isolation cheap to enforce.

The orchestrator is the only component that knows there are multiple agents. The agents themselves are blissfully ignorant. That isolation is what makes them stay accurate as the system grows.

Lesson 3 — context handoff is the hard problem, not routing

Once you have isolated threads, the next question is the obvious one: what happens when the user pivots? “What’s the ETA on that shipment?” — Supply Chain handles it. Then: “And dispatch a tech to the warehouse.” — that is Field Service, and Field Service has no idea what “that shipment” refers to.

You cannot dump the entire Supply Chain thread on Field Service. That would re-introduce every problem isolated threads were meant to solve. You also cannot hand over nothing — the user is mid-thought and expects continuity.

What I settled on is a small, deliberate handoff payload: a summary of the last N messages from the source agent, written into the destination agent’s thread as a context message before the user’s new turn lands. Enough grounding to resolve “that shipment”. Not enough to confuse tool selection. The summary is generated by the same Azure OpenAI deployment the agents use, with a tight system prompt — give me entities, IDs, and the last user intent. No prose.

Routing gets the headlines. Handoff is what actually breaks in production if you get it wrong.

Lesson 4 — tools must be MECE within an agent, not across all agents

MECE — mutually exclusive, collectively exhaustive. It is the rule I borrowed from consulting, and it is the cleanest way to think about tool design in a multi-agent system.

Across the whole platform, similar-sounding tools exist. Traffic’s plan_journey and Supply Chain’s optimize_delivery_route both compute routes. That is fine. They live in different agents and serve different intents — a personal commute is not a multi-stop delivery plan. The router decides which world the user is in. The agent never has to choose between them.

The rule that actually matters: within one agent, no two tools should be confusable. The Traffic Agent has 17 tools, and I spent more time on their names and descriptions than on any other part of the system. get_traffic_incidents queries an area. monitor_saved_journey watches a specific route. suggest_reroute triggers a recompute. Different verbs, different objects, no overlap.

If you cannot explain to a junior engineer in one sentence what makes two tools different, the model will not get it right either.

Lesson 5 — make agents observable from day one

You cannot debug a multi-agent system from the response text alone. You need to see which agent answered and which tool fired. So the chat panel in TrafficIQ shows both.

TRAFI chat panel with agent badges and tool-call indicators.

Every message carries an agent badge — colour-coded per domain. Every tool call streams in real time as a small inline indicator: tool name, parameters, status. When something looks off, I can see immediately whether the routing was wrong, the tool selection was wrong, or the tool itself returned bad data. Three different failure modes, three different fixes, and you cannot tell them apart without the visibility.

This is not UI polish. I would argue it is the single most important user-trust feature in the product. Users are sceptical of agents — rightly. When they can see “Supply Chain Agent → check_shipment_status → D365 F&O”, the agent stops being a black box. It becomes a transparent process they can audit.

Build the observability before you build the second agent. You will need it the moment routing decisions start mattering.

Lesson 6 — ground on enterprise data, not the LLM’s memory

Every tool in TrafficIQ resolves against a real system of record. D365 F&O via the MCP Server for shipments, inventory, work orders. Azure Maps for routing, traffic, weather, POI. Azure IoT Hub for device health and telemetry. Dataverse for application state.

The agents never “remember” entities. They look them up. If the user asks about shipment SH-10042, the agent does not summarise what it thinks it knows — it calls check_shipment_status and reads the live record. If GPT-4.1 hallucinates an ETA, the tool result overwrites it.

That single discipline is what separates a hackathon demo from something an enterprise IT team can own. The model is the reasoning surface. The tools are the truth surface. Keep them strictly separated and the agent’s answers become defensible, auditable, and — most importantly — refreshable when the underlying data changes.

What I would do differently next time

Two honest ones.

First, I would build the router evaluation harness before writing the router. I built it last. I now have a CSV of representative queries with the expected target agent, and it runs as a test suite — but I had to retrofit it after the architecture was already set. If I had started with the eval, I would have caught two keyword collisions weeks earlier.

Second, I would put a hard token budget on per-agent system prompts from day one. The Traffic Agent’s prompt drifted from 600 tokens to nearly 1,400 over the course of the build, because every new tool came with “and remember to use this when…” instructions. A budget forces the discipline of writing better tool descriptions instead of patching the prompt. Treat the system prompt like a constitution, not a notepad.

Closing

The headline is small but the implication is large: when a single agent’s tool surface grows past where its selection accuracy holds, the answer is not a smarter prompt. It is a smaller agent.

Six specialists with clear scopes, isolated threads, tiered routing, MECE tools, visible execution, and grounded data — that is the recipe that survived production hardening in TrafficIQ. None of it is exotic. All of it is boring engineering applied carefully.

If you want to see the code, the TrafficIQ repo is on GitHub. The Microsoft winner announcement is here. And the full demo video walks the router, the handoffs, and the tool execution in real time.

TrafficIQ operational dashboard.
May 18, 2026
🥈 1st Runner Up — Microsoft Dynamics 365 Customer Insights ITeS Hackathon
🥈 1st Runner Up — Microsoft Hackathon: Dynamics 365 Customer Insights (ITeS) · Feb–Mar 2021

In early 2021, our ITC Infotech team was selected as the 1st Runner Up at the Microsoft Dynamics 365 Customer Insights ITeS Hackathon — a four-week build challenge judged by an eight-member Microsoft jury. The objective: solve a stated business problem within four weeks, leveraging Microsoft Azure and Dynamics 365 — Customer Insights, Power BI / Power Apps. Solutions were validated against industry coverage, number of data sources, ideation of measures, integration breadth and presentation quality.

The Microsoft / ITC Infotech announcement — 1st Runner Up, Dynamics 365 Customer Insights ITeS Hackathon, Feb–March 2021.

The team
- Pradeep Bhaganna — Sr. Principal Consultant
- Shanthi Chenna Reddy — Technical Architect
- Astha Jaggi — Data Scientist
- Raghav Mishra — Technical Consultant
The business challenge — Financial Crime

The business area we focused on was Financial Crime in banking — a domain under constant regulatory stress. The core problem: banks were drowning in compliance alerts. More than 80% of those alerts turned out to be false positives, and banks did not have an effective single risk view of a customer, forcing large compliance teams to manually triage and investigate cases that mostly went nowhere.

The solution — Intelligent automation on the Microsoft stack

We built an intelligent-automation financial-crime solution on top of ITC Infotech’s CIP Digital Banking Capability, combining a machine-learning model developed in Azure ML Studio with Dynamics 365 Customer Insights to create a single risk view of each customer. The solution then used Dynamics 365 case management to identify true and false positive alerts, automating the alert-triage process.

The stack
- Azure ML Studio — the financial-crime classification model
- Dynamics 365 Customer Insights — unified customer profile and risk view
- Dynamics 365 case management — automated alert triage and investigation workflow
- Power BI / Power Apps — the operations dashboards and compliance UI
- ITC Infotech CIP Digital Banking + E² Framework — the underlying delivery accelerator
The impact (modelled)
- ✅ 30% improvement in case resolution time through automated triage
- ✅ 35% improvement in SAR (Suspicious Activity Report) disclosure rate
- ✅ Significant reduction in manual human effort for compliance investigators
- ✅ Improved customer and colleague experience on the compliance journey
What Microsoft said

Congratulations ITC Infotech team! Great performance and brilliant solutioning. We should immediately think about taking this solution to market via AppSource.
Nitin Santosh — Global Partner Technology Strategist, Microsoft

Congratulations Team ITC Infotech! Learning, ideating and building — and finally winning! Let us build on this success and bring in some early customer wins together!
Srividya Lakshminaraghavan — Director, Partner Technology, Microsoft India

Looking back

This was my first major Microsoft hackathon recognition — and a defining moment in shaping how I think about enterprise AI. The lesson that stuck with me: AI is only as valuable as the business workflow it lives inside. A model that classifies financial-crime alerts is interesting; a model wired into Dynamics 365 case management with a clear human-in-the-loop is shipped value.

Five years and two hackathon wins later, that thesis still drives what I build today — most recently TrafficIQ on the modern Microsoft AI Foundry / Agent Framework stack.

Links
- 🥈 Original hackathon announcement
- 💼 LinkedIn announcement post
May 16, 2026
🏆 Winning Best Use of Microsoft Foundry at AI Dev Days Hackathon — TrafficIQ
🏆 Best Use of Microsoft Foundry — Microsoft AI Dev Days Hackathon · 2026

I am honored to share that TrafficIQ — Supply Chain Transport Intelligence won the Best Use of Microsoft Foundry Project award at Microsoft’s AI Dev Days Hackathon. The hackathon brought together a global community of 2,041 registrants and 401 submitted projects, with winners selected across two Grand Prize categories and four special category awards.

TrafficIQ Dashboard — the operational command centre.

What TrafficIQ does

TrafficIQ is an enterprise-grade multi-agent AI platform built entirely on the Microsoft AI Platform. It brings real-time traffic intelligence into fleet, logistics and supply-chain workflows — so dispatchers, drivers and operations leaders can make smarter routing and delivery decisions before disruptions hit the bottom line.

The Microsoft stack underneath
- Azure AI Foundry — model hosting, agent orchestration and evaluation
- Microsoft Agent Framework — multi-agent coordination and tool calling
- Azure Maps — routing, traffic incidents and geospatial intelligence
- Azure IoT Hub — fleet GPS telemetry and vehicle sensor streams
- Dynamics 365 Finance & Operations — orders, shipments and field service
- MCP (Model Context Protocol) — standardised tool integration across agents
- Dataverse, Power Apps & Power Automate — the human-in-the-loop UI and workflow layer
The TRAFI multi-agent architecture — 6 specialist agents, 49 composable tools.

TRAFI — the multi-agent core

At the heart of TrafficIQ is TRAFI, a multi-agent AI orchestration system with 6 specialist agents and 49 composable tools. The agents proactively monitor traffic incidents, optimise delivery routes, and reduce operational disruptions before they impact supply chains. Each agent owns a clear responsibility — incident monitoring, route planning, ETA recalculation, fleet health, customer notifications, escalation — and they coordinate through the Microsoft Agent Framework.

What the platform delivers
- ✅ Real-time traffic awareness across the fleet
- ✅ Intelligent route optimisation with live re-planning
- ✅ Fleet GPS visibility and IoT telemetry
- ✅ Predictive maintenance insights
- ✅ Automated ETA updates to customers
- ✅ Field service & inventory management
- ✅ Enterprise notifications and operational dashboards
Delivery Planner — AI-assisted scheduling and route optimisation.

Fleet Management — live vehicle health and GPS telemetry.

Operational analytics — KPIs that decision-makers actually read.

The TRAFI chat agent — natural-language ops co-pilot for dispatchers.

Why this project

This project was focused on solving practical enterprise challenges using agentic-AI patterns and Microsoft technologies in a production-oriented architecture. Hackathon code often optimises for the demo. With TrafficIQ I tried to optimise for what would survive a 3-month production hardening cycle: typed contracts between agents, explicit human-in-the-loop checkpoints, and a Dataverse-backed operational model that an enterprise IT team could actually own.

Links
- 🏆 Microsoft official winner announcement
- 💻 TrafficIQ on GitHub
- 💼 LinkedIn announcement post
May 16, 2026