Microsoft Foundry
Enterprise AI Upskilling

# Microsoft
Foundry

One platform for enterprise AI: models, agents, tools, knowledge, and the control plane that governs them.

Jon-Paul Boyd · June 2026
Welcome. Over the next two hours we go end to end on Microsoft Foundry: what it is, the portal, how you deploy and call models, how you build agents, how you ground them with Foundry IQ, and how you govern a whole fleet from the control plane. The product was called Azure AI Foundry until Microsoft dropped the "Azure" at Ignite in November 2025; the portal is still ai.azure.com. Mixed room today, so each section has a plain-language takeaway and a live demo. Let us set expectations for who gets what first.
00 · Orientation

## Something for everyone

DEVELOPERS Builders

- Choose and compare models: catalog, leaderboard, side by side
- Prototype fast in model and agent playgrounds
- Build agents with tools, MCP, memory, human in the loop, SDKs
- Ground on Foundry IQ; evaluate, trace and fine-tune

IT ADMINS Platform owners

- One Foundry resource, many projects: 1:1 and 1:N patterns
- Hub and spoke routing through an APIM gateway
- RBAC roles, regions, quota and tokens-per-minute limits
- Cost overview and tagging; provision with Bicep and Terraform

SECURITY Risk & compliance

- Guardrails: Prompt Shields, content filters, PII blocklists
- Red teaming: AI Red Teaming Agent on PyRIT, attack success rate
- Governance: Azure Policy deny rules on model deployments
- Control plane: Entra Agent ID, Defender, Purview, continuous evals

1 What it is› 2 Portal› 3 Deploy› 4 Inference› 5 Agents› 6 Foundry IQ› 7 Control plane› 8 Wrap

Build for business value outcomes, then govern it at scale.

Quick map of the room and the route. Developers: the portal, models, agents and Foundry IQ acts are yours. IT admins: deployment pattern and provisioning. Security: guardrails, red teaming and the control plane. The arc along the bottom is the running order. We will not go deep on every line of code; the point is the shape of the platform and where each of you plugs in.
01 · What it is

## One platform

◆ Microsoft Foundry

#### Observability

Tracing · Monitoring Evaluation · Experimentation

#### Frameworks & SDKs

Foundry SDK Agent Framework LangGraph LangChain LlamaIndex CrewAI GitHub Copilot SDK

#### Foundry Agent Service the runtime

Managed orchestration Conversations + state Network isolation OBO auth

#### Models

Microsoft & partners Fine-tuned Bring your own

#### Knowledge & Tools

Azure AI Search Bing SharePoint Fabric Functions Logic Apps OpenAPI MCP

#### Agent integrations

Foundry SDK REST Responses API A2A protocol

#### Guardrails, security & governance

Prompt Shields Content filters PII detection Custom blocklists Protected material Policy & compliance Cost management Entra · Defender · Purview
BUILD Multi-agent · 1,400+ tools · Agent memory · Foundry IQ
OPERATE Observability · Agent 365 identity · Fleet ops

Unified breadth: build with agents, tools, memory and knowledge, then operate them with identity, observability and fleet governance, all behind one resource.

This is the whole platform on one slide, and it is a faithful redraw of Microsoft's own architecture diagram with two corrections baked in. Foundry unifies three jobs, running AI in production, building models, and building apps, under a single Azure resource-provider namespace, so RBAC, networking and policy are shared rather than stitched together. Reading the stack: observability and your choice of framework on top; the Foundry Agent Service runtime in the middle; models, knowledge and tools, and integration protocols beneath; guardrails, security and governance as the base. Two corrections from the original Microsoft slide: the integration surface is the Responses API now, not the Assistants API, which is retiring, and Semantic Kernel plus AutoGen have merged into the Microsoft Agent Framework. The ribbon under the diagram is the why-it-matters: the headline capabilities split into build (multi-agent orchestration, a catalogue of over fourteen hundred tools and agent memory, both in preview, and Foundry IQ for grounded knowledge) and operate (real-time observability, an Agent 365 Entra identity per agent, and fleet governance in the Operate hub). The deck follows that shape: build first, then govern. We unpack each layer over the next two hours.
02 · Portal

## The portal is organised around jobs to be done

◆ Microsoft Foundry / proj-foundry-core Search with AI (Ctrl + K) New Foundry Home Discover Build Operate Docs ☼ ✉ ☉
Welcome back
Start building ›
Project endpoint proj-foundry-core.services.ai.azure.com Region Sweden Central API key auth disabled
Home Land, resume recent work, quick start.
Discover Netflix-style catalog of models and tools.
Build Create agents, apps and workflows.
Operate Admin and fleet view across projects.
Docs Documentation, in context, never leave.

Flip on New Foundry at ai.azure.com and you get five tabs by job. Everything sits inside a Foundry resource (a subscription + resource group: the billing and governance boundary).

Sign in at ai.azure.com and toggle on New Foundry. The portal is deliberately organised around jobs to be done, not Azure resource types. Home is your landing and recent work. Discover is the catalog. Build is where developers create agents, apps and workflows. Operate is the admin and fleet view across subscriptions and projects, that is the control plane we deep-dive later. Docs is integrated so you stay in context. Top left is the current project, click to switch; the search bar is backed by Ask AI, which is itself several agents. The one structural thing to remember: a project always lives inside a Foundry resource, provisioned into a subscription and resource group, which is the billing, management and governance boundary. Note the new portal only shows assets created in the new experience, not classic ones.
03 · Discover

## A model catalog you can compare, then deploy

◆ Microsoft Foundry / proj-foundry-core Search with AI (Ctrl + K) New Foundry Home Discover Build Operate Docs ☼ ✉ ☉
Discover what's possible 1,900+ models · explore by provider, collection, leaderboard
Azure OpenAI Anthropic Microsoft Meta Mistral AI xAI DeepSeek

Model leaderboard | Quality | Safety | Throughput | Est. cost
--- | --- | --- | --- | ---
gpt-5.2-codex ★ | 0.93 | 0.18% | 32 t/s | $4.81
gpt-5.2 | 0.93 | 1.87% | 60 t/s | $4.81
claude-opus-4-6 | 0.93 | 2.41% | 43 t/s | $10.00
claude-sonnet-4-6 | 0.92 | 2.19% | 61 t/s | $6.00
grok-4 | 0.91 | 1.41% | 48 t/s | $5.20

Tools view: Foundry Tools catalog: remote & local MCP servers, OpenAPI and A2A. Configure once, add to any agent or workflow.

Optimise the choice across quality, safety, throughput and cost, compare models side by side, then quick-deploy (global standard) straight from the card.

Discover is a Netflix-style catalog of models and tools, over nineteen hundred models. You browse by provider and by collection (Direct from Azure, Foundry Labs, Foundry Local), open a model card, and crucially you can rank the leaderboard by quality, safety, throughput or cost, then multi-select and compare models side by side before you commit. From a card you quick-deploy at global standard with one click. The Tools view is the other half: the Foundry Tools catalog, where you discover remote and local MCP servers, OpenAPI tools and A2A endpoints, configure them once, and attach them to any agent. The leaderboard numbers shown are a snapshot from the portal. We will deploy one of these for real in the demo.
Live demo

## Compare and deploy from the catalog

Discover → leaderboard → compare → quick-deploy, then the Foundry Tools / MCP catalog.

in the portal · ai.azure.com → Discover
Live in the portal: open Discover, sort the leaderboard by quality then by cost to show the trade-off, multi-select two models and hit Compare to see them side by side, then open a card and quick-deploy at global standard. Finish in the Tools view filtering the MCP catalog by publisher and supported authentication. This sets up the deployment pattern and inference acts that follow.
04 · Build

## Where developers create and manage every asset

◆ Microsoft Foundry / project-admin-c2676f Search with AI (Ctrl + K) New Foundry Home Discover Build Operate Docs ☼ ✉ ☉
New Foundry supports V2 agents only. Classic agents and the Assistants API are not carried over: save them as new agents.
Browse templates Code agent Create agent

Name | Version | Type | Description
--- | --- | --- | ---
contoso-bank-agent | 1 | prompt | Customer-service agent
contoso-pmo-agent | 1 | prompt | PMO knowledge-base agent
aria-rm-briefing-agent | 1 | prompt | Private-banking RM briefing

Build is where developers manage every asset they deploy or create, and it is the home of the model and agent playgrounds.

Build is where developers create and manage every asset. Everything you deploy or create shows up here: deployed models, agents, workflows, fine-tuned models, tools, knowledge bases, evaluations and guardrails, all the building blocks of an agent in one place. It is also where the playgrounds live and where the agent builder runs. One important note banner: the new Foundry experience supports V2 agents only; classic agents and the legacy Assistants API are not carried over, you save them as new agents. The agents listed here are the ones we use later in the deck: a bank customer-service agent, a PMO knowledge agent and a private-banking briefing agent.
05 · Operate

## And one tab to govern it all

◆ Microsoft Foundry / fleet Search with AI (Ctrl + K) New Foundry Home Discover Build Operate Docs ☼ ✉ ☉
PILLAR 1 Controls Runtime guardrails on inputs, outputs and tool calls.
PILLAR 2 Observability Tracing, continuous evaluation, cost.
PILLAR 3 Security Entra Agent ID, Defender, Purview.
PILLAR 4 Fleet ops One view across every project and cloud.

Operate is the Foundry Control to see, govern and act on every agent across the fleet.

The fifth tab, Operate, is the Foundry Control Plane, currently in public preview. This is where you stop thinking about one agent and start thinking about a fleet: controls, observability, security and fleet-wide operations, the four pillars on screen. We are only teasing it here so the shape is in your head; act seven is a full deep dive with live demos for each pillar, including red teaming. For now, just notice that build and operate live in the same portal, so a developer can jump from an alert straight into fixing the agent.
06 · Pattern

## One resource, many projects, two ways to slice it

Subscription› Resource group› Foundry resource› Project(s)
1 : 1 · dedicated one account, one team: a hard infrastructure boundary
◆ aif-spoke-alpha
project-alpha
For strict compliance or cost isolation, and to separate dev / test / prod.
1 : N · shared one account, many isolated project workspaces
◆ aif-spoke-multi
project-beta project-delta project-gamma iq obs cu
For teams on the same cost centre: cost-efficient, shared RBAC, per-project isolation.

The Foundry resource is the billing, networking and quota boundary.
The project is the isolated workspace (its own agents, data and connections).

The new resource model is two levels: a single Foundry resource, the account, contains one or more projects. The resource is the boundary for billing, networking, Azure Policy and model-deployment quota; the project is the isolated workspace, with its own agents, data and connections. So the first design decision is how you map teams to resources. One to one gives each team its own account, a hard infrastructure boundary, which you want for strict compliance or cost isolation and for separating dev, test and prod. One to many puts several teams as separate projects under one shared account, which is cost-efficient and the right default for teams on the same cost centre. The payoff, which the whole repo demonstrates, is that you absorb new capabilities, Foundry IQ, observability, content understanding, by adding projects to the shared account rather than spinning up new accounts.
07 · Architecture

## Centralise inference behind one AI gateway

Models live only in the 'hub' Foundry accounts*
◆ aif-core
East US 2 · general
gpt-4.1-mini embeddings
◆ aif-research
Norway East · reasoning
o3-deep-research
◆ aif-oss
West US 3 · open-weights
Phi-4
↑ managed identity · routed by URL (most specific wins)
⇉ APIM gateway · apim-foundry
swaps client key → managed-identity token · per-team rate limits & quota · routes by URL to hubs
↑ each project: its own connection core-{team} + gateway key
Spoke 1 : 1 no models · deny policy
◆ aif-spoke-alpha
project-alpha
reaches models via the core-alpha connection
Spoke 1 : N no models · deny policy
◆ aif-spoke-multi
beta delta gamma iq obs cu
each project: own key, own quota
* 'Hub' here means the central model-hosting Foundry accounts in this topology, not the legacy Foundry v1 'hub' resource type.

One Azure API Management gateway fronts every model. Spokes hold zero deployments: they call the hub through a per-team key, so cost, content filters and observability stay unified.

This is the reference architecture for running Foundry at scale: a centralised AI gateway. One Azure API Management instance is the single entry point for all model traffic, and it routes by URL to the right regional hub account, general-purpose in East US 2, a reasoning model in Norway East, open-weights in West US 3. The crucial rule: model deployments live only in the hubs. Every spoke, whether a one-to-one dedicated account or the shared one-to-many account, holds zero local deployments and reaches models only through the gateway, using its own APIM subscription key for independent rate limiting and usage tracking. The gateway also swaps the client key for a managed-identity token, so the hub accounts have key auth disabled and only the gateway can call them. Net effect: one place for cost, content filtering, quota and observability, instead of fragmentation across teams.
08 · Governance

## Make the pattern impossible to break

    // Azure Policy: deny-model-deployments
    {
      "if": {
        "field": "type",
        "equals": "Microsoft.CognitiveServices/accounts/deployments"
      },
      "then": { "effect": "deny" }
    }

rg-foundry-core EXEMPT
rg-foundry-spoke-alpha DENIED
rg-foundry-multi DENIED
Deploy a model into a spoke and Azure returns
RequestDisallowedByPolicy. Spokes can still use models through the gateway, just never deploy their own.

A one-rule Azure Policy denies Microsoft.CognitiveServices/accounts/deployments in spoke resource groups. The hub stays exempt, so the architecture cannot drift.

Architecture diagrams do not enforce themselves, so this is the governance that does. A custom Azure Policy, deny-model-deployments, blocks the creation of any deployment resource in the spoke resource groups; the hub resource group is exempt because that is where the platform team deploys the shared models. The rule is genuinely this small: if the resource type is a Cognitive Services account deployment, deny. Try to deploy a model into a spoke and you get RequestDisallowedByPolicy. Spokes can still use the models through the gateway, they just cannot deploy their own, which is exactly what keeps cost, content filters and observability from fragmenting. This is the kind of guardrail an IT or platform team sets once at the management-group or subscription level.
09 · Deploy

## Deploying a model: pick a type, get an endpoint

Standard pay per token

- Elastic, no commitment, billed per token
- The default: global standard, one click
- Best for spiky or early workloads

Provisioned / PTU reserved capacity

- Predictable latency and throughput
- One PTU pool can be shared across different provisioned models
- Best for consistently high utilisation, latency-sensitive production

Global
max throughput, data may leave region
Data Zone
stays within a geography (EU / US)
Regional
pinned to one region for residency
Defaults vs custom: accept global standard + default quota, or customise SKU, quota (TPM) and guardrails. Partner models (Llama, Claude) need an Azure Marketplace subscription; models sold directly by Azure do not.

A deployment is a model + a deployment type. The type sets the cost model (per-token vs reserved) and the data-residency and throughput guarantees; quota is tracked per region and subscription, as PayGo (standard) or PTU (provisioned).

Deploying a model is choosing the model plus a deployment type, and the type is the decision that matters. Standard is pay-per-token and elastic, the default being global standard with one click, ideal for spiky or early workloads. Provisioned, or PTU, is reserved capacity for predictable latency and throughput, and PTU is fungible, you can move that capacity across provisioned deployments. Orthogonal to that is scope: global maximises throughput but data may leave the region; data-zone keeps it within a geography like the EU; regional pins it to one region for residency. You can take defaults or customise the SKU, quota and guardrails. Two gotchas for admins: quota is tracked as tokens per minute per region and subscription, and partner models such as Llama or Claude require an Azure Marketplace subscription, whereas models sold directly by Azure do not.
Live demo

## Model Deployment Playground

Set the system prompt, attach web search → Monitor for tokens, cost and latency.

in the portal · ai.azure.com → Build → Deployments → Playground
Every deployment has the same three working tabs. Playground is where you try it: set the system prompt, adjust temperature and max tokens, and attach tools to extend the model, web search for grounding in live data, a code interpreter for things like charting an Excel file, or SharePoint retrieval. Details is the integration tab: the target URI, the API key if you have enabled key auth, provisioning state, and the TPM and RPM rate limits. Monitor is the operations tab: total requests, input and output token counts, estimated cost, and time to first byte over a time window, and you can ask AI canned questions over those metrics. So one screen takes you from trying a model to running it in production.
10 · Endpoints

## One resource, three endpoint surfaces

OpenAI SDK
*.openai.azure.com/openai/v1
Full OpenAI API surface: chat completions, embeddings, Responses, fine-tuning. No agents or evaluations.
key or token
Foundry SDK
*.services.ai.azure.com/api/projects/*
Foundry-native: agents, evaluations, connections, tracing. Responses API on its /openai route.
token only · Entra
Foundry Tools SDKs
*.cognitiveservices.azure.com
The other AI services: Speech, Vision, Language, Content Safety (formerly Azure AI Services).
key or token
One Foundry resource exposes all three · an Azure OpenAI resource has only /openai/v1

A Foundry resource is multi-surface: the OpenAI endpoint for raw inference, the project endpoint for Foundry-native agents and evals (token-only via Entra), and Cognitive Services for the other AI tools. Match the SDK to the endpoint.

A quick reference that resolves a common point of confusion. A single Foundry resource exposes three endpoint surfaces, and you pick the SDK to match. The OpenAI host, resource.openai.azure.com/openai/v1, is the pure OpenAI-compatible surface: chat completions, embeddings, Responses and fine-tuning, with maximum client compatibility, but no Foundry-native features. The project host, resource.services.ai.azure.com/api/projects, is the Foundry SDK surface for agents, evaluations, connections and tracing, and it requires token auth via Entra, no keys. The Cognitive Services host, resource.cognitiveservices.azure.com, is the older multi-service surface, now called Foundry Tools, for Speech, Vision, Language and Content Safety. The key point: an Azure OpenAI resource only gives you the openai endpoint, whereas a Foundry resource gives you all three. Source: the Foundry API and SDKs reference, 04-03.
11 · Inference

## Calling models through the gateway

Direct client gateway key
AzureOpenAI(base_url=gateway, api_key=key)
Chat completions Embeddings Deep research
Full Azure OpenAI surface; the team holds the gateway key.
Foundry project client keyless
responses.create(model="core-alpha/gpt-4.1-mini")
Responses API Multi-turn Streaming
Keyless via Entra; routes through the project's core-{team} connection, which speaks the Responses API only.
↓ both paths reach the models through one APIM gateway · managed identity onward ↓

Pick by surface and auth: the direct client (gateway key) for the full OpenAI surface, or the Foundry project client (keyless via Entra) whose core-{team} connection speaks the Responses API. Both reach the same models through the one gateway.

Once models are deployed centrally, teams call them two ways, both through the one APIM gateway. The direct Azure OpenAI client points at the gateway with a subscription key and gives you the classic surfaces: chat completions, embeddings and the deep-research model. The Foundry project client uses DefaultAzureCredential, so no keys, and because the project's gateway connection is an API Management connection it speaks the Responses API specifically: the path for agents, server-side multi-turn via previous_response_id, and a simpler input parameter. Both are keyless to the models themselves; key-based access to the hub accounts is disabled by policy. Multi-turn keeps state server-side, so you pass previous_response_id rather than resending history, though every prior input token is billed on each call. Streaming reads output_text.delta events for incremental tokens.
Live demo

## Model inference, end to end

The direct AzureOpenAI client and the Foundry AIProjectClient → chat, embeddings, deep research → the Responses API, multi-turn and streaming.

The live run of the inference notebook, 07-01. It sets up the two clients - the direct Azure OpenAI client on the gateway key and the keyless Foundry project client - then exercises every surface in turn: chat completions, embeddings, the deep-research model, the Responses API, a multi-turn follow-up via previous_response_id, and token streaming. It also creates a short-lived agent in each team project to show the connection/model routing. The throughline: one gateway, many surfaces, no keys.
12 · Agents

## An agent is a model, instructions and tools

Input
User messages
System events
Agent messages
→
Agent
LLM
Instructions
Tools
→
Output
Agent messages
Structured output
↓ tool call ↑ result
Tool calls
Retrieval
Actions
Memory

It takes unstructured input, reasons with the model under your instructions, calls tools mid-flight to retrieve or act, and returns a message or structured output.

Strip away the hype and an agent is three things: a model for reasoning and language, instructions that set its goals and constraints, and tools that let it retrieve knowledge or take action. It receives unstructured inputs, user messages, system events, even messages from other agents, and produces outputs, either a message or structured output. The important bit is the loop at the bottom: mid-flight the agent calls tools, retrieval to fetch knowledge, actions to do something in the world, memory to recall, then folds the result back into its reasoning. This three-part model is the mental picture for everything in this act.
13 · Lifecycle

## Foundry is an assembly line for agents

Six stages, secure and testable end to end: models → customization → knowledge and tools → orchestration → observability → trust.

Microsoft frames Foundry as an assembly line for intelligent agents, six stages that take you from idea to production-ready. One, models, the raw intelligence. Two, customizability, fine-tuning, domain prompts, your own data. Three, knowledge and tools, enterprise knowledge like Bing, SharePoint and Azure AI Search plus actions like Logic Apps and Functions. Four, orchestration, the workflow layer that manages tool calls, conversation state, retries and logging, that is the magenta node because it is the part that ties everything together. Five, observability, logs, traces and evaluations. Six, trust, identity, RBAC, content filters, encryption and network isolation. The deck has already touched models and tools; this act is mostly stages three and four, and the control plane act is stages five and six.
14 · Runtime

## The Agent Service runs the loop, not your code

your code · client.responses.create(model, input, agent_reference)
↓
Foundry Agent Service · on the Responses API
1 Load the agent version: system prompt, tools, model binding
2 Persist the conversation: resume via previous_response_id
3 Call the model through the gateway, RBAC-scoped, no keys
4 Dispatch tools: Code Interpreter, File Search, MCP server-side
5 Stream and trace: output deltas plus OpenTelemetry spans
6 Content safety on input and output
↓
grounded result + citations · FunctionTool calls handed back to you for human-in-the-loop

For a prompt agent there is nothing to deploy: you submit a request and the service owns thread state, tool dispatch, retries and content safety server-side.

A common misconception is that you run the agent loop. You do not. The Foundry Agent Service is the runtime. You submit a request via the Responses API against an agent reference, and the service does the rest: loads the agent version with its prompt, tools and model binding; persists the conversation so you resume with previous_response_id rather than resending history; calls the bound model through the project's gateway, RBAC-scoped with no admin keys; dispatches tools, where Code Interpreter, File Search and MCP run server-side while your FunctionTool calls are handed back to your code, which is the basis of human-in-the-loop; streams output deltas while emitting OpenTelemetry spans; and applies content safety on input and output. For a prompt agent there is no container, no pod, nothing to deploy. Which raises the obvious question: what if you DO want to bring your own runtime?
15 · Knowledge & tools

## What an agent can know and do

#### Knowledge (grounding)

Azure AI Search
File search
Web / Bing grounding
Foundry IQ knowledge base
Work IQ (Microsoft 365)
Microsoft Fabric (Fabric IQ)
SharePoint

#### Tools & actions

Code Interpreter
Function calling
Azure Functions
Logic Apps connectors
Image generation
Browser / computer use

#### Custom & protocols

Model Context Protocol
OpenAPI tools
Agent-to-Agent (A2A)
Toolbox (curated set of tools)
Tool catalog
Skills (reusable instructions)
Agent memory

Grounding sources and actions plug in by configuration, not custom plumbing. MCP and OpenAPI let an agent call almost any tool server or API, and the IQ family (Foundry, Work, Fabric) grounds it in your data.

What an agent can know and do, configured rather than coded. Knowledge for grounding: Azure AI Search, uploaded files, the web via Bing, SharePoint, and your enterprise data through the IQ family - Foundry IQ over your Azure and web sources, Work IQ over live Microsoft 365 data like email, chat, meetings and files through Microsoft Graph with permissions enforced, and Fabric IQ over your Fabric lakehouse and semantic models. Tools and actions: a code interpreter, your own functions, Azure Functions and Logic Apps connectors, image generation, and browser or computer use. Extensibility: the Model Context Protocol and OpenAPI let an agent call almost any tool server or external API, agent-to-agent handles multi-agent calls, the Toolbox and tool catalog package and govern tools, skills add reusable behavioral guidance as versioned files, and agents have built-in memory. The headline: knowledge and tools are configuration, not custom plumbing. These capabilities move between preview and GA quickly, so the status labels are intentionally left off - check the docs for the current state.
16 · Hosted

## When you want to bring your own runtime

Prompt agent declarative, runs on the Agent Service

- You define model + instructions + tools
- Nothing to deploy, no container
- The service owns the loop

Hosted agent bring your own code and runtime

- Deploy from source (Foundry builds it, no Docker), or ship your own container
- Foundry provisions compute and a dedicated endpoint
- Per-session VM sandbox, scale to zero, OpenTelemetry
- Gets its own Microsoft Entra agent identity

Bring your framework Microsoft Agent Framework LangGraph OpenAI Agents SDK Anthropic Agent SDK GitHub Copilot SDK your own code

Most agents need no container. For your own runtime, hosted agents run any framework with a managed identity and a sandbox: deploy straight from source (no Docker), or bring your own container.

Prompt agents cover most cases with nothing to deploy. But sometimes you want your own runtime, your own agent loop, your own framework, and that is hosted agents. You ship a container, or just zip your Python or C-sharp source and let Foundry build it with no Dockerfile; Foundry then provisions compute, assigns the agent a dedicated Microsoft Entra identity and endpoint, and runs it in a per-session VM-isolated sandbox with scale-to-zero and built-in OpenTelemetry. Crucially the framework is your choice: Microsoft Agent Framework, LangGraph, the OpenAI Agents SDK, the Anthropic Agent SDK, the GitHub Copilot SDK, or your own code. The next slide shows exactly what that looks like with the Copilot SDK.
17 · Hosted SDK

## A whole agent harness inside one container

Foundry hosted runtime · your container
main.py + InvocationAgentServerHost · outer loop
POST /invocations→one user turn→stream events out as SSE
spawned Copilot CLI subprocess · inner agentic loop
model call→tool?→run shell / python→feed result back↻idle
inference → ◆ your Foundry gpt-5.x via managed identity · /openai/v1/responses · no secrets in the container

main.py is a thin invocations shell; the real reason-act-observe loop runs in a spawned CLI subprocess. Inference is bring-your-own-key to your own Foundry model, so no secrets ship in the image.

Here is the shape of a serious hosted agent: the GitHub Copilot SDK, the engine behind Copilot CLI, running inside a Foundry container. There are two nested loops. The outer loop is main.py with the invocations server host: Foundry calls POST /invocations once per user turn, and main.py just bridges events out as a stream. The inner loop, the actual agentic reason-act-observe cycle, runs inside a Copilot CLI subprocess that the SDK spawns: model call, decide if a tool is needed, run shell or Python, feed the result back, repeat until idle. So main.py is a thin protocol shell, not the brains. And inference is bring-your-own-key: the SDK points at your own Foundry model over the Responses API using the container's managed identity, so no GitHub or OpenAI keys ever ship in the image. Sovereign, observable, and the same demo you would run locally.
Live demo

## Build and run an agent

Build a prompt agent in the playground.

portal Build → playground
Live, in two parts. First, build a prompt agent in the agent builder: set its instructions, attach the tools and knowledge it needs, and try it in the playground, watching the tool calls and traces. Second, invoke the example hosted Copilot SDK agent and watch it run tools inside its own per-session sandbox. The use case is generic on purpose; for a given audience we prepare a relevant scenario, instructions and tools beforehand.
18 · Foundry IQ

## Grounded knowledge, as a managed layer

Knowledge sources
Azure Blob · OneLake
SharePoint
Existing search indexes
Web (Grounding with Bing)
→
◆ Knowledge base
One endpoint, shareable across many agents. Permission-aware: honours ACLs and Purview sensitivity labels under the caller's identity.
engine: Azure AI Search agentic retrieval
→
Agents attach via MCP
prompt agent
multi-agent system
Foundry IQ · enterprise knowledge Fabric IQ · analytics Work IQ · M365

Foundry IQ is a managed knowledge layer over Azure AI Search. You build a knowledge base from your sources once and any agent grounds on it, with citations and permissions enforced.

Foundry IQ, announced at Ignite, is Microsoft's managed knowledge layer for agents, and underneath it is Azure AI Search's agentic retrieval. The core object is a knowledge base: a single, shareable endpoint composed of one or more knowledge sources, indexed ones like Blob, OneLake, SharePoint and existing search indexes, and remote ones like web via Grounding with Bing. Agents attach it as a tool over MCP. Two things to stress for this room: it is permission-aware, it enforces ACLs and honours Purview sensitivity labels under the caller's Entra identity, so retrieval respects who is asking; and Microsoft's own benchmark puts agentic retrieval about thirty-six percent above single-shot RAG on response quality. Foundry IQ is the enterprise-knowledge sibling of Fabric IQ for analytics and Work IQ for Microsoft 365. Next, how the LLM actually lives inside the retrieval path.
19 · Retrieval

## The model lives inside the search, not just after it

Build the index
documents3k arxiv-nlp abstracts → embedtext-embedding-3-large · 3072d → indexHNSW vector + semantic + ACL trim
◆ Knowledge source registers the arxiv-nlp index
kb-fast
minimal effort · no LLM
direct index-driven retrieval
fastest, no model quota
kb (standard)
low effort · one LLM pass
gpt-4.1-mini (APIM) plans sub-queries
better relevance, multi-turn
Both return raw chunks (EXTRACTIVE_DATA) for the agent's own LLM; each KB auto-exposes an MCP endpoint. Effort: minimal → low → medium → high.

Two knowledge bases over one index: kb-fast (minimal effort, no LLM) for speed, kb (standard) (low effort, gpt-4.1-mini) for relevance. The effort knob puts the model inside the search, trading depth for latency and cost.

This is the concrete Foundry IQ setup the demo builds. First the index, notebooks 10-01 and 10-02: three thousand arxiv NLP abstracts are embedded by text-embedding-3-large at 3072 dimensions through the gateway and written into an Azure AI Search index with HNSW vectors, a semantic configuration and a group-ids field for ACL security trimming. Then in 10-03 the index is registered as a knowledge source and exposed as two knowledge bases that differ on one knob, retrieval reasoning effort. kb-fast uses minimal effort: no LLM at all, direct index-driven retrieval, fastest and no model quota. kb standard uses low effort: one LLM planning pass with gpt-4.1-mini through the gateway that decomposes the question into sub-queries, for better relevance on complex or multi-turn questions. Both return raw chunks, extractive data, for the agent's own model to reason over, rather than the knowledge base synthesizing the answer itself. Each knowledge base auto-exposes an MCP endpoint the agent connects to. The effort knob runs minimal, low, medium, high: that is the model living inside the search.
Live demo

## Build a knowledge base, then retrieve

Index a corpus → build kb-fast and kb-standard → retrieve and compare the two reasoning efforts.

Live: build the full Foundry IQ stack. Deploy search and index the corpus, create the two knowledge bases, kb-fast at minimal effort with no LLM and kb-standard at low effort with gpt-4.1-mini, then run retrieval against each and watch how the reasoning-effort knob changes the results, before wiring a knowledge base to an agent. Same corpus, two latency-and-relevance trade-offs.
20 · Control plane

## One agent is easy. A fleet is the hard part.

Data plane
Agents in action
Chatting, calling tools, retrieving data, generating responses. The work running.
Control plane
See, govern, act
One surface for identity, policies, security, observability and cost, across every project and cloud.
Risk: Prompt injection
Untrusted content in a tool result hijacks the agent's instructions.
Risk: Task drift
The agent quietly does something other than what it was asked.
Risk: Data leakage
Access plus a confused instruction plus an outbound channel equals exfiltration.

Agents add failure modes apps never had, and they compound as you add tools and data. Their underlying intelligence is a probabilistic, non-deterministic LLM, not deterministic code, so the same input can behave differently each time.

Here is the pivot of the whole talk: from building one agent to running a fleet. In cloud terms, the data plane is the work running, agents chatting, calling tools, retrieving data. The control plane is the management surface above it: see, govern, act, one place for identity, policy, security, observability and cost, across every project and every cloud. Why you need it: agents add failure modes traditional apps never had. Prompt injection, where untrusted content in a tool result hijacks the instructions. Task drift, where the agent quietly does the wrong thing. Sensitive-data leakage, where access plus a confused instruction plus an outbound channel equals exfiltration. And these compound as you add tools, data and multi-agent workflows. The canonical story: a company had about ten agents in production, one retrieved a document carrying a hidden instruction, and it wove customer names and transactions into its answers for three days before anyone noticed. No single team owns this, which is why it needs a dedicated surface.
21 · The pillars

## What the control plane brings together

CONTROLS Guardrails Prompt Shields, content filters and blocklists, validated by red teaming.
OBSERVABILITY See inside Tracing, continuous evaluation on live traffic, per-agent cost.
SECURITY Identity & data Entra Agent ID, Microsoft Defender, Microsoft Purview.
FLEET OPS At scale One inventory and to-do list across projects, frameworks and clouds.

The control plane brings four essentials into one surface: runtime guardrails, observability, agent security and fleet operations, across every project, framework and cloud.

The control plane brings together four essentials. Controls: runtime guardrails not just on prompts and responses but on tool calls and tool responses too. Observability: tracing, continuous evaluation on live traffic, and per-agent cost. Security: a durable Entra identity per agent, plus Defender and Purview. And fleet-wide operations: one inventory and to-do list across projects, frameworks and clouds. We will walk each pillar with a demo.
22 · Controls

## Guardrails on inputs, outputs and tool traffic

User input
↓
INPUT Prompt Shields (jailbreak + indirect) · content filters · custom blocklists
↓
Agent + model
↓
TOOL CALL / RESPONSE task adherence on the call · indirect injection on the response
↓
OUTPUT content filters · protected material · groundedness · sensitive data / PII · custom blocklists
↓
Response

A guardrail is a set of controls at four points: input, tool call, tool response, and output. The tool-call and tool-response checks are the agent-specific part, catching indirect prompt injection before the agent acts.

Controls, the first pillar. A guardrail is a named set of controls; each control is a risk, an intervention point and an action. Guardrails fire at four points in a turn, in order. On the user input: Prompt Shields for jailbreak and indirect attacks, content filters, and custom blocklists. At the tool call, before the tool runs: task adherence, checking the planned call and its arguments match the user's intent. At the tool response, when data comes back: Prompt Shields for indirect, cross-prompt injection, the genuinely new defence, malicious instructions hidden in a document, web page or tool result, caught before the agent acts on them. And on the output: content filters, protected material, groundedness against the retrieved context, and sensitive data or PII detection. Note that groundedness is currently model-level and not yet enabled for agents. In the demo we paste an injection and a fake SSN into the bank agent; the input shield blocks the injection before the model ever sees it, at zero token cost.
Live demo

## Trip the guardrails

Prompt Shields block a jailbreak.

Live: a fictional bank customer-service agent with one RAI policy attached to its model deployment. Send a clean banking question first to set the baseline, then three attacks. A prompt injection, "ignore all previous instructions", is caught by Prompt Shields before the model ever sees it. A message containing a fake SSN is caught by the PII blocklist. A question naming a competitor or an internal codename is caught by the custom blocklist. Each block is input-side, so it costs zero tokens, and the agent's own system prompt has no defensive language: the guardrails are the perimeter.
23 · Observability

## Trace it, evaluate it, cost it

Tracing OpenTelemetry traces, prompt to model to tool, into Application Insights.

- Walk back any run, step by step
- Auto-traced for Agent Framework, LangChain, LangGraph

Continuous evaluation Score live production traffic, not just a pre-ship test suite.

- Set a threshold, alert when quality drops
- Groundedness, task adherence, tool-call accuracy

Cost Per-agent token spend across the fleet, near real time.

- Agents burn tokens fast: watch them
- Sort the fleet by cost or error rate

You cannot human-review every step, so you evaluate live traffic against a threshold and trace what crosses it, while watching cost per agent.

Observability, the second pillar. You cannot put a human on every agent step, that defeats the purpose, so the model is: trace everything, evaluate continuously, and watch cost. Tracing is OpenTelemetry, prompt to model inference to the specific tool call, landing in your Application Insights, and it is automatic for the Microsoft Agent Framework, LangChain and LangGraph. Continuous evaluation runs against real production traffic, not just a pre-ship suite: set a threshold and get alerted when groundedness or task adherence drops, with agent-specific evaluators like tool-call accuracy. And cost: per-agent token spend across the fleet, because agents burn tokens fast. In the demo we open a code-interpreter trace and an evaluation run in the portal.
Live demo

## Show an eval run

Live: take one agent run and open its trace in Application Insights. Walk the span tree top to bottom, the prompt, the model call with its tokens and latency, then each tool call and what it returned, so the whole decision path is visible after the fact. Then open an evaluation run: continuous evaluation samples production traffic and scores it on groundedness, task adherence and tool-call accuracy against a threshold, the signal that fires an alert when quality drops. One run traced, live traffic scored.
24 · Security

## Every agent gets a real identity

Entra Agent ID Publish an agent and it gets a Microsoft Entra identity, automatically.

- Access control like any principal
- Ownership: know who to call at 2am
- Lineage across its lifecycle

Microsoft Defender AI security posture and threat detection extended to agents.

- Attack-path analysis, gap recommendations
- A jailbreak surfaces as a security alert

Microsoft Purview Agent interactions available for audit and compliance.

- Org-wide content-safety policies
- Sensitivity labels honoured

Controls = each developer configures
Opt in or out on your own agent, e.g. Prompt Shields, content filters, a PII blocklist.
Policies = the organisation mandates
"Every agent must have indirect-injection protection on", scanned continuously.

Publishing an agent issues an Entra identity (app + object id) for access, ownership and lineage.
Policies turn a developer's optional control into an org-wide mandate.

Security, the third pillar, and the headline is identity. The moment you publish an agent it gets a Microsoft Entra identity, an application id and an object id, automatically, with no extra steps. That buys three things: access control, you govern it like any user or service principal; ownership, it is tied to a named team so when it breaks at 2am you know who to call; and lineage, the identity follows the agent through its lifecycle. Defender extends to agents with posture management, attack-path analysis and threat detection, so a jailbreak shows up as a security alert with full context. Purview makes agent interactions auditable and honours sensitivity labels. And the key distinction: controls are what each developer configures on their own agent, opt in or out, whereas policies are an organisation-level mandate, every agent in this subscription must have indirect-injection protection on, scanned continuously across every agent in scope.
Live demo

## Mandate a control across the fleet

Create a policy in Operate / Compliance → mandate a control across a subscription → scanned continuously across every agent.

Live: take a control a developer would set on a single agent and make it an org-wide rule. In Operate, Compliance, create a policy, choose the control type, say indirect-injection protection or a content filter, set its threshold, then set the scope to a subscription or resource group with optional exceptions, and activate. From then on the control plane scans every agent in scope continuously and flags any that fall out of compliance. A developer's optional control becomes a mandate nobody can quietly switch off.
25 · Red teaming

## Attack your own agents, on purpose

Find the holes before someone else does

- Probe: automated attacks, every category and strategy
- Score: an Attack Success Rate baseline, by technique
- Harden: add or tighten guardrails where attacks land
- Re-scan: the same suite, and watch the ASR fall
- Gate: re-run on every release, in CI/CD

##### Attack strategies (PyRIT)

Base64 ROT13 Unicode confusables Crescendo multi-language

##### Risk categories

Hate Violence Sexual Self-harm

The AI Red Teaming Agent (built on PyRIT, preview) runs automated adversarial scans and scores an Attack Success Rate, so you can prove a guardrail policy actually moved the needle.

Guardrails are defence; red teaming is offence, and you want both. The AI Red Teaming Agent, in preview, is built on Microsoft's open-source PyRIT. It runs automated adversarial scans against a model or agent endpoint, scores each attack-response pair, and produces a scorecard with Attack Success Rate as the headline metric, broken down by attack technique and risk category. Strategies include obfuscation like Base64, ROT13 and Unicode confusables, the multi-turn Crescendo attack, and multi-language prompts; categories are the four content-safety harms: hate, violence, sexual and self-harm. The powerful move, shown in the demo, is running the same scan before and after attaching a guardrail policy: the attack-success-rate drop is your evidence the control works. The bars here are illustrative; your numbers depend on your agent and corpus.
Live demo

## Run the advanced attack suite

Mutate attacks with encodings (Base64, ROT13, Unicode) and other languages → add your own attack prompts → score the ASR per category and strategy.

Live: the advanced scan. The AI Red Teaming Agent takes a set of attack objectives and mutates them with PyRIT converters, Base64, ROT13, character-space and Unicode confusables, plus composite chains, then translates them into other languages like Spanish and French, and folds in custom attack objectives seeded from a JSON file. It fires the whole suite at the gpt-4.1-mini deployment through the APIM gateway, scores every attack-response pair, and produces a scorecard with the Attack Success Rate broken down by strategy and risk category. Re-running the same scan after a guardrail policy is your before-and-after evidence.
26 · Fleet ops

## One to-do list for every agent, any cloud

#### Fleet to-do

Jailbreak attempt blocked · contoso-bank-agent
Eval below threshold · aria-rm-briefing-agent
Policy gap: indirect injection off · 2 agents
Cost spike · contoso-pmo-agent, +180% today
Error rate 100% · contoso-bank-agent
2 agents in Unknown state · github-copilot

#### Assets · Agents

Name | Source | Project | Status | Errors | Tokens | Runs
--- | --- | --- | --- | --- | --- | ---
github-copilot | Foundry | project-copilot-sdk | Unknown | 0.00% | - | 18
aria-rm-briefing-agent | Foundry | project-admin-c2676f | Running | 1.92% | 153.8K | 52
contoso-bank-agent | Foundry | project-admin-c2676f | Running | 100.00% | - | 40
contoso-pmo-agent | Foundry | project-admin-c2676f | Running | 0.00% | 15.3K | 4
SpaceExpert | Foundry | project-alpha-mesh | Running | 0.00% | 860 | 3
arxiv-nlp-agent | Foundry | iq-project | Running | 0.00% | 27K | 5
team-beta-agent | Foundry | project-beta-c2676f | Running | 0.00% | 56 | 1
team-delta-agent | Foundry | project-delta-c2676f | Running | 0.00% | 56 | 1
claims-triage | AI Gateway | LangGraph | Running | 0.00% | 4.1K | 12
legacy-helpdesk | AI Gateway | AWS | Blocked | - | - | 0

Operate is a to-do list for the fleet: blocked jailbreaks, evals below threshold, policy gaps. External agents (e.g., AWS) join the same view by routing through the AI Gateway.

Fleet-wide operations, the fourth pillar. The Operate view is best thought of as a to-do list for the whole fleet: blocked jailbreak attempts, evaluation scores below threshold, policy-compliance gaps, cost spikes, sortable by error rate or token cost across every project. The assets inventory lists every agent with its source, status, cost and risk, and from an alert you jump straight into that agent's build experience to fix it, no context switching. The big one for this room: non-Foundry agents, a LangGraph app, something on AWS, join the exact same view by routing their traffic through the AI Gateway, which is Azure API Management, and you can block or unblock them at the gateway even though Foundry does not run their infrastructure.
Live demo

## The admin fleet overview

Operate: the fleet overview and assets inventory.

portal · Operate
Live in Operate. Start on Overview for the fleet to-do list and cost hero stats. Open Assets to show a Foundry-built agent and an externally-hosted agent, registered through the AI Gateway, in one inventory. This is the IT and security admin's home base.
27 · Provisioning

## The admin essentials

#### Four Foundry roles

Foundry User · build & call: models, agents, evals, data plane
Project Manager · manage projects, assign User; full data plane
Account Owner · deploy, connections, quota; no data plane
Owner · unrestricted: full control + data plane
tip: assign to groups, not people

#### Two-layer RBAC

Layer 1 Foundry plane · portal, projects, build
Layer 2 Cognitive Services plane · content filters, quota, keys
gotcha: the second layer is commonly missed

#### Regions & quota

Pick by model availability + residency
Quota is TPM per region + subscription

#### Cost & SDK

Platform is free; pay per deployment
No hard spend cutoffs · budget alerts
SDK: azure-ai-projects + OpenAI client

For platform owners: four roles (assigned to groups), a two-layer RBAC model, TPM quota by region and subscription, and a free platform you meter at the deployment level.

One reference card for the platform owners in the room. Four built-in Foundry roles, recently renamed: Foundry User to build and call, Project Manager to manage projects, Account Owner for deployment and the control plane, and Owner for unrestricted access, and you assign them to Entra groups, not individuals. RBAC is two layers that are configured separately: the Foundry plane for portal and build, and the underlying Cognitive Services plane for content filters, quota and keys, and the second layer is the one people forget. Regions are chosen by model availability and data residency, quota is tokens per minute per region and subscription. Cost: the platform itself is free, you pay at the deployment level, there are no hard spending cutoffs so you wire up budget alerts, and the SDK story is the unified azure-ai-projects package plus the standard OpenAI client.
28 · Why Foundry

## One platform, two jobs done well

#### Build

- Deploy any model behind one governed gateway
- Agents on the Responses API and one SDK
- Knowledge and tools by config, MCP-native
- Hosted runtimes in the framework you choose
- Grounded, cited answers with Foundry IQ

#### Operate

- A Microsoft Entra identity for every agent
- Guardrails on inputs, outputs and tool traffic
- Tracing and continuous evaluation on live traffic
- Red teaming that measures attack success
- One fleet view across projects and clouds

Build with rich primitives and govern them from the same place.

Pulling it together. The reason Foundry is worth your attention is not any single feature, it is the seam between build and operate. On the build side: deploy any model behind one governed gateway; build agents on the Responses API with one SDK; add knowledge and tools by configuration, MCP-native; host your own runtime in whatever framework you like; and ground answers with Foundry IQ. On the operate side: every agent gets a real Entra identity; guardrails sit on inputs, outputs and tool traffic; you trace and continuously evaluate live traffic; you red-team to measure attack success; and you see the whole fleet, including non-Foundry agents, in one place. Most platforms give you one half. Foundry's bet is that you need both, in the same flow.