DFdom@fradley
← ~/writing
$ cat secure-ai-behind-your-perimeter.md

Put your AI behind your own perimeter

A Foundry agent's data doesn't live in the model. In the standard setup its threads, files and vector stores sit in your own Storage, AI Search and Cosmos DB accounts, with secrets in Key Vault. This Resource Graph query finds every resource group holding a Foundry or Azure OpenAI account and shows those five services with their public network setting:

resources
| where type =~ 'microsoft.cognitiveservices/accounts'
| distinct subscriptionId, resourceGroup
| join kind=inner (
    resources
    | where type in~ ('microsoft.cognitiveservices/accounts', 'microsoft.search/searchservices',
        'microsoft.storage/storageaccounts', 'microsoft.documentdb/databaseaccounts',
        'microsoft.keyvault/vaults')
    | extend pna = tolower(tostring(properties.publicNetworkAccess)),
        pe = array_length(properties.privateEndpointConnections)
  ) on subscriptionId, resourceGroup
| join kind=leftouter (
    resourcecontainers
    | where type =~ 'microsoft.resources/subscriptions'
    | project subscriptionId,
        mg = tostring(properties.managementGroupAncestorsChain[0].displayName)
  ) on subscriptionId
| project mg, resourceGroup, name, service = tostring(split(type, '/')[0]),
    publicAccess = iff(isempty(pna), 'enabled (default)', pna),
    privateEndpoints = coalesce(pe, 0)
| order by mg asc, resourceGroup asc

Any row showing enabled with no private endpoint is open to the internet, whatever the model account's own setting. On storage, enabled (default) means nobody set it, so the account firewall decides, and it allows all networks unless someone changed it.

Why the network matters again

In July I wrote that identity is the perimeter. Agents bend that. An agent signs in as itself, holds the roles you gave it, then reads content someone else wrote: a web page, an email, a PDF in the index. Microsoft's Prompt Shields docs call a hidden instruction in that content a document attack and list "Unauthorized data exfiltration or data removal from a system" among the examples. A hijacked session passes every identity check, because the identity is genuine.

Microsoft's Zero Trust for AI guidance (March 2026) applies assume breach to exactly this case: "Design AI systems to be resilient to prompt injection, data poisoning, and lateral movement." IBM's 2025 breach report found 13% of organisations had reported breaches of AI models or applications, and "Of those compromised, 97% report not having AI access controls in place."

Principle Control in the AI workload Owner
Verify explicitly Entra ID only, keys off, one identity per agent Workload team
Least privilege RBAC per agent, approved tools only Workload team
Assume breach No public access in, egress through the hub firewall, document attack detection on Platform (network), workload (shields)

The first two rows are straight from CAF's AI security guidance. Document attack detection is off by default, so someone has to switch it on.

What online doesn't inherit from corp

Part 1 placed agents the way CAF's AI guidance does: internal under corp, internet-facing under online. For security, that choice decides what you inherit. In the Azure landing zone reference, corp carries the Public network access should be disabled for PaaS services initiative, enforced, with deny rules for all five services in the query. Online has no such assignment, so an internet-facing agent lands with whatever its template set, when only its front end needs to be public.

The example repo assigns five built-ins (all support Audit and Deny) to an online AI workload's subscription. The first is part 1's network policy again at subscription scope; the other four cover the data around the AI account:

  • Azure AI Services resources should restrict network access
  • Azure AI Search services should disable public network access
  • Storage accounts should disable public network access
  • Azure Cosmos DB should disable public network access
  • Azure Key Vault should disable public network access

Start in audit, as CAF's agent governance page recommends, and move to deny once the query comes back clean.

Decide egress before the vending request goes in

Spoke VNet workload landing zone Agent subnet delegated 0.0.0.0/0 to hub firewall Private endpoints one per service PUBLIC ACCESS OFF Foundry Storage AI Search Cosmos DB Key Vault Hub VNet platform landing zone Azure Firewall application rules, named FQDNs Approved destinations grounding, MCP, APIs Bing, web search, SharePoint grounding public endpoint, skips the hub 1 2 3
(1) Inbound only through private endpoints. (2) Traffic leaving the spoke goes to the hub firewall. (3) The firewall passes named destinations only. The grounding tools in red leave over a public endpoint and skip the hub.

Foundry's outbound networking is set when the resource is created: "You cannot update your outbound networking settings currently." So the design goes in the vending request. The baseline Foundry architecture for a landing zone gives the agent its own subnet in the spoke and routes its egress through the connectivity subscription. That subnet is delegated to Microsoft.App/environments, /27 at minimum (the agent networking page recommends /24), one Foundry resource each, same region as the VNet.

The Foundry network isolation page has the rest:

  • Outbound injection can't be added to an existing Foundry resource ("You must redeploy Foundry to add outbound networking"), and the delegated subnet can't be swapped.
  • Private endpoints for AI Search, Storage and Cosmos DB "are NOT auto-created". The query shows any that are missing.
  • Bing grounding, web search and SharePoint grounding still go over a public endpoint. Block them with Azure Policy if that breaks your rules.
  • Browser Automation, Computer Use, Image Generation, Logic Apps and the Fabric Data Agent don't support network isolation yet.
  • An AI gateway created from the Foundry portal "is automatically public". Build it through the Azure portal or code instead.
  • The agent subnet must reach login.microsoftonline.com and the other Entra endpoints, with no TLS inspection on them.

Each grounding source and public MCP server becomes a named rule on the hub firewall. Anything that can't route through the hub stays the workload team's problem, in the baseline's words: "You're responsible for mitigating data exfiltration and abuse."

Foundry also offers a managed virtual network with an "Allow only approved outbound" mode. It creates its own Azure Firewall for each Foundry account, in Microsoft's tenant, and the SKU is the only setting you control. It can't be turned off once enabled and doesn't log outbound traffic yet. In a landing zone that already has a hub firewall, I'd rather keep agent egress on the one the platform team already runs and logs.

The query and the Terraform for the five policies are in blog-examples/ai-workload-perimeter. Part 1 covers vending and the token quota, and LLM endpoints are the new public storage accounts covers turning the keys off.

discuss on linkedin → more writing