DFdom@fradley
← ~/writing
$ cat firewall-rules-nobody-can-explain.md

Don't migrate a firewall rule nobody can explain

Run this against the legacy estate before anyone signs off the migration plan. It counts the allow rules in every network security group, and how many of them carry no description.

resources
| where type =~ 'microsoft.network/networksecuritygroups'
| mv-expand rule = properties.securityRules
| where tostring(rule.properties.access) == 'Allow'
| summarize allowRules = count(),
            noReason   = countif(isempty(tostring(rule.properties.description))),
            anySource  = countif(tostring(rule.properties.sourceAddressPrefix) in ('*', 'Internet', '0.0.0.0/0'))
  by subscriptionId
| order by allowRules desc

The description is the only field on an NSG rule that can say what the rule is for. Compare the first two columns. Every rule in the gap between them grants access that nobody wrote a reason down for, and on an estate whose builders have moved on, nobody left can supply one.

The wider numbers suggest the gap will be large. FireMon's Insights 2.0 data, published in June and drawn from 9.2 million policy checks, found 69% of firewall rules unused and 45% with no owner or documentation. Those are firewalls somebody still manages. A ruleset whose authors have all left is unlikely to do better.

The migration plan usually copies those rules into the new landing zone as they are. Nobody can say which ones are safe to drop, and a copied rule at least breaks nothing on cutover night.

Copying the rules moves the exposure with them

Copying looks like the cautious option, and it guarantees that the new estate grants everything the old one granted, including access opened for a supplier who stopped connecting years ago and a temporary test path that nobody closed.

Ownership changes after cutover. On the legacy estate an unexplained rule belongs to the people who left. In the new estate it lives in the platform team's Terraform, under their names in the commit history, and the next audit asks them what it is for.

Copying also undoes the design you are migrating towards. One spoke per workload, with egress forced through the hub, exists to limit how far a compromise can travel. A ruleset nobody can attest to has a blast radius equal to whatever it permits. Copied into the spokes, it rebuilds the flat network inside a topology that only looks segmented on the diagram.

Flow logs show what the network does now

A rule records what somebody intended on the day they wrote it. A flow log records what crosses the network now. Once the people who wrote the rules have gone, the flow log is the one source that has not drifted from reality.

That changes what discovery is for. The workshops still happen, but what people remember becomes a hypothesis to test against the telemetry rather than an input to the design. Someone recalling that a batch server writes to the finance database is useful, and it becomes a rule when the flows confirm it.

Turn on VNet flow logs first

On Azure the telemetry is virtual network flow logs, processed by Traffic Analytics into a Log Analytics workspace.

NSG flow logs, the older option, are on their way out. The azurerm provider documentation notes that new ones have been impossible to create since 30 July 2025, and Microsoft retires the feature on 30 September 2027, deleting the flow log resources that remain. You enable VNet flow logs once per virtual network rather than per NSG, and they also capture traffic through VPN and ExpressRoute gateways, which a legacy estate usually leans on. For this job, the most useful property is that every flow record names the rule that allowed or denied it.

resource "azurerm_network_watcher_flow_log" "legacy" {
  name                 = "fl-legacy-vnet"
  network_watcher_name = "NetworkWatcher_uksouth"
  resource_group_name  = "NetworkWatcherRG"

  target_resource_id = data.azurerm_virtual_network.legacy.id
  storage_account_id = azurerm_storage_account.flowlogs.id
  enabled            = true

  retention_policy {
    enabled = true
    days    = 90
  }

  traffic_analytics {
    enabled               = true
    workspace_id          = azurerm_log_analytics_workspace.discovery.workspace_id
    workspace_region      = azurerm_log_analytics_workspace.discovery.location
    workspace_resource_id = azurerm_log_analytics_workspace.discovery.id
    interval_in_minutes   = 60
  }
}

That is azurerm 5.5.0, where target_resource_id takes the VNet's ID. The storage account has to be a standard tier account in the same region as the VNet, and the provider's own note says to give it a storage account with no existing lifecycle rules, because the flow log overwrites them.

The cost is modest for what it buys. At UK South retail prices, collection is $0.50 per GB after 5 GB free per subscription each month, and Traffic Analytics processing is $2.30 per GB at the 60-minute interval or $3.50 at 10 minutes. Workspace ingestion and blob storage cost extra. Discovery does not need ten-minute freshness, so the 60-minute interval is the right setting.

VNet flow logs have two blind spots. They do not record traffic for a list of platform services that includes App Service, Azure Functions, Logic Apps and SQL Managed Instance, so you read flows involving those from the other end of the conversation or from the service's own diagnostics. And if the legacy estate routes through Azure Firewall, its rules need the same treatment from the AZFWNetworkRule and AZFWApplicationRule log tables, where Policy Analytics will flag low-utilisation rules over a 30-day window.

Sixty days is the minimum, and a business cycle is the real test

Flow logs only show what happened while they were switched on. A month-end reconciliation or the annual disaster recovery test will not appear in a window that did not contain it, and a cutover that drops one of those flows fails quietly until the next time the job runs.

So the window has to include at least one month-end, and a quarter-end if the calendar allows. The remaining gap gets closed by asking rather than observing: finance and operations know what runs on a schedule. Anything that only runs once a year goes on a written list with a named owner, because no practical observation window will catch it.

Turn the flows into a candidate ruleset

Traffic Analytics writes to the NTANetAnalytics table, already aggregated: flows that share source, destination, port, protocol, direction and rule become one record per processing interval. This query collapses sixty days of that into one row per conversation.

NTANetAnalytics
| where TimeGenerated > ago(60d)
| where SubType =~ 'FlowLog'
| where isnotempty(SrcIp) and isnotempty(DestIp)
| where AllowedInFlows + AllowedOutFlows > 0
| summarize flows     = sum(AllowedInFlows + AllowedOutFlows),
            firstSeen = min(FlowStartTime),
            lastSeen  = max(FlowEndTime),
            viaRules  = make_set(AclRule, 10)
  by SrcSubnet, SrcIp, DestSubnet, DestIp, DestPort, L4Protocol
| order by DestSubnet asc, DestPort asc, flows desc

Each row is a candidate rule: this source reached this destination on this port, this often, between these dates, through these existing rules. The flow count is a relative measure rather than an exact one, because flow logs record a conversation between two VMs at both network interfaces. SrcIp and DestIp are blank for traffic to and from public addresses, so this covers east-west and on-premises traffic; the internet-bound half sits in DestPublicIps and gets its own query in the runnable version below.

The second query runs the other way, from the existing rules towards the traffic:

NTANetAnalytics
| where TimeGenerated > ago(60d)
| where SubType =~ 'FlowLog' and isnotempty(AclRule)
| summarize allowed = sum(AllowedInFlows + AllowedOutFlows),
            denied  = sum(DeniedInFlows + DeniedOutFlows),
            lastHit = max(FlowEndTime)
  by AclGroup, AclRule
| order by AclGroup asc, allowed asc

Any NSG rule from the Resource Graph query that never appears in this output matched no traffic for the whole window. Those rules are where the review starts, because they are the cheapest to challenge.

Nothing gets a rule because it happened

Observation produces a candidate ruleset that still needs approval. Some of what crosses a legacy network is a job retrying against a server that was switched off years ago. Some of it could be an attacker who has been inside for a while. The flow log cannot tell either apart from legitimate traffic, so the review has to.

Every candidate rule needs a person who can say what the flow is for, and the review is against intended behaviour: what this workload is supposed to talk to. A flow nobody can account for gets no rule. The ruleset ends in deny, so an unexplained flow fails closed in the new estate instead of being carried into it.

That makes the review a security decision, so your security lead co-signs it rather than leaving it to the platform team alone. It is also where an existing compromise is more likely to surface than to hide, since traffic to a destination nobody can explain is exactly what an intended-behaviour review turns up.

Traffic Analytics helps at the edges of this. It sets FlowType to MaliciousFlow when the public address at the other end appears in Microsoft's threat intelligence feeds, with the detail in the NTAIpDetails table, so run that filter before the review meets. A clean result only means nothing matched a known-bad address. It does not clear the estate.

The review also shrinks the problem to a manageable size. An inherited ruleset nobody can attest to becomes a list with an owner against every line.

Deny before you delete

For legacy rules that matched no traffic, and for flows nobody claims, the first move is an explicit deny at a higher priority rather than a deletion. Removing one rule reverses it in seconds. Leave the deny in place through the next month-end and watch for denied flows in the same workspace. The same idea applied to whole workloads is the retire-first pass in lift-and-shift with a written end date.

Everything above is runnable from blog-examples/ruleset-from-traffic, along with a script that lists the NSG rules that saw no traffic.

This week

  1. Run the Resource Graph query and put its first two columns in the migration plan as a named risk, with the number beside it.
  2. Apply the flow log to each legacy VNet with Traffic Analytics at the 60-minute interval, and set a review date sixty days out. Move it later if the window misses a month-end.
  3. Ask finance and operations what runs on a schedule, and write down every annual process with an owner.
  4. Agree with your security lead now that they co-sign the rule review, before there is a ruleset for them to look at.

If only one of these happens today, make it the second. The sixty days only start counting once the logs are on.

discuss on linkedin → more writing