DFdom@fradley
← ~/writing
$ cat firewall-rules-nobody-can-explain.md

Don't migrate a firewall rule nobody can explain

Run this against the legacy estate before anyone signs off the migration plan. It counts the allow rules in every network security group, and how many of them carry no description.

resources
| where type =~ 'microsoft.network/networksecuritygroups'
| mv-expand rule = properties.securityRules
| where tostring(rule.properties.access) == 'Allow'
| summarize allowRules = count(),
            noReason   = countif(isempty(tostring(rule.properties.description))),
            anySource  = countif(tostring(rule.properties.sourceAddressPrefix) in ('*', 'Internet', '0.0.0.0/0'))
  by subscriptionId
| order by allowRules desc

The description is the only field on an NSG rule that can say what the rule is for. Every rule in the gap between the first two columns grants access that nobody wrote a reason down for, and on an estate whose builders have moved on, nobody can supply one.

The wider numbers suggest the gap will be large. FireMon's Insights 2.0 data, published in June and drawn from 9.2 million policy checks, found 69% of firewall rules unused and 45% with no owner or documentation. Those are firewalls somebody still manages.

Migration plans usually copy such rules across as they are, because a copied rule breaks nothing on cutover night. That copies the exposure too, such as access opened for a supplier who stopped connecting years ago or a temporary test path nobody closed. Ownership moves as well: an unexplained rule that belonged to the people who left now sits in the platform team's Terraform, under their names in the commit history, and the next audit asks them what it is for. One spoke per workload with egress forced through the hub is meant to limit how far a compromise can travel, and a copied ruleset restores the flat network's access inside it.

Flow logs show what the network does now

A rule shows what somebody intended on the day they wrote it; a flow log shows what crosses the network today. Run the workshops, but check what people remember against the telemetry. If someone says a batch server writes to the finance database, it becomes a rule when the flows confirm it.

On Azure the telemetry is virtual network flow logs, processed by Traffic Analytics into a Log Analytics workspace. NSG flow logs, the older option, are on their way out: the azurerm provider documentation notes that new ones have been impossible to create since 30 July 2025, and Microsoft retires the feature on 30 September 2027. You enable VNet flow logs once per virtual network rather than per NSG, and they also capture traffic through VPN and ExpressRoute gateways, which a legacy estate usually leans on. Every flow record names the rule that allowed or denied it.

resource "azurerm_network_watcher_flow_log" "legacy" {
  name                 = "fl-legacy-vnet"
  network_watcher_name = "NetworkWatcher_uksouth"
  resource_group_name  = "NetworkWatcherRG"

  target_resource_id = data.azurerm_virtual_network.legacy.id
  storage_account_id = azurerm_storage_account.flowlogs.id
  enabled            = true

  retention_policy {
    enabled = true
    days    = 90
  }

  traffic_analytics {
    enabled               = true
    workspace_id          = azurerm_log_analytics_workspace.discovery.workspace_id
    workspace_region      = azurerm_log_analytics_workspace.discovery.location
    workspace_resource_id = azurerm_log_analytics_workspace.discovery.id
    interval_in_minutes   = 60
  }
}

That is azurerm 5.5.0, where target_resource_id takes the VNet's ID. The storage account has to be standard tier in the same region as the VNet, with no existing lifecycle rules, because the flow log overwrites them.

At UK South retail prices, collection is $0.50 per GB after 5 GB free per subscription each month, and Traffic Analytics processing is $2.30 per GB at the 60-minute interval or $3.50 at 10 minutes, with workspace ingestion and blob storage on top. Discovery doesn't need ten-minute freshness.

Two blind spots. VNet flow logs don't record traffic for a list of platform services that includes App Service, Azure Functions, Logic Apps and SQL Managed Instance, so read flows involving those from the other end or from the service's own diagnostics. And if the legacy estate routes through Azure Firewall, its rules need the same treatment from the AZFWNetworkRule and AZFWApplicationRule log tables, where Policy Analytics flags low-utilisation rules over a 30-day window.

Sixty days, including a month-end

Flow logs only show what happened while they were switched on. A month-end reconciliation or the annual disaster recovery test won't appear in a window that didn't contain it, and a cutover that drops one of those flows fails quietly until the job next runs. The window has to include at least one month-end, and a quarter-end if the calendar allows. Ask finance and operations what runs on a schedule, and put anything that runs once a year on a written list with a named owner.

Turn the flows into a candidate ruleset

Traffic Analytics writes to the NTANetAnalytics table, already aggregated. This query collapses sixty days into one row per conversation:

NTANetAnalytics
| where TimeGenerated > ago(60d)
| where SubType =~ 'FlowLog'
| where isnotempty(SrcIp) and isnotempty(DestIp)
| where AllowedInFlows + AllowedOutFlows > 0
| summarize flows     = sum(AllowedInFlows + AllowedOutFlows),
            firstSeen = min(FlowStartTime),
            lastSeen  = max(FlowEndTime),
            viaRules  = make_set(AclRule, 10)
  by SrcSubnet, SrcIp, DestSubnet, DestIp, DestPort, L4Protocol
| order by DestSubnet asc, DestPort asc, flows desc

Each row is a candidate rule: this source reached this destination on this port, this often, between these dates, through these existing rules. The flow count is relative rather than exact, because flow logs record a conversation between two VMs at both network interfaces. SrcIp and DestIp are blank for traffic to and from public addresses, so this covers east-west and on-premises traffic; the internet-bound half sits in DestPublicIps and gets its own query in the runnable version below.

The second query starts from the existing rules and counts their traffic:

NTANetAnalytics
| where TimeGenerated > ago(60d)
| where SubType =~ 'FlowLog' and isnotempty(AclRule)
| summarize allowed = sum(AllowedInFlows + AllowedOutFlows),
            denied  = sum(DeniedInFlows + DeniedOutFlows),
            lastHit = max(FlowEndTime)
  by AclGroup, AclRule
| order by AclGroup asc, allowed asc

Any NSG rule from the Resource Graph query that never appears here matched no traffic for the whole window. Start the review there.

Nothing gets a rule because it happened

Some of what crosses a legacy network is a job retrying against a server switched off years ago. Some of it could be an attacker who has been inside for a while, and the flow log can't tell the two apart from legitimate traffic.

Every candidate rule needs a person who can say what the flow is for, judged against intended behaviour: what this workload is supposed to talk to. A flow nobody can account for gets no rule. The ruleset ends in deny, so an unexplained flow fails closed in the new estate instead of being carried into it. Your security lead should co-sign the review, since approving flows is a security decision. It can also surface an existing compromise, as traffic to a destination nobody can explain is exactly what it turns up.

Traffic Analytics sets FlowType to MaliciousFlow when the public address at the other end appears in Microsoft's threat intelligence feeds, with detail in the NTAIpDetails table, so run that filter before the review meets. A clean result only means nothing matched a known-bad address.

Deny before you delete

For legacy rules that matched no traffic, and for flows nobody claims, add an explicit deny at a higher priority before deleting anything; removing the deny reverses it in seconds. Leave it in place through the next month-end and watch for denied flows in the same workspace. The same idea applied to whole workloads is the retire-first pass in lift-and-shift with a written end date.

Everything above is runnable from blog-examples/ruleset-from-traffic, along with a script that lists the NSG rules that saw no traffic.

This week

  1. Run the Resource Graph query and put its first two columns in the migration plan as a named risk.
  2. Apply the flow log to each legacy VNet with Traffic Analytics at the 60-minute interval, and set a review date sixty days out. Move it later if the window misses a month-end. If only one of these happens today, make it this one, because the sixty days only start counting once the logs are on.
  3. Ask finance and operations what runs on a schedule, and write down every annual process with an owner.
  4. Agree with your security lead now that they co-sign the rule review.
discuss on linkedin → more writing