A spoke VM initiates a connection to an on-premises server. The request travels through the hub NVA, gets inspected, and arrives at its destination. The response comes back, but via the ExpressRoute gateway directly into the spoke, skipping the NVA entirely. The firewall sees a return packet with no matching session state and drops it silently. The connection hangs. No error. No obvious log entry. Just a timeout.
This is asymmetric routing. In hub-and-spoke topologies it is not an edge case or a misconfiguration you can avoid with enough care. It is a structural consequence of how Azure SDN, VNet peering, User-Defined Routes, and stateful security appliances interact. The question is not whether you will encounter it. It is whether you designed for it upfront or are debugging it at 2am.
This post covers the five most common asymmetry scenarios in hybrid Azure topologies, the tools to detect them before users do, and a set of practical techniques that are not in the standard documentation.
Asymmetric routing occurs when the forward path and the return path of a network flow traverse different devices or network segments. In a stateless network this is harmless. In a network with stateful inspection (firewalls, NVAs, IDS/IPS) it is fatal: the stateful device only sees one direction of the flow, has no session entry for the return packets, and drops them as invalid.
Hub-and-spoke topology creates the conditions for asymmetry through several compounding mechanisms.
VNet peering is non-transitive. Spokes cannot communicate directly with each other or with on-premises networks. All traffic must be routed through the hub. But Azure automatically creates system routes in every VNet when peering is established, and those system routes can pull traffic directly between the peered VNets if UDRs are not explicitly overriding them.
UDRs must be configured manually on every subnet. There is no global routing policy in self-managed hub-and-spoke. If you forget a subnet, or add a new one without updating route tables, that subnet uses system routes and traffic flows around your intended path.
Gateway route propagation is on by default. When you enable Allow Gateway Transit on a hub peering and Use Remote Gateways on the spoke side, the spoke automatically learns on-premises prefixes from the gateway. Those learned routes are more specific than your UDR 0.0.0.0/0. The spoke sends traffic to on-premises directly through the gateway, bypassing the firewall.
Private Endpoints override routing by design. A Private Endpoint creates implicit /32 host routes that propagate across all peered VNets. These routes take precedence over UDRs because they are more specific. Network Policy for Private Endpoints is disabled by default on all subnets, meaning NSGs and UDRs are ignored for PE traffic unless you explicitly enable it.
The table below summarizes all five scenarios covered in this section. Each is discussed in detail below.
| Traffic Flow | Root Cause | Symptom | Fix |
|---|---|---|---|
| On-premises to spoke (via gateway) | No UDR on GatewaySubnet for spoke prefixes | Return traffic from spoke hits firewall; inbound bypasses it. Stateful firewall drops return packets. | Add per-spoke UDRs to GatewaySubnet (exact /24, not summary). Next hop: firewall private IP. |
| Hub workload to spoke | Hub subnets have no route table; they use system routes from peering | Hub VM reaches spoke directly. Spoke returns via firewall. Session dropped. | Attach route table to every hub workload subnet. UDR to spoke prefixes via firewall. |
| Spoke to internet / on-premises via active-active NVA | ECMP across NVA instances; Azure SDN doesn’t guarantee flow-level symmetry | Inbound and outbound land on different NVA instances. Stateful firewall drops. | SNAT on NVA, or ILB with HA Ports + Floating IP, or session state sync between NVAs. |
| On-premises to Private Endpoint in spoke | PE implicit /32 route bypasses UDR; Network Policy disabled by default | Traffic skips firewall on inbound; PE subnet returns via firewall. Asymmetry and inspection bypass. | Enable Route Table Network Policy on PE subnet. Or add explicit /32 UDR on client subnet. |
| ExpressRoute + VPN coexistence | BGP path selection differs between Azure side and on-premises side | Traffic leaves via ER, returns via VPN (or vice versa). On-prem stateful firewall drops return. | Set Local Preference higher for ER on-prem routers. AS path prepend on VPN side. |
Traffic arrives from on-premises through the ExpressRoute or VPN gateway into the GatewaySubnet. By default, Azure creates system routes in the GatewaySubnet for each peered spoke VNet. Those system routes send traffic directly to the spokes, bypassing the firewall. Return traffic from the spoke goes through the firewall because the spoke has a UDR pointing there. The firewall sees only the return packet and drops it.
The fix: Add UDRs to the GatewaySubnet for each spoke prefix, with next hop set to the firewall’s private IP.
| WATCH OUT GatewaySubnet does not support 0.0.0.0/0 UDRs. You cannot add a default route to the GatewaySubnet. You must add explicit per-spoke prefixes. Using a summary route such as 10.1.0.0/16 to cover all spokes will not work either: the system routes injected by VNet peering are /24 (or whatever the spoke prefix is), which are more specific than the /16 summary and will win. This means every time you add a new spoke, you must also add a new UDR entry to the GatewaySubnet route table. If this is not part of your spoke provisioning automation, you will create asymmetry on every new spoke deployment. |
| PRO TIP Audit all GatewaySubnets in your environment with one command: az network vnet subnet show –name GatewaySubnet –vnet-name <hub-vnet> –resource-group <rg> –query routeTable If routeTable is null, the subnet has no UDRs and is vulnerable to this asymmetry. Script this across all hub VNets and run it as part of your network health checks. |
Hub subnets that host actual workloads (shared services, management VMs, DNS resolvers, jump boxes) are frequently left without route tables. The reasoning is that the hub is the “center” and routes everything else, so it does not need routing itself. This is wrong.
A VM in the hub reaches a spoke via the system route created by VNet peering, going directly. The spoke VM returns traffic through the firewall because its UDR says so. The firewall sees an unsolicited return packet and drops it.
The fix: Attach a route table to every hub workload subnet. Add UDRs for all spoke prefixes and for on-premises prefixes, with next hop set to the firewall. The firewall subnet itself is the only subnet that should not have a route table pointing back to the firewall.
| PRO TIP Use this command to list all subnets in a VNet that have no route table attached: az network vnet subnet list –vnet-name <hub-vnet> –resource-group <rg> –query “[?routeTable==null].name” Run this after every VNet change. Any subnet that appears here and is not the AzureFirewallSubnet or AzureBastionSubnet needs a route table. |
For high availability, NVA instances are deployed in pairs. In an active-active configuration, both instances advertise the same routes (via Azure Route Server or identical UDRs pointing to an Internal Load Balancer). Azure’s SDN performs ECMP to distribute traffic across both.
The problem: Azure’s software-defined networking platform does not guarantee flow-level symmetry. The outbound packet of a flow may go through NVA instance A. The return packet may land on NVA instance B. Instance B has no session state for this flow and drops the packet. This is not a bug. It is the designed behavior of ECMP in Azure SDN.
There are three solutions, each with different tradeoffs.
Option 1: SNAT on each NVA instance. The NVA replaces the source IP of outbound packets with its own interface IP. Return traffic is guaranteed to come back to the same NVA instance because the destination IP is the NVA’s own address. Simple and reliable, but it hides the original source IP from downstream systems and logs.
Option 2: Internal Load Balancer with HA Ports and Floating IP. The ILB distributes flows across NVA instances using a 5-tuple hash, guaranteeing that all packets in the same flow go to the same NVA. Floating IP (Direct Server Return) ensures the NVA sees the original destination IP rather than the ILB’s frontend IP. This is the recommended pattern for East-West and on-premises traffic without SNAT.
Option 3: NVA session state synchronization. Some NVA vendors support replicating session state between active instances. Both NVAs can handle return traffic for any flow because they share state. This is the cleanest solution architecturally but is vendor-specific, adds inter-NVA traffic overhead, and is often expensive in licensing.
| WATCH OUT Do not use active-active NVA with identical AS-PATH advertisements for connections that require stateful inspection unless SNAT or session sync is in place. Azure Route Server with ECMP is designed for stateless workloads (SD-WAN, IPsec NVAs). Using it for next-generation firewalls without compensating mechanisms will produce intermittent and difficult-to-diagnose packet drops. |
| PRO TIP Before deploying NVA HA, check whether your NVA vendor supports asymmetric forwarding mode or session state synchronization. Check the vendor documentation for these exact terms. If neither is supported and SNAT is not acceptable, active-passive (not active-active) with UDR failover is the safer design. For Azure Firewall specifically: it is a managed service that handles SNAT and session tracking internally. Azure Firewall in an active-active secured hub does not produce this problem because Microsoft manages the state distribution. |
Private Endpoints are not simple NICs. When you deploy a Private Endpoint in a VNet, Azure automatically injects /32 host routes for the PE’s private IP into every VNet that is peered with the PE’s VNet. This propagation happens regardless of whether the peered VNets have UDRs in place.
The result: a VM in an on-premises network or in a remote spoke can reach the Private Endpoint directly through the peering system routes, bypassing any firewall that should be in the path. Return traffic from the PE subnet goes through the firewall (because the PE subnet has a UDR). One direction bypasses inspection. The other does not. Classic asymmetry.
Root cause: Network Policy for Private Endpoints is disabled by default on every subnet. When disabled, UDRs and NSGs are bypassed for traffic destined to Private Endpoints in that subnet, regardless of what the route table says.
The fix: Enable Route Table Network Policy on the subnet where Private Endpoints are deployed. This single setting causes UDRs to apply to PE traffic, allowing you to force it through the firewall.
| WATCH OUT Enabling Network Policy is a subnet-level setting that affects all Private Endpoints in the subnet. In existing deployments with many PEs, test the impact before enabling in production. Some legacy configurations depend on the default bypass behavior. As of late 2024, third-party NVAs support a resource tag disableSnatOnPL=true on the NVA NIC that removes the SNAT requirement for PE traffic. Azure Firewall and NVAs in vWAN Secured Hub still require SNAT for PE symmetry as of early 2025. |
| PRO TIP If enabling Network Policy on an existing PE subnet is too disruptive, use this workaround for specific PEs: add an explicit /32 UDR on the client subnet (not the PE subnet) pointing the PE’s private IP to the firewall. The /32 is more specific than the implicit PE route and will win. This surgical approach lets you redirect individual PE flows through inspection without touching the PE subnet’s Network Policy setting. |
When ExpressRoute and Site-to-Site VPN coexist in the same hub, both advertise routes to Azure. Azure always prefers ExpressRoute over VPN for the same prefix. But the return path from on-premises to Azure depends on what the on-premises router prefers, and that is under your control.
If on-premises BGP is not configured with a higher Local Preference for ExpressRoute, some prefixes may be reachable via both links and the router may send traffic via VPN to Azure while Azure sends traffic back via ExpressRoute. A stateful on-premises firewall sitting between the router and the rest of the network sees only one direction and drops the return.
The fix: Set Local Preference higher for routes received via ExpressRoute on all on-premises BGP routers. This ensures that for identical prefixes, the router always prefers ExpressRoute for both sending and receiving, matching Azure’s preference.
| WATCH OUT During ExpressRoute maintenance, Microsoft deliberately prepends the AS path on one of the two ER links to gracefully shift traffic. If your on-premises BGP configuration includes bgp bestpath as-path ignore, the router ignores this signal and may keep sending traffic on the link Microsoft is trying to drain. Traffic goes one way on the primary link and returns on the secondary. On-premises stateful firewall drops the return. Audit your on-premises BGP configuration for as-path ignore before every maintenance window. This is one of the most common sources of asymmetric routing during ER maintenance that looks like an Azure problem but is on-premises. |
| PRO TIP If you need to influence path selection without modifying Local Preference (e.g., you don’t control the on-premises router), use AS path prepending on the VPN BGP advertisement to make VPN routes less preferred. Add two or three copies of your own ASN to the AS path on the VPN side. The on-premises router will then select the shorter AS path, which is the ExpressRoute route. |
Azure Route Server (ARS) is a fully managed service that enables dynamic route exchange between NVAs and Azure SDN using BGP. Instead of manually maintaining UDRs in every spoke as new prefixes are added, the NVA advertises routes to ARS and ARS propagates them automatically to all connected spoke VNets.
This is powerful but introduces route precedence complexity that can produce its own asymmetry.
ExpressRoute routes take precedence over Route Server NVA routes when both advertise the same prefix length. If an NVA advertises 10.0.0.0/8 (a supernet) and ExpressRoute advertises specific /24 prefixes for on-premises, the ExpressRoute routes win for those /24 destinations. Traffic from spokes to those on-premises destinations goes directly through the gateway, bypassing the NVA. Return traffic from the NVA goes via the NVA. Asymmetry.
The fix: Configure NVAs to advertise routes with the no-advertise BGP community (65535:65282). This prevents Route Server from re-advertising NVA routes to other BGP peers, specifically back to on-premises via ExpressRoute. It stops the route injection from creating a feedback loop where on-premises learns Azure spoke prefixes via the NVA path rather than directly.
Route Server does not participate in the data plane. It only controls the control plane (what routes are programmed in VNet route tables). The actual packet forwarding still follows the UDR or system route. This means effective routes on a VM NIC may not match what you expect from ARS advertisements if there are conflicting UDRs or longer-prefix system routes.
| PRO TIP When troubleshooting Route Server behavior, use the hub-level Effective Routes view in Azure Virtual WAN or the Route Server BGP peer status, not just VM NIC effective routes. The hub router view shows which routes are being selected from which sources (ExpressRoute vs NVA vs VPN) and the AS paths, giving a complete picture of route selection decisions that NIC-level views do not show. For non-Virtual WAN: check both az network route-server list-advertised-routes and az network route-server list-learned-routes to see what ARS is distributing and what it has learned from each NVA peer. Compare against VM NIC effective routes to find where the disconnect is. |
Most asymmetric routing problems are detected by users, not by monitoring. The tools below change that.
The most underused asymmetry detection technique is treating effective routes as infrastructure state that must be versioned. After every IaC deployment, capture the effective routes of one representative VM per subnet and compare the diff to the previous state.
| Capture effective routes for a VM NIC az network nic show-effective-route-table \ –name \ –resource-group \ –output table > effective-routes-$(date +%Y%m%d).txt Bulk: all NICs in a resource group for nic in $(az network nic list -g –query ‘[].name’ -o tsv); do echo “=== $nic ===” az network nic show-effective-route-table -n $nic -g –query \ ‘value[].{prefix:addressPrefix[0],nextHop:nextHopType,via:nextHopIpAddress[0]}’ \ –output table done |
Commit this output to git alongside your Terraform or Bicep. A new spoke, a peering change, or an ExpressRoute prefix update will show up as a diff in the effective routes before it manifests as an incident.
| PRO TIP If you find that nextHopType shows VnetPeering or VirtualNetworkGateway for a destination where you expect VirtualAppliance, you have found your asymmetry. The traffic is bypassing your firewall for that destination. |
The combination of allowGatewayTransit (hub side) and useRemoteGateways (spoke side) is required for spoke-to-on-premises traffic. But if Propagate Gateway Routes is not disabled on the spoke route table, the spoke learns on-premises prefixes and routes directly to them, bypassing the firewall.
Run this script across all hub VNets to find mismatches:
| List all peering flag combinations in a hub VNet az network vnet peering list \ –vnet-name \ –resource-group \ –query ‘[].{name:name, allowGatewayTransit:allowGatewayTransit, \ useRemoteGateways:useRemoteGateways, \ allowForwardedTraffic:allowForwardedTraffic}’ \ –output table Check if spoke route table has Propagate Gateway Routes enabled az network route-table show \ –name \ –resource-group \ –query ‘disableBgpRoutePropagation’ |
If disableBgpRoutePropagation is false, gateway routes propagate to the spoke. If this spoke also has UDR 0.0.0.0/0 pointing to the firewall, and on-premises prefixes are more specific than the default route, those on-premises destinations bypass the firewall. Disable BGP route propagation on spoke route tables and rely on the 0.0.0.0/0 UDR for all on-premises traffic.
Connection Monitor is typically deployed after an incident. Deploy it before Go-Live, specifically targeting the traffic paths most vulnerable to asymmetry.
Configure Connection Monitor test groups for at minimum:
Run Connection Monitor for 48 to 72 hours before any production cutover. Any packet loss that tracks with specific source-destination pairs reveals asymmetry before the application team notices it.
| PRO TIP Connection Monitor tests cost fractions of a cent per test per hour. For a 10-test configuration running for a week before Go-Live, cost is under $5. The cost of a post-production asymmetric routing incident is considerably higher. Set alert rules on test failure percentage > 5% rather than 0%. Some packet loss is normal in a complex hybrid topology. Alert on consistent failure, not transient. |
Before committing a UDR change, verify the expected routing decision using Network Watcher: Next Hop. This tool queries the Azure SDN for the routing decision for a specific source IP, destination IP, and NIC, without sending any actual traffic.
| Verify that spoke VM routes on-premises traffic through firewall az network watcher show-next-hop \ –resource-group \ –vm \ –nic \ –source-ip \ –dest-ip Expected output: nextHopType: VirtualAppliance nextHopIpAddress: If you see VirtualNetworkGateway or Internet, the UDR is missing or being overridden |
Run this for both directions of every critical flow: source to destination, and destination to source. Asymmetric routing means these two calls will return different nextHopType values. That mismatch is your problem.
Once asymmetry is identified, the fix depends on the scenario and constraints. The table below compares the four main approaches.
| Approach | When to use | What you lose | Operational cost |
|---|---|---|---|
| SNAT on NVA | Active-active NVA HA, North-South traffic, any stateful inspection with multiple instances | Original source IP hidden in logs. Problems for IP-based security policies and audit trails. | Low once configured. Azure Firewall does SNAT automatically for non-RFC1918 destinations. |
| Precise UDRs on every subnet | Single NVA instance, smaller topologies, when source IP preservation is mandatory | Manual UDR maintenance per spoke per subnet. Route table proliferation. IaC required. | High. Every new spoke, subnet, or on-premises prefix requires UDR updates everywhere. |
| ILB with HA Ports + Floating IP | Active-active NVA HA, East-West and on-premises traffic, when SNAT is acceptable | Load balancer adds latency hop. Floating IP config is non-obvious. | Medium. ILB manages flow symmetry automatically once configured correctly. |
| NVA session state sync | Active-active NVA HA, when source IP preservation is mandatory and SNAT is not acceptable | NVA vendor-specific feature. Adds complexity and licensing cost. | High. Vendor-dependent setup, inter-NVA traffic overhead, failure mode complexity. |
SNAT is the simplest operational choice when source IP preservation is not a hard requirement. For regulated environments where every packet must be logged with the true source IP, precise UDRs combined with ILB HA Ports is the right pattern. Reserve NVA session sync for cases where neither SNAT nor ILB is acceptable and the NVA vendor explicitly supports it.
Because on-premises prefixes learned via BGP from the gateway are more specific than 0.0.0.0/0. A /24 on-premises prefix beats a /0 default route. The spoke routes directly to the gateway for those destinations. Fix: disable Propagate Gateway Routes (set disableBgpRoutePropagation: true on the spoke route table). The 0.0.0.0/0 UDR then wins for all destinations including on-premises.
No. The GatewaySubnet does not support a default route UDR. You must add explicit per-prefix routes for each spoke and destination that should be directed through the firewall. This is a hard platform constraint, not a configuration option.
Propagate Gateway Routes (disableBgpRoutePropagation: false by default) causes a subnet’s route table to automatically learn routes from ExpressRoute and VPN gateways. Disable it (disableBgpRoutePropagation: true) on all spoke subnet route tables when you want the 0.0.0.0/0 UDR to control all non-local traffic, including traffic to on-premises. Leave it enabled on the GatewaySubnet so the gateway can participate in routing decisions.
Virtual WAN’s managed routing engine handles most UDR scenarios automatically through Routing Intent and routing policies. However, Private Endpoints still require Route Table Network Policy to be enabled manually on the PE subnet. Active-active NVA with ECMP in vWAN still requires SNAT or session sync if the NVA performs stateful inspection. The platform eliminates the GatewaySubnet and hub workload UDR problems, but does not eliminate the NVA HA and Private Endpoint problems.
Stateful firewalls and NVAs maintain a session table. When a new connection is established, the device creates a session entry. Return packets are matched against that entry to be allowed through. When the return packet arrives at a device that never saw the original packet, it has no session entry and treats the return packet as unsolicited, dropping it as invalid traffic. The drop is silent because the device is functioning correctly: it is refusing traffic that does not match any known session.
No. System routes created by VNet peering use the exact spoke VNet prefix (e.g., /24). These are more specific than any summary route you add (e.g., /16). More specific routes always win. You must add the exact prefix for each spoke to the GatewaySubnet route table.
A resource tag introduced in GA in October 2024 that can be applied to the NIC (or VMSS instance) of a third-party NVA. When set (key: disableSnatOnPL, value: true), it removes the requirement to SNAT traffic destined to Private Endpoints for that specific NVA. As of early 2025, this applies only to third-party NVAs in standard VNet hub-and-spoke. Azure Firewall in VNet, Azure Firewall in vWAN Secured Hub, and third-party NVAs in vWAN Secured Hub still require SNAT for Private Endpoint traffic.
Asymmetric routing is not a problem you fix after it appears. By the time it manifests in production, the blast radius is wide and the diagnosis is time-consuming. These are the design decisions that prevent it.
Add GatewaySubnet UDRs as part of spoke provisioning automation. Every spoke creation workflow must include adding the spoke’s prefix to the GatewaySubnet route table. This is not optional maintenance. If it is not in the Terraform module or Bicep template that creates spokes, it will be missed.
Disable BGP route propagation on all spoke route tables. This single setting eliminates the most common cause of firewall bypass for on-premises traffic. The 0.0.0.0/0 UDR then governs all non-local destinations.
Enable Route Table Network Policy on every Private Endpoint subnet. The default (disabled) allows PE traffic to bypass UDRs and NSGs. This is almost never what a security-conscious architecture wants.
Treat hub workload subnets the same as spoke subnets. Every subnet in the hub that hosts workloads needs a route table with the same UDR discipline as spokes. The hub is not exempt from routing policy.
Choose active-passive over active-active for stateful NVA HA unless SNAT or ILB HA Ports is in the design from the start. Active-active with ECMP is powerful, but produces intermittent asymmetric routing with stateful firewalls that is difficult to reproduce and diagnose.
Configure Connection Monitor before Go-Live, not after the first incident. The test coverage costs almost nothing. The diagnostic value when something breaks is significant.
Asymmetric routing in hub-and-spoke topologies is not a configuration error you can avoid with sufficient attention. It is a structural property of the topology: VNet peering creates system routes, gateway propagation distributes on-premises prefixes, and Private Endpoints inject implicit host routes. All of these can override your intended routing if UDRs are not applied consistently across every subnet in every VNet.
The five scenarios in this post (missing GatewaySubnet UDRs, unprotected hub workload subnets, active-active NVA with ECMP, Private Endpoints with no Network Policy, and ExpressRoute plus VPN path mismatch) cover the majority of asymmetric routing incidents in production Azure hybrid environments. Each has a specific, verifiable fix.
The diagnostic toolkit (effective routes diff, peering flag audit, Connection Monitor as canary, Next Hop verification) shifts detection from reactive to proactive. Routing symmetry should be verified as part of deployment automation, not discovered during incident response.
If you are building a new environment: Virtual WAN eliminates the GatewaySubnet and hub workload problems but does not eliminate Private Endpoint Network Policy or NVA HA SNAT requirements. It simplifies the routing surface but does not flatten it to zero.
