RansNet Docs - 2. L2 over SD-WAN

L2 over SD-WAN

(optional) Configuration Snips using IPSec

Overview

For most enterprises with many distributed remote offices/outlets, the remote sites are usually connected back to the HQ/DC network via layer-3 IP networks, either through the public Internet, 4G/5G, MPLS, or private leased-line, etc.

Traditional SD-WAN solutions are optimized for Layer-3 (IP) connectivity. However, some deployments require Layer-2 extension across geographically distributed sites, such as:

Extending VLANs across branches
Preserving broadcast-based services (DHCP, legacy protocols)
Supporting VM mobility or industrial automation networks
Supporting some applications that do not support TCP/IP routing

"L2 over SD-WAN" (aka "L2VPN over SD-WAN" or "Ethernet over VPN") addresses this type of requirements, by encapsulating Ethernet frames inside VXLAN tunnels, while using MP-BGP EVPN as the control plane to dynamically distribute MAC and VLAN reachability information. It also creates traffic isolation and enhances network security, by logically separating WAN and LAN traffic - the WAN links can make use of default routing table/underlay for establishing tunnel connectivity, while the LAN devices are communicating directly through a private VLAN, isolating from external reachability (similar to "VRF over SD-WAN" but in layer-2 mode).

Common use cases for "L2 over SD-WAN" include:

Factory automation networks
IoT or Retail chains with centralized services
Maritime / transportation systems
VM or container mobility across sites

Additionally, if only hub-and-spoke topology is requried, L2VPN also simplifies large distributed deployments:

No need address planning and routing on a per location basic (L3 networks need to calculate subnets for each location and manage routing per location)
Enabling seamless roaming/deployments for LAN devices at different locations. The router is "transparent" to LAN devices. No configuration dependency between router and LAN devices. Easy to roam or swap hardware.
Easy to centralize firewall policies. All LAN devices point to central gateway as next-hop (where internal routing and firewall happen).
Easy to pass security audit. The branch routers act like L2 switch in the data path - no visible IP to internal networks (can not be scanned or exploited).

NOTE:

L2VPN is not scalable for spoke-to-spoke topology, eg. all branches have direct access to each other. The broadcast domain will limit the size of a L2 network.
In "hub-and-spoke" topology, the gateway doesn't permit traffic (and broadcast frame) to pass between branch networks, hence it's scalable.

We will use this topology to elaborate the configuration guide.

In this scenario,

Use system default routing table to establish L2VPN tunnels. Configure WAN failover between links, if there's redundancy.
Bridge VPN tunnel to LAN interface, estabishing a flat Layer-2 network between locations.
Permit hub-and-spoke (branch to DC) or spoke-to-spoke (branch to branch) access.

Underlying key technologies:

Use Multipoint VXLAN as the encapsulation tunnel for layer-2 LAN extension.
Use BGP-EVPN with Multipoint GRE to exchange MAC/VLAN mapping (only needed for spoke-to-spoke topology).
Use WireGuard or IPSec at outer layer for data encryption.

Configuration on Gateway

As per typical RansNet SD-WAN configuration, most of the settings are done on the gateway. We just assign each branch to the VPN instance to auto push configurations to all devices. CLI sample snips are for references and expert mode only.

Step 1: Prepare and enable LAN interfaces for all routers (gateway and branch).

In this case we use VLAN1 for the LAN segment. There's no need to configure any other setting since it's just a layer-2 interface and it will be bridged to VPN later.

Step 2: Configure SD-WAN --> VPN Instance.

Set SD-WAN parameters. Select VPN topology and encryption prototol according to your actual requirements.

Bridge VXLAN (tunnel) and LAN interfaces.

Step 3: Assign branch routers to the VPN instance

This will auto compile and push all needed configurations to all devices within the same SD-WAN instance.

CLI Configuration Snip (on the gateway)

CLI can be overly complex for SD-WAN configuration. It's highly recommended to use mfusion for SD-WAN orchestration. For experts who want to understand the underlying CLI commands, below are the relevant gateway CLI configs (spoke-to-spoke setup, generated by mfusion).

NOTE: For testing & verification purpose, we configured IP on the br1 interface so that we can ping routers directly using bridge interface. In real life deployment, this is not necessary. You just need to configure LAN devices to be in the same subnet (can be any IP/network), and they'll be able to communicate directly, as if they're attached to a large virtual layer-2 switch.

!interface eth0 description "Default connection to WAN" enable ip address dhcp!interface gre1 tunnel local 10.1.168.1 enable ip address 10.1.172.1/22 ip neigh 10.1.172.2 10.1.168.2 ip neigh 10.1.172.3 10.1.168.3!interface vxlan1 vx-local 10.1.172.1 enable bridge-group 1!interface wg1 enable ip address 10.1.168.1/32 wg-peer b0-bb-8b-00-e7-a8 remote-net 10.1.168.2/32 wg-peer b0-bb-8b-00-ea-20 remote-net 10.1.168.3/32!interface vlan 1 1 enable bridge-group 1!interface bridge br1 description "Auto Interface from IPSec VPN (1)" enable ip address 10.1.1.1/24!router bgp 65051 bgp timer 5 15 neighbor 0168_RansNet_SSL2WG_1 as-peer neighbor 0168_RansNet_SSL2WG_1 as-remote 65051 neighbor 0168_RansNet_SSL2WG_1 next-hop-self neighbor 0168_RansNet_SSL2WG_1 route-reflector-client neighbor 0168_RansNet_SSL2WG_1 soft-reconfiguration neighbor 0168_RansNet_SSL2WG_1 weight 0 neighbor range 10.1.172.0/22 as-peer 0168_RansNet_SSL2WG_1 address-family-l2vpn advertise-all-vni neighbor 0168_RansNet_SSL2WG_1 activate neighbor 0168_RansNet_SSL2WG_1 route-reflector-client neighbor 0168_RansNet_SSL2WG_1 soft-reconfiguration!

Verifications

Gateway-1# show interface bridgeSummary of br1 -----------------------------------------------------------bridge name bridge id STP enabled interfacesbr1 8000.7a091ea3d1a0 no vlan1 vxlan1Gateway-1#Gateway-1# show ip bgp summary
IPv4 Unicast Summary (VRF default):BGP router identifier 10.18.18.194, local AS number 65051 vrf-id 0BGP table version 0RIB entries 0, using 0 bytes of memoryPeers 2, using 1448 KiB of memoryPeer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc*10.1.172.2 4 65051 2845 2845 0 0 0 03:55:55 0 0 N/A*10.1.172.3 4 65051 536 536 0 0 0 00:43:52 0 0 N/A
Total number of neighbors 2* - dynamic neighbor2 dynamic neighbor(s), limit 2000Gateway-1#
Gateway-1# show bgp l2vpnBGP table version is 58, local router ID is 10.18.18.194Status codes: s suppressed, d damped, h history, * valid, > best, i - internalOrigin codes: i - IGP, e - EGP, ? - incompleteEVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 10.18.18.169:2*>i[2]:[0]:[48]:[00:40:9d:23:e9:cb] 10.1.172.2 100 0 i RT:65051:1 ET:8*>i[2]:[0]:[48]:[00:90:0b:44:a6:73] 10.1.172.2 100 0 i RT:65051:1 ET:8*>i[2]:[0]:[48]:[30:65:ec:6a:e7:55] 10.1.172.2 100 0 i RT:65051:1 ET:8*>i[2]:[0]:[48]:[88:dc:96:87:3a:e2] 10.1.172.2 100 0 i RT:65051:1 ET:8*>i[2]:[0]:[48]:[88:dc:96:87:3a:f6] 10.1.172.2 100 0 i RT:65051:1 ET:8*>i[3]:[0]:[32]:[10.1.172.2] 10.1.172.2 100 0 i RT:65051:1 ET:8Route Distinguisher: 10.18.18.194:2*> [3]:[0]:[32]:[10.1.172.1] 10.1.172.1 32768 i ET:8 RT:65051:1Route Distinguisher: 10.18.18.251:2*>i[3]:[0]:[32]:[10.1.172.3] 10.1.172.3 100 0 i RT:65051:1 ET:8
Displayed 8 out of 8 total prefixesGateway-1#Gateway-1# ping 10.1.1.2 source 10.1.1.1PING 10.1.1.2 (10.1.1.2) from 10.1.1.1 : 56(84) bytes of data.64 bytes from 10.1.1.2: icmp_seq=1 ttl=64 time=1.36 ms64 bytes from 10.1.1.2: icmp_seq=2 ttl=64 time=1.26 ms64 bytes from 10.1.1.2: icmp_seq=3 ttl=64 time=1.21 ms64 bytes from 10.1.1.2: icmp_seq=4 ttl=64 time=1.17 ms64 bytes from 10.1.1.2: icmp_seq=5 ttl=64 time=1.26 ms
--- 10.1.1.2 ping statistics ---5 packets transmitted, 5 received, 0% packet loss, time 4006msrtt min/avg/max/mdev = 1.168/1.251/1.362/0.065 msGateway-1# ping 10.1.1.3 source 10.1.1.1PING 10.1.1.3 (10.1.1.3) from 10.1.1.1 : 56(84) bytes of data.64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=1.35 ms64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=1.13 ms64 bytes from 10.1.1.3: icmp_seq=3 ttl=64 time=1.19 ms64 bytes from 10.1.1.3: icmp_seq=4 ttl=64 time=1.29 ms64 bytes from 10.1.1.3: icmp_seq=5 ttl=64 time=1.15 ms
--- 10.1.1.3 ping statistics ---5 packets transmitted, 5 received, 0% packet loss, time 4006msrtt min/avg/max/mdev = 1.125/1.222/1.351/0.086 msGateway-1# show arpAddress HWtype HWaddress Flags Mask Iface10.1.1.2 ether 1e:cc:8b:49:fb:e7 C br110.1.1.3 ether 4a:e4:80:5f:64:4f C br1Gateway-1#

Configuration on Branch Routers

On the branch router, we just need to prepare VLAN1 interface (as per Step #1 above), and ensure the necessary network and firewall settings are in place.

All other related VPN/SD-WAN configs will be auto generated from previous gateway configuration steps.

CLI Configuration Snip

!interface eth0 description "Connection to WAN" enable ip address dhcp!interface gre1 tunnel local 10.1.168.2 remote 10.1.168.1 enable ip address 10.1.172.2/22!interface vxlan1 vx-local 10.1.172.2 enable bridge-group 1!interface wg1 enable ip address 10.1.168.2/32 wg-peer 00-60-e0-a3-59-f7 remote-ip sdwan.ransnet.com remote-net 10.1.168.1/32!interface vlan 1 1 description "Default VLAN for all LAN ports" enable bridge-group 1!interface bridge br1 description "Auto Interface from IPSec VPN (1)" bridge enable ip address 10.1.1.2/24!router bgp 65051 bgp timer 5 15 neighbor 0168_RansNet_SSL2WG_1 as-peer neighbor 0168_RansNet_SSL2WG_1 as-remote 65051 neighbor 0168_RansNet_SSL2WG_1 next-hop-self neighbor 0168_RansNet_SSL2WG_1 soft-reconfiguration neighbor 0168_RansNet_SSL2WG_1 weight 0 neighbor 10.1.172.1 as-peer 0168_RansNet_SSL2WG_1 address-family-l2vpn advertise-all-vni neighbor 0168_RansNet_SSL2WG_1 activate neighbor 0168_RansNet_SSL2WG_1 route-reflector-client neighbor 0168_RansNet_SSL2WG_1 soft-reconfiguration!

Verifications

Branch-2# show interface bridgeSummary of br-br1 -----------------------------------------------------------bridge name bridge id STP enabled interfacesbr-br1 7fff.1ecc8b49fbe7 no vlan1 vxlan1Branch-2#Branch-2# show ip bgp summary
IPv4 Unicast Summary (VRF default):BGP router identifier 10.18.18.169, local AS number 65051 vrf-id 0BGP table version 0RIB entries 0, using 0 bytes of memoryPeers 1, using 712 KiB of memoryPeer groups 1, using 32 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc10.1.172.1 4 65051 2951 2951 0 0 0 04:04:42 0 0 N/A
Total number of neighbors 1Branch-2#Branch-2# show ip bgp summary
IPv4 Unicast Summary (VRF default):BGP router identifier 10.18.18.169, local AS number 65051 vrf-id 0BGP table version 0RIB entries 0, using 0 bytes of memoryPeers 1, using 712 KiB of memoryPeer groups 1, using 32 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc10.1.172.1 4 65051 2972 2972 0 0 0 04:06:26 0 0 N/A
Total number of neighbors 1Branch-2#Branch-2# show bgp l2vpnBGP table version is 58, local router ID is 10.18.18.169Status codes: s suppressed, d damped, h history, * valid, > best, i - internalOrigin codes: i - IGP, e - EGP, ? - incompleteEVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 10.18.18.169:2*> [2]:[0]:[48]:[00:40:9d:23:e9:cb] 10.1.172.2 32768 i ET:8 RT:65051:1*> [2]:[0]:[48]:[00:90:0b:44:a6:73] 10.1.172.2 32768 i ET:8 RT:65051:1*> [2]:[0]:[48]:[30:65:ec:6a:e7:55] 10.1.172.2 32768 i ET:8 RT:65051:1*> [2]:[0]:[48]:[88:dc:96:87:3a:e2] 10.1.172.2 32768 i ET:8 RT:65051:1*> [2]:[0]:[48]:[88:dc:96:87:3a:f6] 10.1.172.2 32768 i ET:8 RT:65051:1*> [3]:[0]:[32]:[10.1.172.2] 10.1.172.2 32768 i ET:8 RT:65051:1Route Distinguisher: 10.18.18.194:2*>i[3]:[0]:[32]:[10.1.172.1] 10.1.172.1 100 0 i RT:65051:1 ET:8Route Distinguisher: 10.18.18.251:2*>i[3]:[0]:[32]:[10.1.172.3] 10.1.172.3 0 100 0 i RT:65051:1 ET:8
Displayed 8 out of 8 total prefixesBranch-2#Branch-2# ping 10.1.1.1 source 10.1.1.2PING 10.1.1.1 (10.1.1.1) from 10.1.1.2: 56 data bytes64 bytes from 10.1.1.1: seq=0 ttl=64 time=1.486 ms64 bytes from 10.1.1.1: seq=1 ttl=64 time=1.466 ms64 bytes from 10.1.1.1: seq=2 ttl=64 time=1.300 ms64 bytes from 10.1.1.1: seq=3 ttl=64 time=1.451 ms64 bytes from 10.1.1.1: seq=4 ttl=64 time=1.314 ms
--- 10.1.1.1 ping statistics ---5 packets transmitted, 5 packets received, 0% packet lossround-trip min/avg/max = 1.300/1.403/1.486 msBranch-2# ping 10.1.1.3 source 10.1.1.2PING 10.1.1.3 (10.1.1.3) from 10.1.1.2: 56 data bytes64 bytes from 10.1.1.3: seq=0 ttl=64 time=2.813 ms64 bytes from 10.1.1.3: seq=1 ttl=64 time=2.249 ms64 bytes from 10.1.1.3: seq=2 ttl=64 time=2.587 ms64 bytes from 10.1.1.3: seq=3 ttl=64 time=2.234 ms64 bytes from 10.1.1.3: seq=4 ttl=64 time=2.479 ms
--- 10.1.1.3 ping statistics ---5 packets transmitted, 5 packets received, 0% packet lossround-trip min/avg/max = 2.234/2.472/2.813 msBranch-2#Branch-2# show arpIP address HW type Flags HW address Mask Device10.1.1.1 0x1 0x2 7a:09:1e:a3:d1:a0 * br-br110.1.1.3 0x1 0x2 4a:e4:80:5f:64:4f * br-br1Branch-2#

(optional) Configuration Snips using IPSec

Some customers may require IPSec as the encryption protocol for compliance reasons. You just need to choose "IPSec" as the VPN protocol in step #2 above, then mfusion will auto generate the respective configurations using IPSec.

Below is a CLI snip for the gateway

interface eth0 description "Default connection to WAN" enable ip address dhcp!interface gre1 tunnel local 10.1.168.1 enable ip address 10.1.172.1/22 ip map 10.1.172.2 10.1.168.2 ip map 10.1.172.3 10.1.168.3!interface lo enable ip address 10.1.168.1/32!interface vxlan1 description "Auto Interface from VPN (1)" vx-local 10.1.172.1 enable bridge-group 1!interface vlan 1 1 enable bridge-group 1!interface bridge br1 description "Auto Interface from IPSec VPN (1)" enable!ipsec ike-policy 1 authentication psk policy AES SHA 5!ipsec esp-policy 1 policy AES SHA 5!ipsec peer b0-bb-8b-00-e7-a8 local-net 10.1.168.1/32 remote-id b0-bb-8b-00-e7-a8 remote-ip any remote-net 10.1.168.2/32 policy ike 1 esp 1 psk xxx!ipsec peer b0-bb-8b-00-ea-20 local-net 10.1.168.1/32 remote-id b0-bb-8b-00-ea-20 remote-ip any remote-net 10.1.168.3/32 policy ike 1 esp 1 psk xxx!router bgp 65051 bgp timer 5 15 neighbor 0168_RansNet_SSL2IPSEC_1 as-peer neighbor 0168_RansNet_SSL2IPSEC_1 as-remote 65051 neighbor 0168_RansNet_SSL2IPSEC_1 next-hop-self neighbor 0168_RansNet_SSL2IPSEC_1 route-reflector-client neighbor 0168_RansNet_SSL2IPSEC_1 soft-reconfiguration neighbor 0168_RansNet_SSL2IPSEC_1 weight 0 neighbor range 10.1.172.0/22 as-peer 0168_RansNet_SSL2IPSEC_1 address-family-l2vpn advertise-all-vni neighbor 0168_RansNet_SSL2IPSEC_1 activate neighbor 0168_RansNet_SSL2IPSEC_1 route-reflector-client neighbor 0168_RansNet_SSL2IPSEC_1 soft-reconfiguration!

Below is a CLI snip for the branch

!interface gre1 tunnel local 10.1.168.2 remote 10.1.168.1 enable ip address 10.1.172.2/22!interface lo enable ip address 10.1.168.2/32!interface vxlan1 description "Auto Interface from VPN (1)" vx-local 10.1.172.2 enable bridge-group 1!interface vlan 1 1 description "Default VLAN for all LAN ports" enable bridge-group 1!interface bridge br1 description "Auto Interface from IPSec VPN (1)" bridge enable!ipsec ike-policy 1 authentication psk policy AES SHA 5!ipsec esp-policy 1 policy AES SHA 5!ipsec peer 10.18.18.194 local-id b0-bb-8b-00-e7-a8 local-net 10.1.168.2/32 remote-net 10.1.168.1/32 policy ike 1 esp 1 psk xxx!router bgp 65051 bgp timer 5 15 neighbor 0168_RansNet_SSL2IPSEC_1 as-peer neighbor 0168_RansNet_SSL2IPSEC_1 as-remote 65051 neighbor 0168_RansNet_SSL2IPSEC_1 next-hop-self neighbor 0168_RansNet_SSL2IPSEC_1 soft-reconfiguration neighbor 0168_RansNet_SSL2IPSEC_1 weight 0 neighbor 0168_RansNet_SSL2IPSEC_11 as-peer neighbor 0168_RansNet_SSL2IPSEC_11 as-remote 65051 neighbor 0168_RansNet_SSL2IPSEC_11 next-hop-self neighbor 0168_RansNet_SSL2IPSEC_11 soft-reconfiguration neighbor 0168_RansNet_SSL2IPSEC_11 weight 0 neighbor 10.1.172.1 as-peer 0168_RansNet_SSL2IPSEC_1 address-family-l2vpn advertise-all-vni neighbor 0168_RansNet_SSL2IPSEC_1 activate neighbor 0168_RansNet_SSL2IPSEC_1 route-reflector-client neighbor 0168_RansNet_SSL2IPSEC_1 soft-reconfiguration!