SD-WAN with VPN bonding & PBR
In this demo, we will combine senarios of VPN bonding (using OSPF) and Policy-Based Routing (PBR).
VPN bonding (with OSPF) routes packets on a per packet basis across multiple tunnels, packets are routed by routing table with equal paths generated by OSPF (see VPN bonding with OSPF), and PBR supersedes routing table to route packets based on specific source/destination/applications (see PBR detail).
In this demo setup
eth0 of HQ and remote CMG connect to Internet (VPN tunnel tap1 goes through Internet)
eth1 of HQ and remote CMG connect to MPLS (VPN tunnel tap2 goes through MPLS)
eth2 of HQ connect to hosting services (web & ftp)
eth2 of remote connect to user LAN
Requirements (traffic flow between remote and HQ)
All web traffic (tcp/80/443) must route through MPLS (via tap2)
All ftp traffic (tcp/20/21) must route through Internet (via tap1)
All other traffic between remote and HQ must be load balanced
VPN bonding (OSPF) with PBR
Configuration notes:
use "firewall-set" to mark target traffic, on the inbound of eth2 (NOTE: for PBR, we need to mark traffic on the inbound direction the of interface closest to source hosts).
configure PBR on remote CMG for outgoing traffic to HQ (mark destination port)
Configure PBR on HQ CMG for returning traffic to remote (mark source port)
(see attached complete config for both HQ CMG and remote CMG).
Show outputs on remote CMG
BRANCH# show ip routeCodes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, A - Babel, > - selected route, * - FIB route
S>* 0.0.0.0/0 [1/0] via 121.58.189.1, eth0C>* 10.10.10.0/30 is directly connected, eth1O 11.11.11.0/24 [110/10] is directly connected, tap1, 00:03:30C>* 11.11.11.0/24 is directly connected, tap1O 12.12.12.0/24 [110/10] is directly connected, tap2, 00:03:30C>* 12.12.12.0/24 is directly connected, tap2O>* 30.1.1.0/24 [110/20] via 11.11.11.1, tap1, 00:03:00 <--equal paths learnt from OSPF makes traffic load balancing * via 12.12.12.1, tap2, 00:03:00C>* 121.58.189.0/25 is directly connected, eth0C>* 127.0.0.0/8 is directly connected, loC>* 172.16.10.0/24 is directly connected, eth2MANDIRI-BRANCH#
BRANCH# show firewall set-list Chain PREROUTING (policy ACCEPT 45 packets, 2763 bytes) pkts bytes target prot opt in out source destination 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 20 */ tcp dpt:20 MARK set 0x14 <--mark traffic to server (dport) at inbound direction of host side, interface eth2 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 21 */ tcp dpt:21 MARK set 0x14 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 80 */ tcp dpt:80 MARK set 0x50 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 81 */ tcp dpt:443 MARK set 0x50
MANDIRI-BRANCH# show ip pbr policy 20: from all fwmark 0x14 lookup 20 80: from all fwmark 0x50 lookup 80
MANDIRI-BRANCH# show ip pbr route 20default via 11.11.11.1 dev tap1 <--path for ftp goes through tap1/InternetMANDIRI-BRANCH# show ip pbr route 80default via 12.12.12.1 dev tap2 <--path for ftp goes through tap2/MPLS
Show outputs on HQ CMG
HQ# show ip routeCodes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, A - Babel, > - selected route, * - FIB route
S>* 0.0.0.0/0 [1/0] via 121.58.189.1, eth0C>* 10.10.10.0/30 is directly connected, eth1O 11.11.11.0/24 [110/10] is directly connected, tap1, 00:11:52C>* 11.11.11.0/24 is directly connected, tap1O 12.12.12.0/24 [110/10] is directly connected, tap2, 00:11:52C>* 12.12.12.0/24 is directly connected, tap2O 30.1.1.0/24 [110/10] is directly connected, eth2, 00:11:51C>* 30.1.1.0/24 is directly connected, eth2C>* 121.58.189.0/25 is directly connected, eth0C>* 127.0.0.0/8 is directly connected, loO>* 172.16.10.0/24 [110/20] via 11.11.11.2, tap1, 00:09:25 <--equal paths for remote branch network * via 12.12.12.2, tap2, 00:09:25HQ#
MANDIRI-HQ# show firewall set-list Chain PREROUTING (policy ACCEPT 4817 packets, 497K bytes) pkts bytes target prot opt in out source destination 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 20 */ tcp spt:20 MARK set 0x14 <--mark return traffic from server (sport) at inbound direction of server side, interface eth2 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 21 */ tcp spt:21 MARK set 0x14 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 80 */ tcp spt:80 MARK set 0x50 0 0 MARK tcp -- eth2 * 0.0.0.0/0 0.0.0.0/0 /* set-list 81 */ tcp spt:443 MARK set 0x50
MANDIRI-HQ# show ip pbr policy 20: from all fwmark 0x14 lookup 20 80: from all fwmark 0x50 lookup 80 MANDIRI-HQ# show ip pbr route 20default via 11.11.11.2 dev tap1 <--return traffic from ftp server goes through tap1/InternetMANDIRI-HQ# show ip pbr route 80default via 12.12.12.2 dev tap2 <--return traffic from web server goes through tap1/InternetMANDIRI-HQ#
When problem happens, use tcpdump to verify packets expected to arrive at each interface.