Fortigate ECMP with BGP
It's like clock work...around this time the seasons are changing..autumn colors are out, and the colder air reminds us that things are changing around here, embrace it! One thing that I've been working with is AWS Transit gateways, a common theme this year is all cloud! AWS Transit Gateways make it easier to move towards that!
AWS Transit gateways relatively new, (couple years old now) so instead of attaching to each AWS VPC environment independently we can connect to the transit gateway and attach VPCs to the gateway. If you enable BGP your routes will propagate. I've been working with ECMP, specifically with Fortigates.
There are two ways to have redundancy in a network when you have multiple paths to a destination. Similar routes that have a higher cost/weight then ones that don't are only used when there is a problem with primary path. So you have a backup route on standby until the primary route fails. This could be a good option if the backup route isn't the same bandwidth/media or really not equal to the primary link which is why different cost/weight is important to have.
However if you have multiple paths that were equal bandwidth/media we could/would want to use both of them at the same time so that traffic is load balanced across the network, this is called ECMP (Equal-cost multi-path routing)
Let's walk though the steps on how to setup ECMP on a Fortigate using BGP. I'm already guessing you have an AWS account with an AWS transit gateway deployed that supports ECMP, as well as configured your customer gateway in AWS with BGP Autonomous system (AS) number.
In this example I have a Fortigate Firewall running version 6.4.2, I have set the BGP "AS" number to 65000 and the AWS "AS" number is 64512.
Amazon gives you a text file to download the configuration for these two tunnels but depending on your Fortigate setup we may have to tweak the file, (not a good idea to just paste it) In this example here is what I have for tunnel one:
1# Configure Phase 1 of the tunnel, aka IKE
2config vpn ipsec phase1-interface
3edit "AWS-0615bcc6f-1"
4set dpd on-idle
5
6# Outside Interface to reach AWS
7set interface outside
8
9# External IP address using (RFC 5737) in this example
10set local-gw 198.51.100.1
11set dhgrp 2
12set proposal aes128-sha1
13set keylife 28800
14
15# Remote Peer this is AWS Transit, using (RFC 5737) in example`
16set remote-gw 192.0.2.124
17set psksecret lewiryan.github.io/ciscoskills
18set dpd-retryinterval 10
19next
20end
21
22# Configure Phase 2 of the Tunnel, aka IPSEC
23config vpn ipsec phase2-interface
24edit "AWS-0615bcc6f-1"
25set phase1name "AWS-0615bcc6f-1"
26set proposal aes128-sha1
27set dhgrp 2
28set pfs enable
29set keylifeseconds 3600
30end
31
32# Build the tunnel interface
33config system interface
34edit "AWS-0615bcc6f-1"
35set vdom "root"
36set ip 169.254.150.142 255.255.255.255
37set allowaccess ping
38set type tunnel
39set tcp-mss 1379
40set remote-ip 169.254.150.141/32
41set mtu enable
42set mtu 1427
43set interface "outside"
44next
45end
After the tunnel one is created, AWS gives us configuration for the second tunnel, basically the same configuration with some differences, here is tunnel two.
1# Configure Phase 1 of the tunnel, aka IKE
2
3config vpn ipsec phase1-interface
4edit "AWS-0615bcc6f-2"
5set dpd on-idle
6# Outside Interface to reach AWS
7set interface outside
8# External IP address using (RFC 5737) in this example
9set local-gw 198.51.100.1
10set dhgrp 2
11set proposal aes128-sha1
12set keylife 28800
13# Remote Peer this is AWS Transit, using (RFC 5737) in example
14set remote-gw 203.0.113.25
15set psksecret ciscoskills.com
16set dpd-retryinterval 10
17next
18end
19
20# Configure Phase 2 of the Tunnel, aka IPSEC
21
22config vpn ipsec phase2-interface
23edit "AWS-0615bcc6f-2"
24set phase1name "AWS-0615bcc6f-2"
25set proposal aes128-sha1
26set dhgrp 2
27set pfs enable
28set keylifeseconds 3600
29end
30
31# Build the tunnel interface
32config system interface
33edit "AWS-0615bcc6f-2"
34set vdom "root"
35set ip 169.254.157.90 255.255.255.255
36set allowaccess ping
37set type tunnel
38set tcp-mss 1379
39set remote-ip 169.254.157.89/32
40set mtu enable
41set mtu 1427
42set interface "outside"
43next
44end
We now have two tunnels configured, these won't show online until the Fortigate has a policy attached to them. So you could put firewall rules on each tunnel interface or better yet is to create a zone within the Fortigate and add the two tunnel interfaces into same zone. By creating a zone within Fortigate the same firewall rules apply to each interface instead of creating/copying rules to each tunnel interface. (This would be an example)
1config system zone
2edit AWS-0615bcc6f
3set interface AWS-0615bcc6f-1
4set interface AWS-0615bcc6f-2
5set intrazone allow
6next
7end
Once those interfaces are either in a zone and in a Firewall policy you should be able to bring those tunnel interfaces online. The final thing is configure our Fortigate with BGP so that when you attach a VPC to the transit gateway that IP space gets advertised automatically. We also want to enable ECMP with BGP.
1# Enable BGP on the Fortigate
2
3config router bgp
4# This is the number we set in AWS, so it should match what TG has
5set as 65000
6set router-id `198.51.100.1`
7
8# Since we are using "AS" Numbers, we tell the Fortigate to use
9# External Multipath with out it BGP will use one route at time
10# Active/Passive
11
12set ebgp-multipath enable
13config neighbor
14edit "169.254.157.89"
15set remote-as 64512
16next
17edit "169.254.150.141"
18set remote-as 64512
19next
20end
If we attached a VPC to the transit gateway, and in this example I attached a VPC range of 10.0.0.0/16 to the transit gateway. That IP range would show up in the Fortigate, if we look at the routing table via get router info routing-table all
1FW01 # get router info routing-table all
2Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
3O - OSPF, IA - OSPF inter area
4N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
5E1 - OSPF external type 1, E2 - OSPF external type 2
6i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
7- candidate default
8
9Routing table for VRF=0
10S* 0.0.0.0/0 [5/0] via `198.51.100.`25, ppp1
11B 10.0.0.0/16 [20/100] via 169.254.150.141, AWS-0615bcc6f-1, 03:07:57
12 [20/100] via 169.254.157.89, AWS-0615bcc6f-2, 03:07:57
13C `198.51.100.1`/32 is directly connected, ppp1
14C 169.254.150.141/32 is directly connected, AWS-0615bcc6f-1
15C 169.254.150.142/32 is directly connected, AWS-0615bcc6f-1
16C 169.254.157.89/32 is directly connected, AWS-0615bcc6f-2
17C 169.254.157.90/32 is directly connected, AWS-0615bcc6f-2
We can see that 10.0.0.0/16 is available to us via the VPN interfaces, we have equal paths as well, traffic will load balance between each tunnel interface. BGP with ECMP is alive and active! ... That's all I got for this post, like always I hope this information is helpful, got questions? Comment below!