Oh...Spanning Tree
It's the "S" word we don't like to hear, and often times it has degraded networks or even took them down entirely. (All Systems Down - an older but classic story) It's also one of those things that nobody really likes, we even had network companies, as well as IEEE try to replace it with something else. Remember the names like TRILL, and IEEE 802.1aq (SPB) or Cisco's FabicPath technologies? You had all of these different flavors that showed a lot progress, but they never really took off for one reason or another. The funny part is this protocol we are all trying kill off is still alive and doing well, so let's look at spanning tree in the meantime because you will likely run into it.
Remember there are multiple types of spanning tree protocols but they all have a common theme to them and in this post we are going to be focusing on those common themes. If you need a refresher about the reasons and basic functions of spanning tree you can check out these posts I did quite a while ago.
- The Beginnings of Layer Two Redundancy
- The Middle of Layer Two Redundancy
- The “Ending” of Layer Two Redundancy
So we should already know that by default spanning tree is enabled on all ports when the switch has blank config. There are of course tweaks we can do to adjust convergence times like using RSTP, if thirty seconds is too long, using portfast when there is not another switch downstream, lots of customization. Here is a simple topology, two switches two links, one of links will be blocked otherwise expect loops and network inconsistency.
Out of the box STP will protect you from loops, and a simple way to figure what type of spanning tree protocol is running as well as to figure out who is root you can use this command:
1SW2#show spanning-tree
2
3VLAN0001
4 Spanning tree enabled protocol rstp
5 Root ID Priority 32769
6 Address aabb.cc00.0200
7 This bridge is the root
8 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
9
10 Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
11 Address aabb.cc00.0200
12 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
13 Aging Time 300 sec
14
15Interface Role Sts Cost Prio.Nbr Type
16------------------- ---- --- --------- -------- ----------------------
17Et0/0 Desg FWD 100 128.1 P2p
18Et0/1 Desg FWD 100 128.2 P2p
19Et0/2 Desg FWD 100 128.3 P2p
20Et0/3 Desg FWD 100 128.4 P2p
We can see that we are on the root bridge, by looking at Root ID information. This shows information about who is root and in this case we are on the root switch, seeing the message This bridge is the root, confirms that. We can also see the type of spanning tree protocol we are running, which is RSTP. Running the command on the other switch, that is not root we can the BLK status on one of its ports (Et0/1) which is what we are expecting. We can also see who is root. If we look at the Bridge ID that's the local switch we are on. Notice when you are the root switch both the Root ID and Bridge match. So our topology looks like something like this:
1SW1#show spanning-tree
2
3VLAN0001
4 Spanning tree enabled protocol rstp
5 Root ID Priority 32769
6 Address aabb.cc00.0200
7 Cost 100
8 Port 1 (Ethernet0/0)
9 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
10
11 Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
12 Address aabb.cc00.0300
13 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
14 Aging Time 300 sec
15
16Interface Role Sts Cost Prio.Nbr Type
17------------------- ---- --- --------- -------- ----------------------
18Et0/0 Root FWD 100 128.1 P2p
19Et0/1 Altn BLK 100 128.2 P2p
20Et0/2 Desg FWD 100 128.3 P2p
21Et0/3 Desg FWD 100 128.4 P2p
So by default spanning tree helped us block a redundant path without us really doing anything. So why bother with it? Well there is always fine tuning like turning on portfast which is used for edge devices, timers for STP to make it converge faster. There are also some protections that should be enabled like using BPDU guard which help guard the spanning tree topology from introducing unauthorized devices from sending BPDUs and usually used on access ports. Another one is Root guard which is used to help protect the root bridge from changing. Root guard is similar to BPDU guard but I like to think this protection is for us network engineers for when we introduce a switch into the network that somehow as a lower priority than the existing root bridge we can protect the network from changing the STP topology.
Failures
Am I doing a good job of making this protocol shine? Well like everything there is always another side of it and I would say some of the failures can be a misconfiguration, software incompatibilities/bugs (which is more common than you would think) and overgrowth of the STP design. When this happens the STP topology can become disruptive to the entire layer two design while trying to reconverge into a new shape, meanwhile switch CPUs hit all time highs and network availability hits all zeros. Another valid point is its a waste of money, you buy two of everything but you can't use both at the same time because you can't have multiple L2 paths. Sure you can add MLAGs/vPCs but spanning tree is still running and if that protocol becomes unstable, or changes the topology it has the potential to still be disruptive. ...and this is why a lot people dislike this technology, It really wasn't designed for high-availability, multi-path, layer two domains.
Move up a layer?
Traditionally L2/L3 boundary is at the distribution/aggregation layer of the network, but this traditional design is changing to IP routing at the access layer (Routed Access Layer) to remove spanning tree However this may work well for client/end-user access, this type of design fails at the data center. You may have racks of servers in your data center, you want the maximum amount a flexibility to move hardware around as well as add additional hardware in the future. If you did a routed access layer at the data center, you start to lock yourself in and although its 2019 a lot applications still operate at a layer two level and even if the apps you are running don't have that problem you run into scalability issues.
Smoke and Mirrors?
So where's the magic? How do you use a reliable layer three network like you did in on the client/end-user side but put it in a data center? Think network overlays like VxLAN or BGP EVPN, (EVPN BGP), (order does not matter, they're the same thing) 😉 Basically these are network overlay technologies, you have you underlayer network that is running and you put this technology on top of that network, I call it network virtualization but you may have also heard SDN, (Software Define Networking). Whatever you want to call it this is how you kill spanning tree. Spanning tree by default just runs on a layer two network, we saw that from the beginning. So you remove layer two from the entire stack and then overlay an additional network on top of that. Using these type of technologies your applications don't know the difference. Another helpful post I saw when reading about overlay networks was from David Varnum's post about how Arista Networks uses BGP EVPN, pretty neat technology, if interested.