Maximizing Network Bandwidth: How Port Channels and vPCs Enhance Performance

Spanning Tree Protocol (STP) is a double-edged sword. While it prevents network loops, it can also become a bottleneck for your data center's performance. On the last blog, we discussed how STP creates a loop-free topology in redundant networks. We saw how STP aids redundancy by actively reacting to failed links and paths, ensuring that backup routes are used when necessary.

However, STP, while preventing loops, can limit bandwidth by blocking redundant links. STP only utilizes these redundant links if the primary path fails. This means that despite having multiple physical paths, only one path actively forwards traffic. While STP provides redundancy, it doesn't address the need for increased bandwidth.

While modern networks can utilize Per-VLAN Spanning Tree (PVST+) to create separate spanning-tree instances for different VLANs, this approach might not fully utilize the network's capabilities, as some VLANs might have significantly more traffic than others. To fully unleash the network's potential, we need to explore alternative technologies.

How do Port Channels address STP's limitations?

Port channels address this limitation. With classic port channeling technology, bundles of point-to-point links between two devices can be logically combined into one huge link. This helps optimize bandwidth usage, improve network convergence, avoid spanning tree blocking, and still provide resilience against physical link failures.

With port channeling, STP views the links as just one link and allows all links to be used at full capacity. Port channels usually have built-in algorithms that help load balance traffic between the physical ports based on multiple criteria like destination or source MAC address.

What is LACP and why is it important for Port Channels?

By default, Cisco operates port channels in static mode where both ends of the network need to be statically set to "on" to form a port channel. However, there is an industry protocol that helps negotiate port channel formation. This protocol is known as LACP (Link Aggregation Control Protocol) - IEEE 802.1AX.

LACP negotiates parameters such as link speed, duplex setting, and VLAN information before connecting. It ensures these variables are consistent. If there is a mismatch in these variables and static port channel configuration is used, the port channel might perform suboptimally or not connect properly.

How do the LACP modes work?

LACP has two modes:

• Active Mode
• Passive Mode

An interface in LACP active mode actively initiates LACP negotiations with the interface at the other end of the link, while an interface in passive mode only responds to negotiation request without ever initiating one

When using LACP to configure a port channel, at least one of the ends has to be set to Active mode, and with statically configured port channels, both ends have to be set to "on." It is good to know that LACP is disabled by default.

How does load balancing work with Port Channels?

When physical links are bundled into a single logical link, the system uses load balancing algorithms to distribute traffic. Users can choose from various algorithms, with traffic load balanced based on source/destination MAC addresses, IP addresses, or a combination of these.

For example, if a port channel decides to load balance by destination IP, all traffic destined for networksbymuna.com would go through the same physical link and all traffic destined for another website would go through another physical link thereby spreading out the traffic among physical links

Based on the illustration above every packet destined for the storage server traversed a single link, and every packet destined for the file server traversed a different link

Based on the illustration above every packet originating from host 1 used a link while every packet originating from host 2 used another link. 

What are Virtual Port Channels (vPC), and what advantages do they offer?

Regular port channels are limited to point-to-point connections. Virtual Port Channels (vPCs) were introduced to overcome this constraint, allowing a single device to form a port channel across two upstream devices.

vPCs provide several advantages:

• Use all available uplink bandwidth.
• Maintain a loop-free topology.
• Eliminate STP blocked ports.
• Offer higher availability by protecting against both link and device failures (compared to classic port channeling, which only protects against link failures).

How do vPCs work?

With vPC, the two upstream devices are treated as a single logical switch, even though they remain physically separate and have independent control planes. This logical entity is called a vPC domain. A vPC domain can only have two devices as peers, and a vPC switch can only belong to one vPC domain. However, a vPC domain can have multiple port channel connections to downstream devices.

This "single switch" illusion is achieved through constant communication and synchronization between the vPC peer devices. This communication occurs over a dedicated link called the vPC peer link, which is itself a port channel that carries control traffic like BPDUs and LACP packets. vPC peers can also connect to non-vPC devices through ports known as orphan ports. The vPC peer link typically only carries control traffic and data from orphan ports or flooded traffic.

The communication between vPC peers is managed by Cisco Fabric Services (CFS). CFS uses CFS Over Ethernet (CFSOE) Protocol Data Units to exchange state information. For instance, all MAC addresses learned by each peer switch are synchronized through CFSOE.

Additionally, vPCs have an out-of-band connection called the vPC peer keepalive link, which is a Layer 3 link used to verify the availability and health of the vPC peers by quickly detecting if one peer becomes unavailable

CFSOE handles various control plane functions, including:

• MAC address table synchronization
• ARP table synchronization
• STP management
• IGMP snooping
• Configuration consistency

This synchronization ensures a loop-free environment. vPCs also employ data plane forwarding rules to prevent loops. The most critical rule is that a packet entering a vPC peer switch through the peer link cannot exit through a vPC member port. It can only exit through a different Layer 3 link or an orphan port. This prevents traffic loops between the vPC peers.

Within a vPC domain, there is a primary and secondary switch, determined through an election process. The primary switch takes the lead in generating and processing control plane information.

It's important to note that vPC operates at Layer 2. Therefore, it cannot be used to create Layer 3 port channels

For details on configuring vPC check out these links by Cisco

• Configure and Understand vPC in nexus 9000
• Best Practices for Configuring VPC on Cisco Nexus Switches

Conclusion

In this blog post, we explored the limitations of Spanning Tree Protocol (STP) in modern data center networks and introduced port channels and Virtual Port Channels (vPCs) as powerful solutions for maximizing bandwidth and achieving high availability. By aggregating multiple physical links into a single logical connection, these technologies overcome the constraints of STP, allowing you to unlock the full potential of your network infrastructure. Remember that careful planning and configuration are essential for successful implementation. By considering factors such as link speed, LACP modes, and load balancing algorithms, you can ensure optimal performance and resilience for your critical data center operations.