- Designing High Availability in the Enterprise Campus
- Designing an Optimum Design for Layer 2
- Developing an Optimum Design for Layer 3
- Supporting Infrastructure Services
- Summary
- References
- Review Questions
This chapter is from the book
This chapter is from the book
This chapter is from the book
Designing an Optimum Design for Layer 2
Layer 2 architectures rely on the following technologies to create a highly available, deterministic topology: Spanning Tree Protocol (STP), trunking (ISL/802.1q), Unidirectional Link Detection (UDLD), and EtherChannel.
The following section reviews design models and recommended practices for Layer 2 high availability and optimum convergence of the Cisco Enterprise Campus Infrastructure.
Recommended Practices for Spanning-Tree Configuration
For the most deterministic and highly available network topology, the requirement to support STP convergence should be avoided by design. You may need to implement STP for several reasons:
- When a VLAN spans access layer switches to support business applications.
- To protect against user-side loops. Even if the recommended design does not depend on STP to resolve link or node failure events, STP is required to protect against user-side loops. There are many ways that a loop can be introduced on the user-facing access layer ports. Wiring mistakes, misconfigured end stations, or malicious users can create a loop. STP is required to ensure a loop-free topology and to protect the rest of the network from problems created in the access layer.
- To support data center applications on a server farm.
If you need to implement STP, use Rapid Per-VLAN Spanning-Tree Plus (RPVST+). You should also take advantage of the Cisco enhancements to STP known as the Cisco STP toolkit.
Cisco STP Toolkit
The Cisco enhancements to STP include the following. (Note that the enhancements marked with an* are also supported with Rapid Per-VLAN Spanning-Tree Plus [RPVST+].)
- PortFast*: Causes a Layer 2 LAN interface configured as an access port to enter the forwarding state immediately, bypassing the listening and learning states. Use PortFast only when connecting a single end station to a Layer 2 access port.
- UplinkFast: Provides from three to five seconds convergence after a direct link failure and achieves load balancing between redundant Layer 2 links using uplink groups.
- BackboneFast: Cuts convergence time by max_age for indirect failure. BackboneFast is initiated when a root port or blocked port on a network device receives inferior bridge protocol data units (BPDU) from its designated bridge.
- Loop guard*: Prevents an alternate or root port from becoming designated in the absence of BPDUs. Loop guard helps prevent bridging loops that could occur because of a unidirectional link failure on a point-to-point link.
- Root guard*: Secures the root on a specific switch by preventing external switches from becoming roots.
- BPDU guard*: When configured on a PortFast-enabled port, BPDU guard shuts down the port that receives a BPDU.
- Unidirectional Link Detection (UDLD): UDLD monitors the physical configuration of fiber-optic and copper connections and detects when a one-way connection exists. When a unidirectional link is detected, the interface is shut down and the system alerted.
STP Standards and Features
STP enables the network to deterministically block interfaces and provide a loop-free topology in a network with redundant links. There are several varieties of STP:
- STP is the original IEEE 802.1D version (802.1D-1998) that provides a loop-free topology in a network with redundant links.
- Common Spanning Tree (CST) assumes one spanning-tree instance for the entire bridged network, regardless of the number of VLANs.
- Per VLAN Spanning Tree Plus (PVST+) is a Cisco enhancement of STP that provides a separate 802.1D spanning tree instance for each VLAN configured in the network. The separate instance supports PortFast, UplinkFast, BackboneFast, BPDU guard, BPDU filter, root guard, and loop guard.
- The 802.1D-2004 version is an updated version of the STP standard.
- Multiple Spanning Tree (MST) is an IEEE standard inspired from the earlier Cisco proprietary Multi-Instance Spanning Tree Protocol (MISTP) implementation. MST maps multiple VLANs into the same spanning-tree instance. The Cisco implementation of MSTP is MST, which provides up to 16 instances of Rapid Spanning Tree Protocol (RSTP, 802.1w) and combines many VLANs with the same physical and logical topology into a common RSTP instance. Each instance supports PortFast, UplinkFast, BackboneFast, BPDU guard, BPDU filter, root guard, and loop guard.
- RSTP, or IEEE 802.1w, is an evolution of STP that provides faster convergence of STP.
- Rapid PVST+ (RPVST+) is a Cisco enhancement of RSTP that uses PVST+. It provides a separate instance of 802.1w per VLAN. The separate instance supports PortFast, UplinkFast, BackboneFast, BPDU guard, BPDU filter, root guard, and loop guard.
STP Standards and Features
To configure a VLAN instance to become the root bridge, enter the spanning-tree vlan vlan_ID root primary command to modify the bridge priority from the default value (32768) to a significantly lower value. The bridge priority for the specified VLAN is set to 8192 if this value will cause the switch to become the root for the VLAN. If any bridge for the VLAN has a priority lower than 8192, the switch sets the priority to one less than the lowest bridge priority Manually placing the primary and secondary bridges along with enabling STP toolkit options enables you to support a deterministic configuration where you know which ports should be forwarding and which ports should be blocking.
Figure 2-9 illustrates recommended placements for STP toolkit features:
- Loop guard is implemented on the Layer 2 ports between distribution switches, and on the uplink ports from the access switches to the distribution switches.
- Root guard is configured on the distribution switch ports facing the access switches.
- UplinkFast is implemented on the uplink ports from the access switches to the distribution switches.
Figure 2-9 Layer 2 Hardening
- BPDU guard or root guard is configured on ports from the access switches to the end devices, as is PortFast.
- The UDLD protocol allows devices to monitor the physical configuration of the cables and detect when a unidirectional link exists. When a unidirectional link is detected, UDLD shuts down the affected LAN port. UDLD is often configured on ports linking switches.
- Depending on the security requirements of an organization, the port security feature can be used to restrict a port's ingress traffic by limiting the MAC addresses allowed to send traffic into the port.
)
Recommended Practices for Trunk Configuration
A trunk is a point-to-point link between two networking devices that carry the traffic of multiple VLANs. Trunks are typically deployed on the interconnection between the access and distribution layers.
The current recommended practice is to use IEEE 802.1Q trunks. Cisco extensions to 802.1Q avoid security concerns related to the 802.1Q nontagged, native VLAN. The native VLAN is assigned to an unused ID, or the tagged, native VLAN option is used to avoid VLAN hopping.
VLAN Trunking Protocol (VTP) is a protocol that enables network managers to centrally manage the VLAN database. VTP transparent mode is now a recommended practice because it decreases the potential for operational error.
By default, Cisco switches are configured as a VTP server with no VTP domain name specified.
Therefore, it is also recommended when configuring switches along with setting the mode to Transparent, to set the VTP domain name. This is important if you are connecting your switch to other domains, such as a service provider switch. Misconfiguration of the switch as a server or client with no VTP domain name will cause it to accept the domain name of an adjacent VTP server and overwrite the local VLAN database.
As a recommended practice, when configuring switch-to-switch interconnections to carry multiple VLANs, set Dynamic Trunking Protocol (DTP) to Desirable and Desirable with Encapsulation Negotiate to support DTP negotiation.
Another recommended practice is to manually prune unused VLANs from trunked interfaces to avoid broadcast propagation. You should avoid automatic VLAN pruning.
The final recommendation for trunk configuration is to disable trunks on host ports, because host devices will not need to negotiate trunk status. This practice speeds up PortFast and is also a VLAN-hopping security measure.
VLAN Trunking Protocol
VTP version 3 supports centralized VLAN administration in a switched network. VTP runs only on trunks and provides the following four modes:
- Server: Updates clients and servers. The VTP server switch propagates the VTP database to VTP client switches.
- Client: Receives updates but cannot make changes.
- Transparent: Does not participate in the VTP domain. Lets updates pass through.
- Off: Ignores VTP updates.
With VTP, when you configure a new VLAN on a switch in VTP server mode, the VLAN is distributed through all switches in the VTP domain. This redistribution reduces the need to configure the same VLAN everywhere.
With hierarchical networks that do not span VLANs across the distribution layer, there is little need for a shared common VLAN database. In the recommended campus design, the same VLAN should not appear in two access layer switches. Adding and removing VLANs is generally not a frequent network management practice. In most cases, VLANs are defined once during switch setup with few, if any, additional modifications to the VLAN database in an access layer switch. The benefits of dynamic propagation of VLAN information across the network are not worth the potential for unexpected behavior due to operational error. For these reasons, VTP transparent mode is the recommended configuration option.
Dynamic Trunking Protocol
DTP provides switch ports to negotiate the trunking method with another device and to automatically allow a link to become a trunk.
With Cisco devices, there are five Layer 2 port modes:
- Trunk: Puts the port into permanent trunking mode and negotiates to convert the link into a trunk link. The port becomes a trunk port even if the neighboring port does not agree to the change.
- Desirable: Actively attempts to form a trunk, subject to neighbor agreement. The port becomes a trunk port if the neighboring port is set to On, Desirable, or Auto mode.
- Auto: Makes the port willing to convert the link to a trunk link. The port becomes a trunk port if the neighboring port is set to on or desirable mode.
- Access: This is the access mode in Cisco IOS Software that specifies that the port never become a trunk, even if the neighbor tries. This mode puts the LAN port into permanent nontrunking mode and negotiates to convert the link into a nontrunking link.
- Nonnegotiate: Prevents the port from generating DTP frames. You must configure the neighboring port manually as a trunk port to establish a trunk link.
With Cisco devices, there are three Ethernet trunk encapsulation types:
- ISL: Uses Inter-Switch Link (ISL) encapsulation on the trunk link
- Dot1q: Uses 802.1Q encapsulation on the trunk link
- Negotiate: Specifies that the LAN port negotiate with the neighboring LAN port to become an ISL (preferred) or 802.1Q trunk, depending on the configuration and capabilities of the neighboring LAN port
The trunking mode, the trunk encapsulation type, and the hardware capabilities of the two connected LAN ports determine whether a link becomes an ISL or 802.1Q trunk. A common practice is to configure both ends of the trunk to desirable. This has the operational benefit of providing a clear indication of a functional trunking connection with show commands, and is the general recommendation for DTP trunking.
An alternative practice is to set one side of the link (typically the access layer) to Auto and the other end (typically the distribution layer) to Desirable. This setting allows for automatic trunk formation, with DTP running on the interconnection to protect against some rare hardware failure scenarios and software misconfigurations.
For fastest convergence, a third configuration turns DTP to On and On with Nonnegotiate to save a few seconds of outage when restoring a failed link or node. However, DTP is not actively monitoring the state of the trunk with this configuration, and a misconfigured trunk is not easily identified. The Nonnegotiate setting can also cause loss of connectivity if the process is not performed in the correct order and there is no out-of-band connectivity to the farthest switch from where the in-band modifications are being made.
Recommended Practices for UDLD Configuration
UDLD enables devices to monitor the physical configuration of the cables and detect when a unidirectional link exists where bidirectional communication has not been established.
UDLD is typically deployed on fiber topologies where physical misconnections can occur that enable a link to appear to be up/up when there is a mismatched set of transmit/receive pairs. UDLD supports both fiber-optic and copper Ethernet cables connected to LAN ports.
Each switch port configured for UDLD will send UDLD protocol hello packets at Layer 2 containing the device and port identifications of the port, and the device and port identifications of the neighbor as seen by UDLD on that port. Neighboring ports should see their own device and port identifications in the packets received from the other side. If the port does not see its own device and port identifications in the incoming UDLD packets for a specific duration of time, the link is considered unidirectional and is shut down. The default 15-second hello timers are the same for normal and aggressive UDLD. In normal mode, UDLD will error-disable only the end where the UDLD is detected; aggressive mode will error-disable both ends of a connection after aging on a previously bidirectional link in eight seconds.
A recommended practice is to enable UDLD aggressive mode in all environments where fiber-optic interconnections are used. UDLD is enabled globally on all fiber-optic LAN ports with the Cisco IOS software udld {enable | aggressive} command. UDLD is enabled on individual LAN ports with the udld port [aggressive] interface command.
Recommended Practices for EtherChannel
An EtherChannel bundles individual Ethernet links into a single logical link that provides the aggregate bandwidth of up to eight physical links.
EtherChannels are typically deployed between the distribution-to-core and core-to-core interconnections where increased availability and scaled bandwidth are required. EtherChannel link aggregation is used to provide link redundancy and prevent a single point of failure, and to reduce peering complexity because the single logical entity reduces the number of Layer 3 neighbor relationships as compared to multiple parallel links.
EtherChannels create channels containing up to eight parallel links between switches. If the channels are on interfaces that are on different physical line cards, there is increased availability because the failure of a single line card does not cause a complete loss of connectivity.
There are two variants for the control mechanism for EtherChannel: the prestandard Cisco implementation that uses Port Aggregation Protocol (PAgP), and the IEEE 802.3ad standards-based implementation that uses Link Aggregation Control Protocol (LACP). PAgP and LACP do not interoperate. You can manually configure a switch with PAgP on one side and LACP on the other side in the on/on mode. When this is done, ports configured in the on mode do not negotiate, and therefore there is no negotiation traffic between the ports. This configuration results in the EtherChannel being hard-coded.
When connecting a Cisco IOS Software device to a Catalyst operating system device, make sure that the PAgP settings used for establishing EtherChannels are coordinated. The defaults are different for a Cisco IOS Software device and a Catalyst operating system device. As a recommended practice, Catalyst operating system devices should have PAgP set to Off when connecting to a Cisco IOS Software device if EtherChannels are not configured. If EtherChannel PAgP is used, set both sides of the interconnection to Desirable.
Port aggregation should be disabled when not needed. Port aggregation can most effectively be controlled by disabling it on interfaces facing end users with the set port host macro on the Catalyst operating system or the switchport host macro on Cisco ISO Software. These macros disable both trunking and EtherChannel while enabling STP PortFast.
Port Aggregation Protocol
PAgP is one of the control mechanisms for EtherChannel. PAgP has four modes related to the automatic formation of bundled, redundant switch-to-switch interconnections (see Figure 2-12):
){kind=link}
- On: Mode that forces the LAN port to channel unconditionally. In the on mode, a usable EtherChannel exists only when a LAN port group in the on mode is connected to another LAN port group in the on mode. Because ports configured in the on mode do not negotiate, no negotiation occurs traffic between the ports. You cannot configure the on mode with an EtherChannel protocol. If one end uses the on mode, the other end must also.
- Desirable: Places a port into an active negotiating state, in which the port starts negotiations with other ports by sending PAgP packets. This mode is not supported when the EtherChannel members are from different switches in the switch stack (cross-stack EtherChannel).
- Auto: Places a port into a passive negotiating state, in which the port responds to PAgP packets it receives but does not start PAgP packet negotiation. This setting minimizes the transmission of PAgP packets. This mode is not supported when the EtherChannel members are from different switches in the switch stack (cross-stack EtherChannel).
- Off: Do not become a member.
As with DTP, the long-standing practice for EtherChannel/PAgP has been to set one side of the interconnection (typically the access switch) to Auto and the other side (typically the distribution switch) to Desirable, or both sides to Desirable. In these configurations, an EtherChannel is established when configuration is complete, and connectivity to the remote switch is always available, even when the EtherChannel is not completely established.
Link Aggregation Control Protocol
LACP is another control mechanism for EtherChannel. LACP has four modes related to the automatic formation of bundled, redundant switch-to-switch interconnections:
- On: Mode that forces the LAN port to channel unconditionally. In the on mode, a usable EtherChannel exists only when a LAN port group in the on mode is connected to another LAN port group in the on mode. Because ports configured in the on mode do not negotiate, no negotiation traffic occurs between the ports. You cannot configure the on mode with an EtherChannel protocol. If one end uses the on mode, the other end must also.
- Active: LACP mode that places a port into an active negotiating state, in which the port initiates negotiations with other ports by sending LACP packets.
- Passive: LACP mode that places a port into a passive negotiating state, in which the port responds to LACP packets it receives but does not initiate LACP negotiation.
- Off: Do not become a member.