Managing Large Networks: Problems and Solutions

This chapter is from the book

Network Management, MIBs and MPLS: Principles, Design and Implementation

Learn More Buy

This chapter is from the book

This chapter is from the book 

Network Management, MIBs and MPLS: Principles, Design and Implementation

Learn More Buy

MPLS: Second Chunk

Chapter 2 gave a brief introduction to MPLS and described how it enables IP networks to be operated in a fashion similar to layer 2 networks. Currently, ATM is the only layer 2 technology that provides traffic engineering and guaranteed QoS for a range of network service classes—voice, video, and data. A specified class of service can be allocated to new services without affecting the existing ones (provided the network has the appropriate bandwidth resources). MPLS promises to bring similar flexibility to layer 3 networks.³

This second chunk of MPLS digs a little more deeply into MPLS and more fully describes the principal components of the technology. This is another example of a linked overview, which strips the technology down to its essential components. These can be considered to be the managed objects of MPLS:

• Explicit Route Objects (ERO), strict and loose

• Resource blocks

• Tunnels and LSPs

• In-segments

• Out-segments

• Cross-connects

• Routing protocols

• Signaling protocols

• Label operations: lookup, push, swap, and pop

• Traffic engineering

• QoS

As we'll see, the hardest way to manage MPLS networks is to not use signaling to set up LSPs. Why would a user not want to use signaling? The NEs might not support signaling. Another reason is control: The operator alone might want to dictate all the objects that get provisioned in an unsignaled network. Some large service providers actually operate their ATM networks in this way and might decide to do the same with MPLS. With no signaling support, it is up to the operator to create all of the above objects—we will do just this in Chapter 8 when we manually create an LSP. Another reason for not wanting signaling protocols running in the network is the extra resources they consume. All networks are only as strong as their weakest link, and with complex signaling protocols running in the background, bottlenecks may suddenly appear unexpectedly. We don't necessarily agree with running or not running any particular signaling protocols—we just mention the above as possible explanations. One important point is that signaling is probably required as networks become very large (tens to hundreds of thousands of nodes).

We describe LSP setup using the MPLS MIBs in some detail in Chapter 8.

Explicit Route Objects

An ERO is a list of layer 3 address hops inside an MPLS cloud. Similar to an ATM designated transit list (DTL), it describes a list of MPLS nodes through which a tunnel passes. The path taken by the tunnels in Figure 3-3 is an example of an ERO. The purpose of an ERO is to allow the user to specify the route that a tunnel will take. In other words, it allows the user to constrain the route. EROs can be either strict or loose. A strict ERO specifies all the hops in the path. A loose ERO allows for intermediate networks in the path, such as another cloud (e.g., an SP network). EROs are stored in a MIB table on the originating node and can be used by more than one tunnel originating on that MPLS node. EROs are not used in the manual creation of LSPs.

EROs are used by signaling protocols (such as RSVP-TE) to create tunnels. The path specified in the ERO must be realizable (i.e., links must exist between the designated nodes) and any required bandwidth resources (described in the next section) must be available.

Resource Blocks

MPLS permits the reservation of resources in the network. This provides a means for network operators to deterministically carve up their bandwidth and allocate it to specific LSPs. Resource blocks provide a means for recording the bandwidth settings, and they can then be assigned to specific LSPs. The components of a resource block include:

• Maximum reserved bandwidth

• Maximum traffic burst size

• Packet length

A given LSP, such as one of those in Figure 3-3, could have an associated end-to-end bandwidth reservation of 2Mbps. This means that the LSP is engineered to carry 2Mbps of traffic along the specified route. Best-effort LSP operation is also possible, in which case the resource block is null.

Tunnels and LSPs

MPLS tunnels (as we saw in Chapter 2, Figure 2-8) represent a type of container for paths made up of nodes with configured in-segments, cross-connects, and out-segments. The tunnel is the on-ramp into the associated label switched path. What we didn't say in Figure 2-8 is that tunnel instances are also supported. A tunnel instance is a separate tunnel, which is in some sense owned by the first tunnel. In other words, the tunnel and its tunnel instances have a relationship with each other. This relationship consists of sharing the same endpoints and might be needed for backup (a disjoint path) or for load sharing. We will see this more clearly in Chapter 8.

MPLS-encapsulated packets enter the tunnel, pass across the appropriate path, and exhibit three important characteristics:

• Forwarding is based on MPLS label (rather than IP header) lookups (this is no longer an advantage enjoyed by MPLS nodes, as IP routers can now forward at line rate).

• Resource usage is fixed, based on those reserved at the time of connection creation.

• The path taken by the traffic is constrained by the path chosen in advance by the user.

Tunnels and LSPs provide reachability for traffic with specific destination IP addresses. Routing protocols direct packets onto specific tunnels and LSPs in order to reach the appropriate IP destination.

In-Segments and Out-Segments

In-segments on an MPLS node represent the point of ingress for traffic. Out-segments represent the point of egress for traffic. The two types of segment objects are logically combined using a cross-connect.

Cross-Connects

Cross-connects are objects that associate in- and out-segments together. The MPLS node uses the cross-connect settings to decide how to switch traffic between the segments. The cross-connect table supports the following connection types:

• Point-to-point

• Point-to-multipoint

• Multipoint-to-point

A cross-connect entry has both an administrative status and an operational status. The administrative status indicates the state required by the operator, whereas the operational status indicates the actual state of the cross-connect in the node. Operationally down cross-connects will not forward packets.

Routing Protocols

MPLS incorporates standard IP routing protocols such as OSPF, IS-IS and BGP4. This is done because these protocols have been used and proven over the course of many years. Incorporating them into the MPLS standards improved the chances of widespread deployment of MPLS. Traffic engineering extensions added to the routing protocols means that they can advertise and distribute both routing and resource (e.g., link bandwidth) details. This is crucial for facilitating the creation of route-constrained LSPs (i.e., tunnels). This ultimately allows the user requirements to influence the path taken by IP traffic through an MPLS cloud.

Signaling Protocols

As we've seen, the creation of LSPs and tunnels can be achieved either manually (similar to the way ATM PVCs are created) or via signaling. Signaled connections have resources reserved, labels distributed, and paths selected by protocols such as RSVP-TE or LDP.

Label Operations

As MPLS-labeled traffic traverses an LSP or tunnel, it is forwarded based on its encapsulated label value. The IP header is no longer consulted while the packet is inside the MPLS domain. MPLS labels can be stacked in a last-in-first-out fashion; that is, more than one label can be applied to a packet. Labels can be stacked (if the hardware supports it) up to the limit allowed by the layer 2 protocol. The label counts as part of the layer 2 message size. The outermost label (i.e., the last one pushed on the stack) is the one used for forwarding the packet. If more than one label is on the stack, then stripping off (or popping) the topmost label exposes the next label down. Forwarding is then carried out based on this label. This is illustrated below with an MPLS-encapsulated packet that has a stack containing two labels.

The current MPLS node uses Label2 when forwarding this packet. The operations that can be executed against labels are:

• Lookup: The node examines the value of the topmost label. This operation occurs at every node in an MPLS cloud. In our example, lookup would occur using Label2. Typically, a label lookup results in the packet being relabeled and forwarded through a node interface indicated by the incoming label.

• Swap: This occurs when an MPLS node replaces the label with a new one.

• Pop: This occurs when the topmost label is removed from the stack. If the label stack has a depth of one, then the packet is no longer MPLS-encapsulated. In this case, an IP lookup can be performed using the IP header.

• Push: This occurs when a label is either pushed onto the label stack or attached to an unlabeled packet.

In Chapter 4, “Solving the Network Management Problem,” we will see that the MPLS shim header contains a bit field called Stack. A value of 1 in the stack field indicates that this is the last remaining label in the stack; the value zero indicates that other labels are pushed beneath the current label. The value of the Stack field changes appropriately as labels are pushed and popped. An important point to note is that the MPLS labels have local significance only. The contents of MPLS labels can also assist in the QoS scheme (we will see this use of labels in more detail in Chapter 4).

MPLS Encapsulation

FR and ATM can accommodate MPLS labels in their layer 2 headers. Other technologies use a shim header for the label. This is a mini-header (more than one is allowed because stacking is supported) that sits beside the IP header. FR uses the data link connection identifier (DLCI) field, and ATM uses the VPI/VCI fields. ATM has another field called cell loss priority (CLP) that is used for QoS support. This field is used to mark cells for two service levels: Cells with a CLP of 1 are discarded (if incoming traffic levels are too high) prior to cells with a CLP of 0. Cells with a CLP of 0 are not guaranteed to be forwarded, but they will have precedence over cells with CLP of 1.

The MPLS encapsulation specifies four reserved label values:

• 0 – IPv4 explicit null that signals the receiving node to pop the label and execute an IP lookup

• 1 – Router alert that indicates to the receiving node to examine the packet more closely rather than simply forwarding it

• 2 – IPv6 explicit null

• 3 – Implicit null that signals the receiving node to pop the label and execute an IP lookup

When an MPLS node operates in SIN mode (ATM and MPLS running simultaneously), there may be additional constraints on the label range, but this is platform-specific.

QoS and Traffic Engineering

The Internet currently offers a single default service level: best effort. Many enterprises also offer just best-effort QoS for IP traffic but on an overengineered underlying network. Bandwidth in the LAN is relatively cheap and can be augmented as needed using switches. This means that the excess bandwidth helps avoid congestion, but this is a blunt instrument that doesn't scale well and is prone to congestion during very high bursts of traffic. In effect, overengineering passively avoids congestion but doesn't proactively prevent it. It's not unknown for the developers of NMS software to inadvertently flood their local LAN with SNMP traffic. Without proactive QoS and traffic engineering in the network, the enterprise network manager often has to resort to restricting access to the network to avoid such problems. A better solution would be to provide specific chunks of bandwidth (via MPLS LSPs) to the developers. WAN links normally have strictly limited bandwidth and provide an even stronger case for QoS management.

Traffic engineering is set to become a mandatory element of converged layer 3 enterprise networks [MPLS&Profits]. MPLS provides resource advertisements in its routing protocols (extended from the regular IP operation). Link bandwidth states are included as extensions of the standard IP routing protocols such as OSPF and IS-IS. These are traffic-engineering enhancements, and the modified protocols are referred to as OSPF-TE and IS-IS-TE. The purpose of the enhancements is to allow MPLS routers to construct a complete link-state database of the network, including available/allocated bandwidth.

QoS

The need for definable levels of QoS is due to the increasingly mission-critical, real-time applications being deployed on enterprise and SP networks. Overengineering of the core gets you only so far, but the WAN may then become a bottleneck (remember that a network is only ever as strong as its weakest link). Once the overengineered cores become depleted, there is a need for IP QoS. The issue of SP core depletion is interesting and slightly controversial because it is estimated [TimesMarch2002] that between 1998 and 2002, service providers invested about $500 billion in fiber-optic networks and that only four percent of the new fiber is in use. The bulk of the fiber is idly sitting underground. Quite possibly, core depletion is therefore not an immediate issue for service providers, but this is not the case for enterprises.

An allied problem is due to the way in which enterprise and SP networks are inextricably interwoven. When an enterprise wants to send, say, email, Web, FTP, and VoIP traffic through an SP network, then a number of approaches can be taken by the enterprise:

1. It can rate all of its traffic as being equally important.

2. It can rate the VoIP traffic as being the most important.

Unfortunately, most enterprises choose the first option. This may be because:

• The enterprise equipment is not capable of marking traffic passed to the service provider.

• There is no incentive from the service provider.

• It is difficult to manage from within the enterprise network.

Whatever the reason, this causes a downstream problem for the service provider because there may be no easy way of differentiating the incoming traffic streams. In practice, it matters little if an email message arrives at its destination one minute or one second after sending it. The same would not be true of real-time, delay-sensitive traffic such as voice or video. As traffic levels increase, these issues become more pressing for service providers, prompting the need for QoS to be applied in both the enterprise and SP networks. Service providers often deploy (and manage) customer premises equipment (CPE) to execute such functions.

There are essentially three approaches that can be adopted for providing different levels of network service [CrollPackmanBW]:

• Best effort, as provided by the Internet

• Fine granularity QoS, such as Integrated Services (IntServ)

• Coarse granularity QoS, such as Differentiated Services (DiffServ)

We're all familiar with the best-effort model from surfing the Internet. The Internet does its best to deliver traffic between senders and receivers, but there is no guarantee. This works well for non-real-time traffic, such as email and Web site access. Internet telephony is also reasonable as long as the users aren't too fussy about quality.

Another approach to service provision is fine granularity QoS of which the IntServ model is an implementation. The IntServ model allows path (called microflow) reservation across the network, and traffic is pushed into these paths in order to get to its destination. The paths have to be explicitly reserved, and they must also be periodically refreshed. IntServ implementations have two elements of overhead made up of path handling and refreshing.

The third approach is coarse granularity QoS, and it uses the technique of traffic marking. This is the model used by DiffServ. It is often considered to be more scalable than IntServ because no microflows are required. Packets are marked (at the point of origination or by a downstream NE such as a customer edge router) with a specific value. This value is then used as the packet is forwarded through the network. For DiffServ, the values are called DiffServ Code Points (DSCP), each of which corresponds to a given traffic-handling treatment. As the marked traffic makes its way through the network, it is processed at each node in accordance with its DSCP. The DSCP specifies what is called a behavior aggregate; that is, all packets with this DSCP fall into the same category. Each category represents a distinct class of service. When a router receives a packet marked for a given class of service, it imposes what is called a per-hop-behavior (PHB) on the packet. This is the way the packet is treated by the node.

The PHB consists of queuing strategy, policing (dropping the packet or remarking it), shaping, classifying, and so on. It has two main functions:

1. Scheduling the packet into the egress interface

2. Discarding the packet using a drop priority

Scheduling refers not to how packets are stored in the queues, but how they are pulled from the queues by the scheduler. The QoS experienced is the final result of all this activity as this packet (and associated packets) makes its way through the network.

In Chapter 4 we will see how these models are used in the context of MPLS. For the moment, we illustrate how IP packets are marked with DSCPs in the Differentiated Services (DS) field (formerly known as the Type of Service) in the IP header. This is illustrated in Figure 3-5, where the IP header DS field is located in the second octet position. The value assigned to this field can be used to provide specified forwarding treatment for the packet as it traverses the DiffServ domain.

Figure 3-5. The IP header.

RFC 3260 provides an update, clarifications, and new elements of DiffServ. Among other things:

• It clarifies that the DSCP field occupies only the first six bits of the DS field.

• The other two bits will be used for Explicit Congestion Notification (ECN) [DavieRehkter2000], as can be seen in Figure 3-5.

Downstream switches and routers then take the DS value as the cue for special treatment—that is, differentiated services. Packet service metrics can include:

• Bandwidth allocation

• Packet loss rate

• Delay

• Jitter (variation in delay)

The value in the DS field can be used to dictate these metrics. Clearly, the nodes traversed by the marked packets must know the meaning of the DSCPs. This is an important part of DiffServ: the mapping between DSCP and the forwarding treatment. Ultimately, this is reflected in the end-user service level.