This chapter is from the book
This chapter is from the book
This chapter is from the book
DoS Design Considerations
Designing your network to properly deal with DoS/flood attacks is an exercise in compromises. DoS attacks cannot be completely stopped. Anyone who tries to sell you something to completely solve your DoS problems is lying, period. DoS attacks are so easy to launch that they are often considered bad form in the attack community. Three DoS attacks were highlighted in Chapter 3: smurf, DDoS, and TCP SYN. The first two fall into the category of network flooding attacks designed to consume all available bandwidth. The latter is a transport flooding attack designed to consume the resources of a host.
Network Flooding Design Considerations
Detecting a network flood is fairly easy: NIDS, routers, and firewalls can all show signs of a network flood in the log data. Stopping a network flood is something entirely different. The chief problem with stopping a network flooding attack is that, by the time the attack reaches your organization, it is already too late. As you learned in Chapter 3, if an attacker sends your T1 connection a T3's worth of data, it doesn't matter if you drop all these packets when they hit your WAN router. Your T1 is already filled, damage done. So, who can help? Your service provider (SP) can.
Your SP has a few specific technologies available, but be aware that most will stop good and bad traffic from reaching the IP address under attack. As an ISP customer, you should have a plan to deal with this eventuality. It should include answers to the following questions:
How fast can your DNS infrastructure propagate a new IP address for the DNS name under attack?
Do you currently have redundant systems to which you can make a simple cutover instead of losing legitimate flows?
What happens if the IP address under attack is your primary router port or other critical infrastructure device? Do you have contingency plans to deal with this?
Additionally, you should know whether your SP offers several of the attack mitigation capabilities outlined in the next sections and, if so, how and when they will be implemented.
Stopping Network Flooding
If you and the ISP decide to just stop the attack outright, there are three primary options, all of which stop good and bad traffic from reaching the victim IP address: basic ACL, black hole filtering, and sinkhole routing.
Basic ACL
The simplest way to stop an attack against a particular IP address is to drop any traffic destined for that IP address. By configuring these ACLs throughout an ISP's network in response to an attack, all traffic destined for the victim IP address can be dropped, stopping the attack. This is a time-consuming process and is the least effective of the three methods.
Black Hole Filtering
Through the clever propagation of static routes in BGP, it is possible to inject a route into the ISP network, causing any traffic destined for the IP that is under attack to be dropped. Traffic is typically routed to null0 (the bit bucket) because this has less CPU impact than dropping the traffic by an ACL (in addition to being much faster to propagate to all ISP routers). Black hole filtering can also be made available to you as an ISP customer if your ISP allows it; see http://www.secsup.org/CustomerBlackHole/ for more information.
Sinkhole Routing
If the ISP is interested instead in examining the flooding attack and stopping it, it can use sink-hole routing. This works by injecting a more specific route from one of the ISP's routers than the subnet route you advertise, which is under attack. For example, if your subnet is 192.0.2.0/24 and IP address 192.0.2.52 is under attack, the ISP can inject a route specifically to the 192.0.2.52/32 address that redirects the attack traffic to a network honeypot of sorts, where the ISP can examine and classify the traffic.
DDoS Trace Back
If, instead, the ISP wants to trace back the source of the attack, there are separate methods to do this. Be aware that trace back is simple if the attack is not spoofing its source address. If the attack uses spoofed source addresses, one of the two primary techniques is used: manual ACL trace back or backscatter DDoS trace back.
Manual ACL Trace Back
When an ISP first tries to categorize an attack, an ACL can be built with a series of broad permit statements that are made more specific as more information about the attack is learned. By measuring the amount of "hits" each ACL entry gets, the ISP is able to determine the kind of traffic that is causing the attack. Once the attack type is determined, a small sampling of traffic can be logged with the log-input ACL flag discussed in Chapter 5. This allows the source interface and source MAC address to be determined. By using this information, the ISP can repeat this process on the router that is sourcing the attack. This trace back technique can take time and often results in attack sources on different links in the event of a DDoS flood. Each of these must be traced back individually.
Backscatter DDoS Trace Back
This technique was developed by Chris Morrow and Brian Gemberling at UUNET, and it allows a DDoS attack to be stopped and trace back to occur in approximately 10 minutes. The following site provides more information: http://www.secsup.org/Tracking/.
At a high level, the mitigation technique works by combining aspects of the sinkhole and black hole routing discussed previously. When a system is under attack, the black hole routing technique allows ISP edge routers to route the traffic to null0. This causes an ICMP unreachable to be generated by the router for each spoofed source address that is routed to null. Here's where the trick comes in.
The IPv4 address space is only partially allocated; currently no one owns large blocks of addresses. The list can be found here: http://www.iana.org/assignments/ipv4-address-space. Your ISP can advertise these prefixes and set them to not be exportable to other ISPs. ISPs do this by using the sinkhole router. Because these large blocks of IPv4 address space are now routable within the ISP, all of the ICMP unreachables from spoofed sources in the range the ISP is falsely advertising flow to the sinkhole router. The sinkhole router sees these ICMP unreachables with a source IP address of the router that sent them. Then the ISP has a list of the routers that are seeing the flood attack!
I continue to be amazed by this technique of backscatter trace back. I am constantly surprised by the ingenuity of the Internet's users, and I enjoy it even more when the novel idea is for the cause of good rather than evil. As a note, attackers wishing to get around this method of trace back need only ensure that the spoofed source addresses they use are allocated to legitimate networks.
All of the techniques described in the preceding two subsections are detailed in a North American Network Operators Group (NANOG) presentation from NANOG23 titled "ISP Security—Real World Techniques," delivered by Barry Greene, Chris Morrow, and Brian Gemberling. It is available at the following URL: http://www.nanog.org/mtg-0110/greene.html.
CAR
This DDoS mitigation technique is losing favor because more and more attacks fail to be adequately classified by this technology. CAR is a QoS technique that, for the purposes of flooding mitigation, limits traffic matching an extended ACL to a specific rate. For example, you could use CAR to limit the following types of traffic:
ICMP traffic to 100 Kbps
UDP traffic to 5 Mbps
TCP SYN packets to 50 Kbps
To understand how CAR works, it is helpful to use a common QoS metaphor. CAR works as a token bucket QoS implementation (see Figure 6-23). Token bucket means traffic requires a token to pass through the router. Tokens are made available to the limited traffic at the committed rate. If traffic is sent constantly at the committed rate, tokens are constantly spent to pass the traffic. If the rate drops below the committed rate, these tokens can accumulate in a token bucket. The depth of this bucket is equal to the burst rate defined in the CAR statement. If traffic has been below the committed rate for some time, the token bucket will be full. If traffic suddenly exceeds the committed rate for a short period of time, the extra tokens in the bucket can allow the traffic to pass. When the token bucket is completely exhausted, the router is able to take a loan out at the extended burst rate. If the tokens for the extended burst are exhausted, traffic is dropped. As the traffic drops below the committed rate, tokens are first used to pay off the loan for the extended burst before they are put into the token bucket.
)
Figure
6-23 CAR
Like the previous network flooding mitigation techniques, CAR must be implemented by your service provider. Since CAR impacts the performance of a router, expect to pay extra to have your ISP run CAR at all times, or you can work out an agreement in which CAR is turned on after you first detect the attack. To configure CAR to implement the three preceding examples, you start by defining the traffic types by ACLs, as shown in the following example. permit means the traffic should be rate limited; deny means it should be passed unmolested.
! ACL for ICMP Traffic access-list 102 permit icmp any any ! ACL for TCP SYN Traffic access-list 103 permit tcp any any syn ! ACL for UDP Traffic access-list 104 permit udp any any
After the ACLs are defined, the rate-limit statements are applied to each ACL. The rate-limit statements can be applied inbound or outbound; because these statements are generally made from the SP's perspective, they are all outbound. After the access list to match is defined, three rates are provided. The first is the committed rate; in the case of ICMP, this is 100 Kbps. The next two numbers are the burst rate and the extended burst rate. The final statements define what the router should do when traffic conforms to the committed rate and what should be done when it exceeds the committed rate. In most cases, the conform action is transmit and the exceed action is drop. Here is what the commands look like:
Router(config)#interface S0 Router(config-if)#rate-limit output access-group 102 100000 8000 8000 conform-action transmit exceed-action drop Router(config-if)#rate-limit output access-group 103 50000 4000 4000 conform-action transmit exceed-action drop Router(config-if)#rate-limit output access-group 104 5000000 50000 50000 conform-action transmit exceed-action drop
CAR Design Considerations
One of the first tasks in successfully configuring CAR is determining what normal traffic loads are. One of the easiest ways to do this is to start your CAR policy by setting your conform action to transmit and your exceed action to transmit. This command for the previous ICMP example looks like this:
Router(config-if)#rate-limit output access-group 102 100000 8000 8000 conform-action transmit exceed-action transmit
In this way, no traffic is dropped, but the CAR process is still running. You can then check to see what amount of your traffic is conforming and what is exceeding with the show interface int rate-limit command. The following is an example of the output of this command:
Router#sho interface fa0/0 rate-limit FastEthernet0/0 Output matches: access-group 102 params: 96000 bps, 8000 limit, 8000 extended limit conformed 393 packets, 566706 bytes; action: transmit exceeded 4224 packets, 6091008 bytes; action: drop last packet: 0ms ago, current burst: 7072 bytes last cleared 00:03:51 ago, conformed 19000 bps, exceeded 210000 bps matches: access-group 103 params: 48000 bps, 4000 limit, 4000 extended limit conformed 0 packets, 0 bytes; action: transmit exceeded 0 packets, 0 bytes; action: drop last packet: 79586392ms ago, current burst: 0 bytes last cleared 00:03:20 ago, conformed 0 bps, exceeded 0 bps matches: access-group 104 params: 48000 bps, 5000 limit, 5000 extended limit conformed 0 packets, 0 bytes; action: transmit exceeded 0 packets, 0 bytes; action: drop last packet: 79586392ms ago, current burst: 0 bytes last cleared 00:02:42 ago, conformed 0 bps, exceeded 0 bps
From the output, you can see that the router is currently rate limiting a small ICMP flood. You can see the number of packets that exceeded the rate, as well as a number of other interesting statistics.
CAR is powerful because if the attack can be classified properly, the network under attack is unaffected and can continue to service legitimate requests. In the previous ICMP example, a 100 Mbps flood of ICMP traffic would be reduced to a 100 Kbps stream, certainly not enough to adversely affect the network. The main problem with CAR is that it cannot effectively identify certain types of flooding attacks. UDP floods and ICMP floods are easy, but what if you are flooded with TCP 80 traffic with the acknowledgment (ACK) bit set in the TCP header? This is exactly the sort of traffic you should be permitting into the network, so distinguishing the attack by using CAR is impossible.
Also keep in mind that some types of CAR filtering require quite a bit of care in deploying. The TCP SYN option is the most sensitive. Assume that your normal TCP SYN rate is 100 Kbps, and you occasionally spike to 300 Kbps. You implement CAR for TCP SYN and provide a committed rate of 500 Kbps. A TCP SYN flood attack is launched against your network, sending 100 Mbps of TCP SYN traffic—enough to fill up your T3 without CAR. By using CAR, you see only 500 Kbps of the attack, but now any new TCP session won't establish because TCP SYN traffic is being rate limited so extensively. Existing TCP traffic will still pass, but if most are short-lived HTTP connections, the user's web session will quickly stop functioning.
At this point, sessions to that IP address are being dropped, but at least the rest of your network is still functioning. Other systems, routing protocols and so on, continue to work. For this reason, most users choose not to implement TCP SYN flood protection, or any CAR, all the time. Rather, they wait until the attack begins and then work with their ISP to implement the feature.
Design Techniques to Mitigate DDoS
As a security architect, there are two primary techniques you can use to reduce the chances of a successful DDoS attack in the first place: e-commerce-specific filtering and content delivery networks.
E-Commerce-Specific Filtering
In most designs, the e-commerce portion of an organization's network uses the same bandwidth as the rest of the network. Users, mail servers, and e-commerce transactions all occur over the same WAN link. This is suboptimal for several reasons:
A successful flood attack against your Internet connection will affect both general Internet and e-commerce traffic.
A spike in internal user Internet usage can affect e-commerce availability.
Because internal user traffic is so diverse (lots of applications, ports, and protocols), the usage of the WAN link can be unpredictable.
Instead, organizations could choose to separate their internal users from their e-commerce systems in one of two ways:
Move the e-commerce environment to a collocation facility at your SP, as shown in Figure 6-24.
Purchase two separate Internet connections (four if you need redundancy for both services), as in Figure 6-25.
)
Figure
6-24 Collocated E-Commerce
)
Figure
6-25
Dedicated E-Commerce WAN Connection
In the collocation example, you have the benefit of increased bandwidth because you are physically sitting within the ISP's network, whereas in the second example you have greater control and manageability of your e-commerce systems. In either case, specific filtering works the same. In an e-commerce environment, you typically need a very limited set of services to function, including the following:
HTTP (TCP port 80)
SSL/Transport Layer Security (TLS) (TCP port 443)
BGP (TCP port 179)
ICMP (as defined earlier in this chapter)
DNS is not needed if the DNS servers are hosted somewhere else, such as at the ISP. This means UDP as a whole may not be needed. With this level of specificity, it becomes possible to filter e-commerce traffic as it leaves the ISP network destined for your e-commerce systems. This provides two distinct advantages:
DDoS or worms must be very specific in order to reach the e-commerce network.
Traffic that would otherwise consume expensive ISP bandwidth can be stopped. Because the traffic is blocked at the firewall anyway, there is no sense in allowing it on the wire in the first place.
E-commerce-specific filtering is shown in Figure 6-26.
)
Figure
6-26 E-Commerce-Specific Filtering
Don't think of this as a service-provider-managed firewall; all you are asking your SP to do is implement a basic ACL outbound on your interface. If your BGP router IP is 96.20.20.2, the SP router IP is 96.20.20.1, and your web/SSL server is 192.0.2.50, the ACL would look like this:
Router(config)#access-list 101 permit tcp any gt 1023 host 192.0.2.50 eq 80 Router(config)#access-list 101 permit tcp any gt 1023 host 192.0.2.50 eq 443 Router(config)#access-list 101 permit icmp any any Router(config)#interface s0 Router(config-if)#ip access-group 101 out
NOTE
Notice that the BGP traffic did not need to be permitted by the ACL since traffic originated by the router is not filtered in an ACL. You should make the ICMP filtering more specific as discussed earlier in the chapter. This kind of filtering should also be combined with RFC 2827 filtering and bogon filtering as well.
Content Delivery Networks
The second design option is to distribute your critical systems in multiple data centers using network load balancing to distribute the content. This doesn't stop a DDoS attack, but it does lessen its significance because the other systems serving the same content are still online. Content delivery networks are touched on in Chapter 11, "Supporting-Technology Design Considerations."
Network Flooding Design Recommendations
All of the technologies described in this section can be considered potential tools in your network flooding toolkit. All of them require cooperation with your ISP, something that should be put in writing before you are attacked. Most ISPs should be receptive to cooperating on a network flooding policy, particularly if they don't yet have your business. Be sure to discuss such options as the methods that will be used for different attacks as well as how quickly you can expect a turnaround when an attack occurs. Understanding the relationships your ISP has with other ISPs can also be helpful in understanding how well it will respond when attacks occur.
TCP SYN Flooding Design Considerations
TCP SYN flooding was discussed in Chapter 3. Stopping such attacks can be done either at the host only or at the host in combination with the network. The two principal technologies to mitigate SYN flooding are SYN cookies and TCP Intercept.
SYN Cookies
SYN cookies are a host-specific method of mitigating TCP SYN flooding attacks. When the incoming SYN queue fills up from attack, a server normally must block new incoming connections. When using SYN cookies, instead of keeping each SYN in the queue, information from the SYN sent from the client is run through a cryptographic function to determine the ISN to send from the server. This way, the server mustn't keep track of the SYN packet; it must only check an incoming ACK for a new session against this cryptographic function. The ACK from the client should be exactly one more than the ISN sent from the server. By decrypting this value, the server has enough essential information to allow the TCP connection to establish, even without a copy of the original SYN. More information on SYN cookies can be found at http://cr.yp.to/syncookies.html.
TCP Intercept
TCP Intercept is a network-level protection for SYN floods. It works by brokering (on the device running TCP Intercept) a connection to a server on behalf of the client. If an incoming connection never establishes itself, the client is not affected. When the connection does establish, the device running TCP intercept passes the communication on to the real server transparently. The Cisco PIX documentation does a good job of describing the feature in detail, so I've included it here. An embryonic connection in Cisco terminology is one that has not completed the full TCP three-way handshake:
NOTE
Once the optional embryonic connection limit is reached, and until the embryonic connection count falls below this threshold, every SYN bound for the affected server is intercepted. For each SYN, PIX Firewall responds on behalf of the server with an empty SYN/ACK segment. PIX Firewall retains pertinent state information, drops the packet, and waits for the client's acknowledgement. If the ACK is received, a copy of the client's SYN segment is sent to the server, and the TCP three-way handshake is performed between PIX Firewall and the server. If, and only if, this three-way handshake completes, may the connection can resume as normal. If the client does not respond during any part of the connection phase, PIX Firewall retransmits the necessary segment using exponential back-offs.
TCP Intercept has a number of options when implemented on routers; for more information, see the following URL: http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122cgcr/fsecur_c/ftrafwl/scfdenl.htm. On the Cisco PIX Firewall, TCP Intercept is part of the static command and has only one configurable option: the number of half-open connections to accept before starting the intercept function. More information on the static command can be found at http://www.cisco.com/univercd/cc/td/doc/product/iaabu/pix/pix_62/cmdref/s.htm#1026694.
ICMP Unreachable DoS Considerations
If a request comes in to the router directed to a service the router isn't running, an ICMP unreachable message is sent. Sending ICMP unreachables could be used to deny service of the router. If an attacker can keep the router sending unreachables, the overall service of the router could degrade. To silently discard these packets without generating a message, the following command should be configured on each interface:
Router(config-if)#no ip unreachables
Earlier in this section, a DDoS traceback technique is used, which involves ICMP unreachables. Additionally, path MTU discovery uses ICMP unreachable messages, so blocking it will stop path MTU, which often isn't a good thing. If you need unreachables for this or any other reason, consider rate limiting them with the following command instead of dropping them outright:
Router(config)#ip icmp rate-limit unreachable milliseconds
This will prevent the router from being consumed with the process of sending ICMP unreachables.