DDoS Attacks From IoT Botnets Don’t Have to Mean Game Over
When organizing a global, multi-week, high-profile event, there are always chances for things to go wrong – and, given human nature, we tend to simply accept it as a given when things go as planned, and to notice and highlight difficulties in execution.
A great deal has been written and spoken about the challenges facing the organizers, sponsors, and contestants in Brazil during the summer of 2016. And if we think about it, we can extrapolate potentially thousands of potential pitfalls and difficulties which accompany any event of similar complexity.
Success is Blasé
We’ve come to view internet applications and services in much the same way. When they’re working well, we don’t even notice how amazing it is that we’re able to instantly view live streaming video of global events, along with commentary, pretty much anywhere on the globe, on our computers, smartphones, and tablets. But if we somehow can’t get access to the latest and greatest content and information instantly – and share it and discuss it online with our friends – then we become intensely frustrated and vocal with our displeasure. The uninterrupted availability and resiliency of online information services, apps, data, and content is now de rigeur for events of any size, at scale. This is manifestly true for international events.
One recent global event was targeted before, during and after – and largely overcame – significant challenges which at times seemed almost insurmountable. Many problems, some of them factual, some of them less so, have been described and discussed and dissected in excruciating detail.
Even before the event began, public-facing web properties and organizations affiliated with it were targeted by sustained, sophisticated, large-scale DDoS attacks reaching up to 540gb/sec. While many of these attacks were ongoing for months prior to the start of the event, attackers increased their efforts significantly once it began, generating the longest-duration sustained 500gb/sec-plus DDoS attack campaign we’ve observed to date.
And nobody noticed.
This is the sine qua non of DDoS defense – maintaining availability at scale, even in the face of skilled, determined attack. And just like the countless other services we rely upon every day such as electricity, fresh water, transportation and emergency services, the ultimate metric of success is that the general public can go about their business and pursue their interests without ever knowing or caring that titanic virtual struggles are taking place in the background.
By any metric, the security and network personnel involved in the successful mitigation have set the bar for rapid, professional, effective DDoS protection under the most intense scrutiny of any major international event to date. And did we mention that the attacks ranged up to 540gb/sec in size?!
An Ongoing Attack Campaign, Expanded
Over the last several months, several organizations affiliated with the event came under large-scale volumetric DDoS attacks ranging from the tens of gigabits/sec up into the hundreds of gigabits/sec. A large proportion of the attack volume consisted of UDP reflection/amplification attack vectors such as DNS, chargen, NTP and SSDP, along with direct UDP packet-flooding, SYN-flooding and application-layer attacks targeting Web and DNS services. The IoT botnet utilized in most of these pre-event attacks was described in detail in a recent weblog post by our Arbor ASERT colleague Matt Bing. This very same botnet, along with a few others, was also used to generate the extremely high-volume (but low-impact, thanks to the efforts of the defenders!) DDoS attacks against an expanded list of targets throughout the weeks long international gathering.
One of the characteristics of information security in general, and DDoS defense in particular, is that we see new attack methodologies pioneered by more skilled attackers and used sporadically for years (and sometimes decades) before they’re ‘weaponized’ and made more broadly available to low-/no-skill attackers via automation. We’ve encountered various types of high-volume/high-impact reflection/amplification attacks since the late 1990s; and then, 3 1/2 years ago, they suddenly became wildly prevalent due to their inclusion in the arsenal of DDoS botnets-for-hire and so-called ‘booter/stresser’ services. This has led to a highly asymmetrical threat environment which favors even the most unskilled attacker due to the fact that these Internet ‘weapons of mass disruption’ are now available to the masses via a few mouse-clicks and a small amount of Bitcoin. We’ve seen this pattern repeat itself over and over again, with disparate groups of threat actors totally unaffiliated with one another independently rediscovering more sophisticated attack mechanisms, and then proceeding to weaponize them with nice GUIs and even 24/7 online ‘customer’ support!
Everything Old is New Again
For the relatively small number of people who have a reason to think about how the Internet actually works, the only protocols they tend to remember are TCP, UDP, and ICMP. Since those protocols represent by far the largest proportion of Internet traffic, little if any thought is given to other IP protocols.
In reality, there are 256 Internet protocols, numbered 0-255. TCP is protocol 6, UDP is protocol 17, and ICMP is protocol 1. On the IPv4 Internet, only 254 of those protocols should ever be observed – protocol 0 for IPv4 (but not for IPv6!) is reserved, and should never be utilized, even though routers and layer-3 switches will happily forward it along. Protocol 255 is also reserved; most routers and switches won’t forward it. Of the set of less-familiar IP protocols, Generic Routing Encapsulation (GRE), used for unencrypted ad-hoc VPN-type tunnels, is protocol 47.
Starting in late 2000, we began to observe more skilled attackers occasionally using these lesser-known protocols in DDoS attacks – almost certainly in an attempt to bypass router ACLs, firewall rules, and other forms of DDoS defense which were configured by operators who only took TCP, UDP, and ICMP into account. In many cases, these attacks initially succeeded until the defenders finally inferred what was going on, generally via analysis of NetFlow telemetry using collection/analysis and anomaly-detection systems such as Arbor SP.
And now we’ve seen those same attack techniques rediscovered, weaponized and utilized during this international event. In particular, significant amounts of GRE DDoS traffic was generated by the attackers; this ‘new’ attack methodology has now been incorporated into the same IoT botnet referenced above. As with all ‘new’ types of DDoS attacks the miscreants stumble upon, we expect to see other botnets-for-hire and ‘booter/stresser’ services adding GRE to their repertoires in short order.
We also observed uncomplicated, high-volume packet-floods destined for UDP/179. As most (not all) UDP reflection/amplification attacks tend to target UDP/80 or UDP/443 in order to confuse defenders who might not notice that the attackers are using UDP instead of TCP (TCP/80 is typically used for non-encrypted Web servers, and TCP/443 for SSL-/TLS-encrypted Web servers), we believe the attackers were attempting to masquerade an attack on the BGP routing protocol used to weave Internet-connected networks together. BGP runs on TCP/179; the irony is that one of the few best current practices (BCPs) actually implemented on a significant proportion (not all!) Internet-connected networks is to use infrastructure ACLs (iACLs) to keep unsolicited network traffic from interfering with BGP peering sessions.
DDoS Defense– It’s All About Teamwork, Especially for High Profile Events
The defenders knew they’d have their work cut out for them, and prepared accordingly. A massive amount of work was performed prior to the start of the event; understanding all the various servers, services, applications, their network access policies, tuning anomaly-detection metrics in Arbor SP, selecting and configuring situationally-appropriate Arbor TMS DDoS countermeasures, coordinating with the Arbor Cloud team for overlay ‘cloud’ DDoS mitigation services, setting up virtual teams with the appropriate operational personnel from the relevant organizations, ensuring network infrastructure and DNS BCPs were properly implemented, defining communications channels and operational procedures, et. al.
For those turning to their devices to watch, discuss and share the big event, everything went as you’d expect. Behind the scenes, it was months of careful planning and practice. When the expected happened, and skilled attackers began their online assault on the event, the extended DDoS defense team across service providers, enterprise and event organizers, demonstrated that maintaining availability in the face of large-scale, sophisticated and persistent DDoS attacks is well within the capabilities of organizations which prepare in advance to defend their online properties. They understood that this event was a source of national pride and the glare of the international spotlight was on them. They were responsible for delivering to an online audience of billions of people around the world. The combination of skilled defenders, best-in-class DDoS defense solutions, and dedicated inter-organizational teamwork has been proven over and over again to be the key to successful DDoS defense – and nowhere has this been more apparent than this recent international event.