Linked by Thom Holwerda on Thu 28th Mar 2013 00:36 UTC, submitted by MOS6510
Internet & Networking "The New York Times this morning published a story about the Spamhaus DDoS attack and how CloudFlare helped mitigate it and keep the site online. The Times calls the attack the largest known DDoS attack ever on the Internet. We wrote about the attack last week. At the time, it was a large attack, sending 85Gbps of traffic. Since then, the attack got much worse. Here are some of the technical details of what we've seen."
Thread beginning with comment 556918
To view parent comment, click here.
To read all comments associated with this story, please click here.
Laurence
Member since:
2007-03-26

You're both correct and wrong.

This attack wasn't intended to bring down the net; just Cloudflare (and it didn't even succeed at that either).

However what this attack does highlight is the seriousness of unprotected open DNS resolvers, which were the servers exploited to generate the ~300Gb/s DDoS traffic. The repercussions of which could be a threat as it means criminals no longer need large botnets to take smaller organisations offline. All they need now is a modest amount of bandwidth to flood poorly configured DNS servers with forged UDP packets that are then multiplied up at the name servers by a factor of 10.

While DDoS attacks will always be a threat, open resolvers make it easier than ever to disrupt services and this latest story is basically a massive advert to anyone considering a denial of service attack.

I just hope the seriousness of this is taken on board and action is taken to mitigate the effectiveness of this attack (there's a few different approaches to this, one of them being to patch the name servers themselves, but personally I'd rather see ISPs, peers and exchanges to add some reverse engineering to their UDP forwarding - in that they only forward UDP packets if the IP address attached can be routed backwards - thus effectively checking if the sender matches what the UDP packet describes).

Apologies about the crude explanation. I'm not a networking expert (though there is overlap between that an my field in IT) so that last part I'm having to trust online sources as being accurate.

Reply Parent Score: 3

ssokolow Member since:
2010-01-21

I just hope the seriousness of this is taken on board and action is taken to mitigate the effectiveness of this attack (there's a few different approaches to this, one of them being to patch the name servers themselves, but personally I'd rather see ISPs, peers and exchanges to add some reverse engineering to their UDP forwarding - in that they only forward UDP packets if the IP address attached can be routed backwards - thus effectively checking if the sender matches what the UDP packet describes).


That's actually a solution to what both the NYT article and one of the commenters on the CloudFlare blog identified as the real problem... that the 'net is full of routers that perform none of the sanity checks which would block such spoofed packets, regardless of what daemon we discover to be exploitable next week.

I'm no expert either, but your solution sounds more complicated (and, hence, more CPU intensive on the routers) than what they were proposing. It sounded like they were just proposing plain old source-interface checking so, when the attacker sends a spoofed packet to a DNS server, one of the border routers along the way drops it for arriving on the wrong interface.

Also, I believe it was the CloudFlare commenter who pointed out that this isn't the first attack of this kind. Before spoofed UDP flooding via DNS, there was spoofed SYN flooding.

Reply Parent Score: 2

Laurence Member since:
2007-03-26


I'm no expert either, but your solution sounds more complicated (and, hence, more CPU intensive on the routers) than what they were proposing. It sounded like they were just proposing plain old source-interface checking so, when the attacker sends a spoofed packet to a DNS server, one of the border routers along the way drops it for arriving on the wrong interface.

We're talking about the same check. What I was describing was the process behind "plain old source-interface checking".

Also, I believe it was the CloudFlare commenter who pointed out that this isn't the first attack of this kind. Before spoofed UDP flooding via DNS, there was spoofed SYN flooding.

Totally. But AFAIK we've never seen the same degree of amplification (eg every bit being multiplied up to as much as 10bits) before, not even with SYN flooding. Which is where attacking open resolvers come into play.

I might be wrong on this though so welcome any corrections ;)

Reply Parent Score: 2

Soulbender Member since:
2005-08-18

that the 'net is full of routers that perform none of the sanity checks which would block such spoofed packets, regardless of what daemon we discover to be exploitable next week.


A) This should be done on the customer-facing equipment, not on border routers.
B) Most ISP's already do this. Really.
C) You don't need to spoof the source to make use of open DNS resolvers. That is the crux of the problem, that this attack is created by "valid" packets.

Reply Parent Score: 2

puidelup Member since:
2013-03-19

"The repercussions of which could be a threat as it means criminals no longer need large botnets to take smaller organisations offline."
"While DDoS attacks will always be a threat, open resolvers make it easier than ever to disrupt services .... "


Laurence, you are implying here that this is a new attack vector (I understand your statement like this). It definitely isn't.
CloudFlare:
http://blog.cloudflare.com/deep-inside-a-dns-amplification-ddos-att...
- "a known problems at least 10 years old."
The number of Open DNS resolvers that can be used in a DNS amplification attack is actually in decline.

"I'd rather see ISPs, peers and exchanges to add some reverse engineering to their UDP forwarding - in that they only forward UDP packets if the IP address attached can be routed backwards"


This isn't going to happen anytime soon. Adding such checks in the current infrastructure would reduce the capacity of backbones by a few levels of magnitude. "Backbone routers" are optimized to route tons of traffic, but only blindly. Adding checks would cripple their routing capacity.
Such checks (anti spoofing measures) can only be implemented at the "outskirts" of the Internet, not in it's core. Admins of small networks are responsible for such security measures, but since such attacks use their infrastructure without damaging it much, there is little incentive to do it.

Reply Parent Score: 2

Laurence Member since:
2007-03-26


Laurence, you are implying here that this is a new attack vector (I understand your statement like this). It definitely isn't.

It is a new vector in attack in that it's only really been exploited like this in recent years. Or at least I've not been aware of hackers targeting open resolvers for DDoS attacks until recently. So I'm assuming it wasn't a commonly used technique until recently. If you know otherwise then I'll happily accept the correction ;)

I wasn't implying that this is a new vulnerability though, just that this existing vulnerability is getting wider exposure (advertising) so this specific type of exploit is becoming more frequent.


This isn't going to happen anytime soon. Adding such checks in the current infrastructure would reduce the capacity of backbones by a few levels of magnitude. "Backbone routers" are optimized to route tons of traffic, but only blindly. Adding checks would cripple their routing capacity.
Such checks (anti spoofing measures) can only be implemented at the "outskirts" of the Internet, not in it's core. Admins of small networks are responsible for such security measures, but since such attacks use their infrastructure without damaging it much, there is little incentive to do it.

Oh I'm well aware of that. This is why there's a coordinated underway to identify vulnerable name servers and work with the hosts to get them patched (ie it's the most realistic solution to this immediate concern).

My comment regarding the router checks was what I'd prefer to see; "ideal world" thinking etc. But as that post was quickly becoming my second essay in this thread I decided to cut some detail out for the sake of getting back to work ;)

[edit]
reworded a lot of this as it really wasn't clear what I was trying to say.

Edited 2013-03-28 12:52 UTC

Reply Parent Score: 2

Soulbender Member since:
2005-08-18

but personally I'd rather see ISPs, peers and exchanges to add some reverse engineering to their UDP forwarding - in that they only forward UDP packets if the IP address attached can be routed backwards


This is already done by any competent provider and it is why spoofing is much less common today than it used to be. It's not feasible to do this between "Tier 1" peers though (due to, among other things, asymmetric routing) so it's important that providers closer to the customer does this properly.
This wouldn't solve the problem with DNS amplification attacks though since the source was valid and not spoofed. The only way to effectively stop these attacks is to not have open DNS resolvers.

Reply Parent Score: 2

Laurence Member since:
2007-03-26

The source was spoofed. That's how the amplification attack works:

1. send spoofed UDP packet to DNS server.
2. server then replies to the spoofed UDP packet.
3. and the target server goes down because the spoofed UDP packet has the target as the source IP.

It's a bit like me pinging Google pretending to be your IP. Then google responds be sending a reply to you instead of me. Except we're talking several orders of magnitude more bits being exchanged than in a simple IMCP echo request. And the DNS server replying with more data than they received as part of the domain name lookup request.

Reply Parent Score: 2