top | item 29211706

Cloudflare blocks an almost 2 Tbps multi-vector DDoS attack

185 points| sendilkumarn | 4 years ago |blog.cloudflare.com | reply

92 comments

order
[+] schleck8|4 years ago|reply
There is a truly excellent video on Mirai's (the botnet or atleast code in question) origin. It was created in the Minecraft server community by teenagers. The botnet was huge to a point where Akamai had to get help from Google to mitigate an attack on krebs' security blog. It also was used to attack Dyn, the infrastructure provider, and resulted in a huge outage affecting Netflix, Twitter etc.

Sadly it's only in German, but if you are on desktop, you can auto-translate the subtitles.

https://www.youtube.com/watch?v=uletKRPMnuo

[+] raspyberr|4 years ago|reply
I've read that Cloudflare also hosts a lot of DDoS-for-hire services. That seems like a conflict of interest.
[+] winternett|4 years ago|reply
This is 2021, where almost everyone creates a global problem, then makes money off of being the one to "mitigate the problem"... The people dedicated to not creating new problems, but trying genuinely to fix problems simply fail and/or run out of money are increasingly ignored because they don't have the biggest marketing budgets. Honesty isn't making money any more... A huge problem.

The absence of any real accountability, and admiration of hypocrisy, is what threatens us most heading into the future.

[+] systemvoltage|4 years ago|reply
I think this an uncharitable simplification of a complex issue. Cloudflare tries to balance itself between censorship and overreach of what their customers are doing with their service (booting off Parlor earlier this year for example) as well as what the law-enforcement legally requires them to do. If Al Queda hosts a website on AWS, the problem is exactly the same.

And now, we have people essentially conspiring that Cloudflare creates their own DDoS attacks just so to prevent it based on a glib oversimplification.

[+] tpmx|4 years ago|reply
They certainly don't host DDoS network ops. What you're talking about is hosting web pages.
[+] donkarma|4 years ago|reply
I always thought there should be more terabit attacks with the level of home connections nowadays
[+] leros|4 years ago|reply
I would imagine ISPs have some sort of bot prevention measures that would get triggered if you went all out on using a home connection.
[+] wilde|4 years ago|reply
I assume you live outside the US? Our home connections here haven’t improved in two decades.
[+] taf2|4 years ago|reply
Assuming this is about telnxy outages this week and their migration to cloudflare. https://status.telnyx.com/

Maybe premature for cloudflare to be declaring victory?

[+] BuildTheRobots|4 years ago|reply
Whilst I'm a big fan of people updating status pages, copy/pasted updates really rub me up the wrong way.
[+] 14|4 years ago|reply
Can’t they try take the bots offline? Do the bots hide their IP address or could they not start contacting the owners of said ip addresses and tell them they need to remove the infected device from the internet? I know it wouldn’t be that easy but is there nothing they can do to fight back and start getting rid of these bots?
[+] spiffytech|4 years ago|reply
> Do the bots hide their IP address

For this attack and many like it, yes, the bots hide their IP.

Per the article, this attack was a combination of DNS amplification and UDP flood. UDP packets don't use a connection like TCP (where the recipient verifies it can talk back to the sender); instead, the packet just declares where it came from, and the recipient fires-and-forgets a response to that IP, blindly assuming that IP is actually the sender.

So for the UDP flood portion, the victim receives a packet with a fraudulent source IP and no way to tell where it really came from.

For the DNS amplification part (also done over UDP), the attacker finds an open DNS resolver online, sends it a request to resolve a record, and fakes the UDP source IP, telling the DNS server to send the response to the attack victim. Not only does this mean the DDoS packets aren't coming directly from the attacker, but DNS responses can easily be much larger than DNS requests, so an attacker multiplies how many gigabits of traffic they hit the victim with, versus just sending UDP packets directly to the victim.

Here's Cloudflare's primer on DNS amplification attacks: https://www.cloudflare.com/en-gb/learning/ddos/dns-amplifica... and UDP floods: https://www.cloudflare.com/en-gb/learning/ddos/udp-flood-ddo...

As far as solutions go, the answers are broadly 1) get everyone in the world to stop putting up UDP services that send large responses to unverified requests (this attack used DNS, but this happens with other protocols too), and 2) convince ISPs everywhere to deny outbound UDP packets which claim a source IP from outside that ISP's network. Since this is one of those "you have to be perfect, but the attacker only has to find one weakness" scenarios, these sorts of attacks will keep happening until it becomes impractical to find enough abusable networks/services to mount high-volume attacks.

[+] zeusk|4 years ago|reply
In past, they have taken bots offline (mainly by taking over the Command/Control server) but most of these "bots" are just malware infected connected devices operated by clueless average folks - hard to update, hard to take down.
[+] buro9|4 years ago|reply
The article mentions that these were UDP attacks... which are usually reflections based on spoofed IP addresses. So who should Cloudflare contact? In the meantime another few hundred small attacks arrive. It's more constructive to improve the capability to mitigate attacks as they and other network providers have agency over that.
[+] pixl97|4 years ago|reply
How long does it take to contact thousands and thousands of IP owners looking for infected device? Many of which are behind NAT devices which require even further tracing.

What about the ones overseas that just don't care?

[+] short12|4 years ago|reply
What is with ddos these days?

Are they doing it for money ?

It just seems silly with services like cloud flare

[+] 1270018080|4 years ago|reply
If you have that much computing power at your disposal, you might as well just mine cryptocurrency, right?
[+] catlikesshrimp|4 years ago|reply
You can hire a ddos agaisnt your across the street competitor. It could be the other pizzeria, the other hardwareshop. Use your imagination
[+] Ansil849|4 years ago|reply
> The entire attack lasted just one minute.

Did the attack last one minute because Cloudflare 'mitigated' it after that, or because the attackers stopped?

[+] buro9|4 years ago|reply
Botnets tests their capabilities all the time. This could have been a command and control test, a test to see what they could muster, or a demonstration.

When testing they seldom run for a long time.

Cloudflare's mitigation would've dropped in on the metals and still been visible to Cloudflare's monitoring... so the attackers stopped after a minute.

[+] toast0|4 years ago|reply
I used to run the servers for a popular website. It was common to get DDoSed targeting our servers (or more frequently, just a single one out of the group) for exactly 90 seconds (plus or minus a few systems that had poor ntp synchronization). Whether or not that took my servers down, the attack would stop.

To my knowledge, we never got any communication from the people behind the attack, seemed like people just kicking the tires on DDoS as a service. Ocassionally, we'd get a longer interval, sometimes 60 minutes.

[+] maxgashkov|4 years ago|reply
I was responsible for a website (one of a many of this kind) that provided access to a niche auction platform. At some point in the beginning of 2010s it became a subject of a precisely coordinated series of timed attacks designed to disrupt bidding of one of our prominent clients in the specific auctions. It was enough to bring down the service for ~5 minutes to prevent the client from winning.

Eventually we migrated behind CF and the problem was solved but I couldn't help but wonder if there are some applications for which even a few seconds disruption (I assume that's the minimum time Cloudflare needs to begin effectively mitigate the attack of this scale) will be disastrous and what could possibly be done in this case?

[+] toast0|4 years ago|reply
If you can't handle a few seconds disruption, you really need actually private networking. Dedicated lines (or at least dedicated wavelength on shared fiber) and redundancy and very fast failover.

Volumetric udp reflection isn't really too bad to process anyway, as long as you've got the bandwidth --- fancy tricks get you from the UDP stack dropping useless packets to dropping useless packets without the UDP stack, possibly at the edge without using up nearly as much internal bandwidth.

Where it gets pretty hard to manage would be application level bursts, IMHO.

[+] krebsonsecurity|4 years ago|reply
CF: Would it be asking too much to have a date and time stamp on your blog posts somewhere?
[+] dmd|4 years ago|reply
It's right below the title, where you'd expect it.
[+] IYasha|4 years ago|reply
Whatever, guys... Nothing, NOTHING will make me think better of CloudFlare. I won't forgive you, CF, for captcha, tracking and blocking me from accessing a critical server from an airport! Burn in hell!