The rise of multivector DDoS attacks(blog.cloudflare.com)
blog.cloudflare.com
The rise of multivector DDoS attacks
https://blog.cloudflare.com/the-rise-of-multivector-amplifications/#
4 comments
It really depends. A lot of people, myself included, use their free tier service which is excellent for the grand price of zero, however it does not really come with much real DDoS protection. Script kiddies might give up after seeing an IP address belonging to CF, but the more experienced and determined attackers will keep ramping up bandwidth until Cloudflare takes notice and cut you off because you are now more trouble than worth. Free tier breaks only with a little bit of traffic, whereas paid users have some headroom but it is not infinite. From their point of view it's still preferable to lose you as a customer than having every other customer's site lagging because of you.
Apparently even business tier is not immune, Brian Krebs' security blog was DDoS'ed off Akamai after the then-ongoing mitigation cost ended up being far more than they could agree on, and it took him days to find another provider[0].
[0]:https://krebsonsecurity.com/2016/09/the-democratization-of-c...
Apparently even business tier is not immune, Brian Krebs' security blog was DDoS'ed off Akamai after the then-ongoing mitigation cost ended up being far more than they could agree on, and it took him days to find another provider[0].
[0]:https://krebsonsecurity.com/2016/09/the-democratization-of-c...
What's not obvious from outside of Cloudflare is that DDoS attack traffic doesn't increase our costs. While someone like AWS charges based on bytes delivered, we instead pay for bandwidth based on the capacity of our connection. Importantly, we pay for the greater of the traffic into our network (ingress) or out from our network (egress).
To make it tangible with made up numbers, imagine we pay $10/megabit per second per month. If our egress (out) is 10Mbps and our ingress (in) is 1Mbps then we'd pay 10 x $10 = $100/month. If our ingress went up to 9Mbps and our egress stayed at 10Mbps then we'd still pay 10 x $10 = $100/month.
Since we're a caching proxy, you'd expect egress (out) to be higher than ingress (in). That is in fact the case. While DDoS attacks push the ingress up, the spread between ingress and egress is so large that even the largest attacks don't push ingress above egress. In fact, even attacks that are many times larger than the largest attacks ever seen would still not increase our bandwidth costs.
This is different from other providers that run separate networks for DDoS mitigation, or only provide DDoS mitigation without providing other caching services. My understanding is that Akamai runs a separate network for DDoS from their CDN, which is why large attacks drive up their costs. We made different architectural decisions, which is why it doesn't for us. And, as our caching services get more popular, it effectively increases the size of the largest theoretical attack we could handle without it driving up our bandwidth costs.
So, yes, there is a theoretical limit of an attack we could not handle today. That, however, is at least an order of magnitude bigger than the largest attacks the Internet has ever seen. And, if such an attack did happen, I think there would be other parts of the Internet that would fall over before we did.
For the record: since we announced Unmetered DDoS mitigation in September 2017 we haven't terminated any customer, free or paying, for an attack they've received.
To make it tangible with made up numbers, imagine we pay $10/megabit per second per month. If our egress (out) is 10Mbps and our ingress (in) is 1Mbps then we'd pay 10 x $10 = $100/month. If our ingress went up to 9Mbps and our egress stayed at 10Mbps then we'd still pay 10 x $10 = $100/month.
Since we're a caching proxy, you'd expect egress (out) to be higher than ingress (in). That is in fact the case. While DDoS attacks push the ingress up, the spread between ingress and egress is so large that even the largest attacks don't push ingress above egress. In fact, even attacks that are many times larger than the largest attacks ever seen would still not increase our bandwidth costs.
This is different from other providers that run separate networks for DDoS mitigation, or only provide DDoS mitigation without providing other caching services. My understanding is that Akamai runs a separate network for DDoS from their CDN, which is why large attacks drive up their costs. We made different architectural decisions, which is why it doesn't for us. And, as our caching services get more popular, it effectively increases the size of the largest theoretical attack we could handle without it driving up our bandwidth costs.
So, yes, there is a theoretical limit of an attack we could not handle today. That, however, is at least an order of magnitude bigger than the largest attacks the Internet has ever seen. And, if such an attack did happen, I think there would be other parts of the Internet that would fall over before we did.
For the record: since we announced Unmetered DDoS mitigation in September 2017 we haven't terminated any customer, free or paying, for an attack they've received.
All Cloudflare plans have unmetered DDoS protection. Where did you get the idea that they would cut you off if you were on the free tier?
Until late last year that was their policy: https://blog.cloudflare.com/unmetered-mitigation/
Also they specifically refer to volumetric mitigation' as the thing which everyone gets.
The harder to deal with attacks are probably the application aware resource attacks.
Also they specifically refer to volumetric mitigation' as the thing which everyone gets.
The harder to deal with attacks are probably the application aware resource attacks.
I've seen it happen multiple times. Besides, "unmetered" is almost always marketing hype whenever you see it. If your website uses >10TB of non-DDoS traffic per day on CF you are likely to get a call from their sales team soon asking you to upgrade to a pro or business account.
A brief DDoS was (not sure if still is) a common method to expose the real IP because the CDN edge servers could often be easily spooked by a brief surge in traffic and start redirecting DNS back to origin. I suspect this is the kind of "protection" they were really offering: your site will still be down, but at least the backend is never revealed to the world.
A brief DDoS was (not sure if still is) a common method to expose the real IP because the CDN edge servers could often be easily spooked by a brief surge in traffic and start redirecting DNS back to origin. I suspect this is the kind of "protection" they were really offering: your site will still be down, but at least the backend is never revealed to the world.
In earlier times they had to encourage people to move. By now, CF is so big that they can handle losing money on some customers with high traffic on a free plan. The gain from not dropping any customers outweighs the cost caused by that client.
To quote a description of the Joker; "Some people just want to see the world burn."
They do it for "the lulz", for fame, for glory, for breaking in their new botnet, for the brief chance of taking down someone's website... etc.
You sorta have to live with this when you host on the internet.
They do it for "the lulz", for fame, for glory, for breaking in their new botnet, for the brief chance of taking down someone's website... etc.
You sorta have to live with this when you host on the internet.
I'm curious about this too, but at least in some cases the reason is the sites don't start off with DDoS protection, they signup with Cloudflare to mitigate the attack in progress.
> I've often wondered why people would conduct attacks if the attacks don't actually end up doing anything.
Here's one good reason: to test their tooling and botnet. If you've been hitting smaller targets and taking them offline reliably, you might not even max out the bandwidth of your botnet. You have to have something that can handle the traffic to measure how high you can go. Even if not, you might just want to continue testing. It's going to be important if you're selling DDOSes.
Here's one good reason: to test their tooling and botnet. If you've been hitting smaller targets and taking them offline reliably, you might not even max out the bandwidth of your botnet. You have to have something that can handle the traffic to measure how high you can go. Even if not, you might just want to continue testing. It's going to be important if you're selling DDOSes.
At one time we considered publishing real-time DDoS information but decided against it to stop people using us to test the power of their DDoS botnets.
Most of these attacks are pretty short, around 10 minutes on the graphs. Some people are just doing this for the lols. Cloudflare is pretty open about their tech stack, so if there were significant issues, I would expect them to do mitigation work and then blog about it.
I used to see a bunch of DDoS against my work servers, it tended to be exactly 90 seconds, plus or minus a bit of clock skew in the generators. The time frames seemed to be about the same regardless of if the attack was successful or not, although when the attacks weren't successful, I am less likely to have noticed.
Usually, but not always, our www servers were targetted, and not the servers actually doing useful work, presumably because it's cooler to attack www servers. I have to say, sometimes pretty dumb DDoS would crush our servers, but it was always fun to look at the sampled tcpdumps and fix the bottlenecks.
There are a lot of DDoS for hire sites out there, so it takes less skill to get it done than checking host records.
I used to see a bunch of DDoS against my work servers, it tended to be exactly 90 seconds, plus or minus a bit of clock skew in the generators. The time frames seemed to be about the same regardless of if the attack was successful or not, although when the attacks weren't successful, I am less likely to have noticed.
Usually, but not always, our www servers were targetted, and not the servers actually doing useful work, presumably because it's cooler to attack www servers. I have to say, sometimes pretty dumb DDoS would crush our servers, but it was always fun to look at the sampled tcpdumps and fix the bottlenecks.
There are a lot of DDoS for hire sites out there, so it takes less skill to get it done than checking host records.
> presumably because it's cooler to attack www servers
Why would you have anything other than web servers directly accessible on the internet?
Why would you have anything other than web servers directly accessible on the internet?
The internet is more than just web servers. Web servers are more than just www.
I work for a chat service, the servers doing the actual work of the service were rarely attacked, we even have a couple other http(s) endpoints that are public, but not www, and those were rarely attacked as well.
I work for a chat service, the servers doing the actual work of the service were rarely attacked, we even have a couple other http(s) endpoints that are public, but not www, and those were rarely attacked as well.
Thanks for this detailed article, for someone like me interested in Ddos this is super interesting, Cloudflare network and expertise is really impressive! Wow
shimnasuresh(1)
I've often wondered why people would conduct attacks if the attacks don't actually end up doing anything.
The fact that somebody is protected by a DDoS mitigation service should be evident to most of the people capable of conducting an attack.