Netflix Denominator – Portably control DNS clouds using Java or bash(github.com)
github.com
Netflix Denominator – Portably control DNS clouds using Java or bash
https://github.com/Netflix/denominator
4 comments
Why do companies with such large amount of machines all over the world not run their own DNS service?
DNS resilience is best served by diversity at every level of implementation. That includes diversity beyond your own resources. Somewhat topically, DNS services for high profile entities are DDoS targets; there are very few entities that can absorb incoming hundreds of Gbps without the service collapsing.
The emergent downside of sharing the eggbasket is that of collateral damage i.e. a successful DDoS intended for one target also cripples many others.
In Netflix's case, their CDN infrastructure is all about video delivery and is almost certainly optimised for outbound bandwidth and video QoS. Engineering to absorb an inbound DDoS is expensive, and involves compromises. By having DNS elsewhere, they've avoided the basket sharing and created new opportunities for mitigation. That's an example of how diversity systematically enables resilience.
In the worst case ops scenario, having to cold-boot an entire platform from down, then it's pretty much essential to have directory services as independent infrastructure. The larger you get, the more weird interdependencies arise if you don't take architectural steps to avoid them. This can apply to migrations of infrastructure also. Not 100% necessary, but sooo much easier if so.
The emergent downside of sharing the eggbasket is that of collateral damage i.e. a successful DDoS intended for one target also cripples many others.
In Netflix's case, their CDN infrastructure is all about video delivery and is almost certainly optimised for outbound bandwidth and video QoS. Engineering to absorb an inbound DDoS is expensive, and involves compromises. By having DNS elsewhere, they've avoided the basket sharing and created new opportunities for mitigation. That's an example of how diversity systematically enables resilience.
In the worst case ops scenario, having to cold-boot an entire platform from down, then it's pretty much essential to have directory services as independent infrastructure. The larger you get, the more weird interdependencies arise if you don't take architectural steps to avoid them. This can apply to migrations of infrastructure also. Not 100% necessary, but sooo much easier if so.
In addition to these good points, assuming you also host whatever application your MegaCorp makes, you have now put all your eggs in one basket. If they can break DNS, they can (or have) probably also broken your app. Likewise, if they manage to DDoS your app, you can't change DNS records to point them elsewhere since that's also down probably. For the same reason you may use multiple hosting providers (like a nice Digital Ocean droplet for when Linode is under attack), spreading out your DNS options is also very smart.
After yesterday's attach on DynDNS many of us are asking the same thing.
I imagine it's a similar justification for using Cloudflare or another CDN.
I imagine it's a similar justification for using Cloudflare or another CDN.
Not sure if Netflix confirmed officially, but they were down during this week's Ddos attack on DynDNS. Would be interesting to know why
...because they use DynDNS's services?
Are you sure it wasn't just connectivity problems related to it? Netflix uses Route53 for DNS, not Dyn.
Yes but Amazon used Dyn for us-east-1, IIRC, so it might be related.
Are Netflix (and others) using multiple DNS providers simultaneously? If so, why?
The DDOS attack against a single DNS service provider that took down a large number of services this week is probably a pretty good example for the necessity.
I wouldn't say so:
PS D:\> (resolve-dnsname -name netflix.com -type ns).NameHost
ns-1372.awsdns-43.org
ns-1984.awsdns-56.co.uk
ns-659.awsdns-18.net
ns-81.awsdns-10.com
PS D:\> (resolve-dnsname -name netflix.com -type ns).NameHost
ns-1372.awsdns-43.org
ns-1984.awsdns-56.co.uk
ns-659.awsdns-18.net
ns-81.awsdns-10.com
To avoid this: https://news.ycombinator.com/item?id=12759697
https://github.com/Netflix/denominator/issues/374
> As much as I hate to admit it, I'm no longer a good choice to champion this project. It isn't about denominator itself, just I'm stretched too thin to continue. That's been evident in the changelog, where I've not done much in the last year. Netflix softly archived the project in early Dec 2015, but I think we should less softly do so in order to not lead people on.