Ask HN: I want to put my desktop computers in the cloud.
3 comments
patio11 (as usual) said it best. There is a lot more going on behind the scenes of a professional datacenter than you are aware of, and even if you were aware of it, you might not be able to replicate it.
The hardware they use is different. While you use desktop-class drives, they use enterprise class drives with faster platters, bigger buffers and better disk controllers.
While you use Marvell/Broadcom NICS which crash if you torrent too heavily, they use Intel server-class NICS which support TCP/IP processing offload. Which means they process network traffic in hardware.
While you connect everything with an 8-port NEXXT switch you got for $20, they connect everything with $20,000 Cisco / HP switches. While you have a 15 mbps asymmetric connection with dynamic IP from Comcast, they have multiple 1gbps feeds from Level3 and Global Crossing. They run BGP between themselves and their upstream providers, so their IP addresses don't change even if those providers were wiped off the map.
Most importantly? They have huge teams of highly paid, highly qualified engineers building and monitoring these systems. Oh; and they still have outages.
Progressing is futile.
I'm a former sysadmin for a Big-4 accounting firm. Take it from a fellow Pythonista: your time is much better spent at the '>>>' prompt than at the 'root@localhost#' prompt.
The hardware they use is different. While you use desktop-class drives, they use enterprise class drives with faster platters, bigger buffers and better disk controllers.
While you use Marvell/Broadcom NICS which crash if you torrent too heavily, they use Intel server-class NICS which support TCP/IP processing offload. Which means they process network traffic in hardware.
While you connect everything with an 8-port NEXXT switch you got for $20, they connect everything with $20,000 Cisco / HP switches. While you have a 15 mbps asymmetric connection with dynamic IP from Comcast, they have multiple 1gbps feeds from Level3 and Global Crossing. They run BGP between themselves and their upstream providers, so their IP addresses don't change even if those providers were wiped off the map.
Most importantly? They have huge teams of highly paid, highly qualified engineers building and monitoring these systems. Oh; and they still have outages.
Progressing is futile.
I'm a former sysadmin for a Big-4 accounting firm. Take it from a fellow Pythonista: your time is much better spent at the '>>>' prompt than at the 'root@localhost#' prompt.
"Fast by current standards" is probably irrelevant if you're running a scraper: You'll be bandwidth-limited, and if you solve that you'll likely be disk I/O-limited or RAM-limited.
Getting back to the overall question: Yes, lots of us have done this before; fifteen years ago every geeky college student had your proposed setup.
The other responses are correct: From a business standpoint, you're crazy. Not only will you find that bandwidth and power cost you more at home than they do at a datacenter, but your inconsistent, unreliable homebrew infrastructure will consume maintenance time and replacement parts at a rate that slowly but surely eliminates all "savings". It's hard to appreciate that, because you've got little experience at keeping network infrastructure running long-term, and even less experience at billing yourself for your own work at market rates, so failure events feel rare and inexpensive to you. But I assure you that NOCs don't buy all that expensive capital equipment for no reason.
Having said that, I get the impression that you don't really care about uptime or reliability; you're running spare-time hobby projects. If so, then the question is whether assembling your own homebrew "distributed datacenter" out of PCs in various closets is educational. My feeling is that it isn't, not anymore. The ability to run production-quality systems on cheap consumer hardware is not a valuable skill and is no longer worth practicing - not when production hardware can be rented by the hour for pennies. You can practice installing Linux and tuning the LAMP stack on Linodes or AWS boxes and get all the same sysadmin skills, but in a realistic production environment. You can practice debugging disk throughput problems, or networking problems, or DNS headaches on real cloud infrastructure for a few bucks a day. Who would play with a toy bulldozer if a real bulldozer was available to rent for a dollar per hour?
(The counterargument is that homebrewed "infrastructure" is a great toy because it breaks all the time and you're forced to practice lots and lots of debugging. After all, I learned a lot of stuff this way - I had to; the "cloud" didn't exist. But my gut feeling is that this kind of "practice" feels more valuable than it actually is.)
But of course, hobbies are hobbies, and far be it from me to argue you out of your own happiness. If you just can't resist the fun of turning your home computers into a compute cluster, go for it! It's a better way to spend your time than MMORPGs.
Getting back to the overall question: Yes, lots of us have done this before; fifteen years ago every geeky college student had your proposed setup.
The other responses are correct: From a business standpoint, you're crazy. Not only will you find that bandwidth and power cost you more at home than they do at a datacenter, but your inconsistent, unreliable homebrew infrastructure will consume maintenance time and replacement parts at a rate that slowly but surely eliminates all "savings". It's hard to appreciate that, because you've got little experience at keeping network infrastructure running long-term, and even less experience at billing yourself for your own work at market rates, so failure events feel rare and inexpensive to you. But I assure you that NOCs don't buy all that expensive capital equipment for no reason.
Having said that, I get the impression that you don't really care about uptime or reliability; you're running spare-time hobby projects. If so, then the question is whether assembling your own homebrew "distributed datacenter" out of PCs in various closets is educational. My feeling is that it isn't, not anymore. The ability to run production-quality systems on cheap consumer hardware is not a valuable skill and is no longer worth practicing - not when production hardware can be rented by the hour for pennies. You can practice installing Linux and tuning the LAMP stack on Linodes or AWS boxes and get all the same sysadmin skills, but in a realistic production environment. You can practice debugging disk throughput problems, or networking problems, or DNS headaches on real cloud infrastructure for a few bucks a day. Who would play with a toy bulldozer if a real bulldozer was available to rent for a dollar per hour?
(The counterargument is that homebrewed "infrastructure" is a great toy because it breaks all the time and you're forced to practice lots and lots of debugging. After all, I learned a lot of stuff this way - I had to; the "cloud" didn't exist. But my gut feeling is that this kind of "practice" feels more valuable than it actually is.)
But of course, hobbies are hobbies, and far be it from me to argue you out of your own happiness. If you just can't resist the fun of turning your home computers into a compute cluster, go for it! It's a better way to spend your time than MMORPGs.
why am I not just using these to do [almost] everything?
Because their true cost once you factor in your time is substantially higher than racked infrastructure and because the reliability which you would achieve with that setup is not sufficient for professional purposes.
Because their true cost once you factor in your time is substantially higher than racked infrastructure and because the reliability which you would achieve with that setup is not sufficient for professional purposes.
On one hand, the answer is simple: my Comcast 15mbit connection is far too slow. The other quick answer is security: I do not transmit personally sensitive data, so the normal privacy issues old-school security engineers have with the cloud aren't there for my use case.
Ok, so that got me to start thinking about how to put my capital to productive use. The batch processes and scraper type work could easily be done in house. 4 of the 6 desktop computers I would like to put to work are actually fast by current standards (8GB+ DDR3, 2 are i7 2700k, etc). I mean, if I were to rent the same machines "in the cloud", it would cost me a significant chunk of cash.
Ah, but that leads to the other cost: energy. Getting estimates of those costs is pretty simple to do. So, this is where I started thinking about how to actually solve my problem [which is more of an itch than a problem]. Upgrade my Internet? Or, more interestingly, asking friends who have faster Internet already if I can stuff a desktop somewhere in their house and just set up SSH access to it.
I'm beginning to ask friends if they'd be willing to let me subsidize their Internet/utility costs if they'll let me put a computer in their house.
Has any one else done this before? Any input or discussion here would be useful. Thanks.