Yes or no: Should web startups host on Amazon?
32 comments
I think Amazon should be part of your hosting strategy, but not the entirety of it.
We use Amazon (EC2, S3, SQS) but only where appropriate. EC2 for compute intensive but ephemeral data (logging, etc since your storage can disappear). S3 for hosting data files served up to HTTP clients (images, video, music) and SQS for, well, queuing :)
Most importantly though, we host the primary application in a typical data center. It's reliable, supported, we have an SLA and we can expand in any way we need to.
Like I said, use Amazon for part of your hosting strategy but not the entirety of it.
We use Amazon (EC2, S3, SQS) but only where appropriate. EC2 for compute intensive but ephemeral data (logging, etc since your storage can disappear). S3 for hosting data files served up to HTTP clients (images, video, music) and SQS for, well, queuing :)
Most importantly though, we host the primary application in a typical data center. It's reliable, supported, we have an SLA and we can expand in any way we need to.
Like I said, use Amazon for part of your hosting strategy but not the entirety of it.
During the Q&A period at a talk by Jeff Barr, the Amazon Web Services evangelist, a venture capitalist made an insightful remark. He said how they had given a company $4 million in financing. $1 million of that went to purchase/lease server hardware and storage.
Amazon.com's web services flips the traditional startup/venture capital financing paradigm. How?
#1 You can test out your business model on a small scale without investing heavily in infrastructure. Why build it before they come? Is your business model working? Is it profitable? You can find this out. As a side effect, the businesses built on amazon.com's web services may be more self-sustaining than one that is simply siphoning off from a pool of initial capital funding .
#2 You can scale as needed with EC2 and virtual machines. EC2 lets you go from 10 to 100 to 1000 virtual machines in real-time, all running your virtual machine image. The caveat is: you need to learn how to build the right virtual machines and manage your virtual machines. Jeff made a remark about how erlang and map-reduce (both esoteric languages that deal with managing scalability) will be valuable skills in probably just a year.
#3 Your costs can go down. Why? Moore's law and whatever law/maxim regulates storage. As storage costs go down, Amazon.com can transfer on the savings to you - you can actually pay less per gigabyte of storage in 6 months. Compare that to having storage hardware that steadily depreciates and heads to slow obsolescence.
Amazon.com's web services flips the traditional startup/venture capital financing paradigm. How?
#1 You can test out your business model on a small scale without investing heavily in infrastructure. Why build it before they come? Is your business model working? Is it profitable? You can find this out. As a side effect, the businesses built on amazon.com's web services may be more self-sustaining than one that is simply siphoning off from a pool of initial capital funding .
#2 You can scale as needed with EC2 and virtual machines. EC2 lets you go from 10 to 100 to 1000 virtual machines in real-time, all running your virtual machine image. The caveat is: you need to learn how to build the right virtual machines and manage your virtual machines. Jeff made a remark about how erlang and map-reduce (both esoteric languages that deal with managing scalability) will be valuable skills in probably just a year.
#3 Your costs can go down. Why? Moore's law and whatever law/maxim regulates storage. As storage costs go down, Amazon.com can transfer on the savings to you - you can actually pay less per gigabyte of storage in 6 months. Compare that to having storage hardware that steadily depreciates and heads to slow obsolescence.
YES: We're using EC2/S3 24/7 for our startup and extremely happy with it. Our cost right now is $70 per month, but this will go up as we get more users, which is fine as we are running a subscription model. The storage costs are so low, we are going to offer a huge amount of data storage for our users. There is no cost to move data between EC2 and S3. We now don't worry at all about our hardware, and focus totally on our product and business. We were originally hosting at our office, but the bandwidth alone would have cost more per month.
Do you have any issues with instances crashing or DNS problems? How long have you been using it? I'm considering going this way also but I'm not very strong with linux admin skillz.
We are now into our second month with the service still running the same lisp process (which we've updated countless times with the REPL). So far - 0 problems. We have in place contingencies for going down by having realtime backup of our data so that if an instance were to crash, we could boot up another within a few minutes with no data loss. In that case, we would email our customers with the direct ip address to login until the dns updates. But I'm guessing this will be a very rare event. So the worst case we are envisioning is about 5 minutes of downtime for our customer in a crash, unless all of Amazon goes down which I don't think has ever happened?
Given Amazon's extensive data center experience with their own products, I have a lot more trust that their service will stay up vs. a smaller hosting company. And the worst case with them is minutes vs. hours or days with smaller providers. (How many famous sites going down with smaller hosts have hit the news in the last 6 months...)
For Ubuntu, we're using:
Amazon EC2 Ubuntu 7.10 gutsy AMI built using code customized by Eric Hammond <[email protected]> For more information: http://ec2gutsy.notlong.com
If you go for it, I recommend having someone who has a reasonable knowledge of linux avaialable for asking questions or helping you directly, at least to set it up. My linux experience was 0 when we started. Now I can get done what I need to do - but my cofounder is the expert here and guided me on the commands I needed to work with the system. We also set up ssh session that would directly get us in the cloud and then connect to the lisp instance in real time with the REPL through emacs, etc.
We basically had a local box in our office all set up first, and then when our internet connectivity in the office was insufficient, we put it on the cloud. But since it is easy to play around with EC2 and start and stop instances, its probably just as easy to experiment on the cloud as you get things set up.
Given Amazon's extensive data center experience with their own products, I have a lot more trust that their service will stay up vs. a smaller hosting company. And the worst case with them is minutes vs. hours or days with smaller providers. (How many famous sites going down with smaller hosts have hit the news in the last 6 months...)
For Ubuntu, we're using:
Amazon EC2 Ubuntu 7.10 gutsy AMI built using code customized by Eric Hammond <[email protected]> For more information: http://ec2gutsy.notlong.com
If you go for it, I recommend having someone who has a reasonable knowledge of linux avaialable for asking questions or helping you directly, at least to set it up. My linux experience was 0 when we started. Now I can get done what I need to do - but my cofounder is the expert here and guided me on the commands I needed to work with the system. We also set up ssh session that would directly get us in the cloud and then connect to the lisp instance in real time with the REPL through emacs, etc.
We basically had a local box in our office all set up first, and then when our internet connectivity in the office was insufficient, we put it on the cloud. But since it is easy to play around with EC2 and start and stop instances, its probably just as easy to experiment on the cloud as you get things set up.
Wow, thanks for the reply. That's enough encouragement for me to give it a shot. Also, good point about the reliability of Amazon vs. that of a no-name hosting service. Although EC2 is still in beta and not quite the same as Amazon in general, it still gives more credibility to their service.
what's your stack look like?
Ubuntu - Apache - AllegroServe - MySQL - SBCL (Lisp)
Have you culled any extraneous processes from the Ubuntu image you used?
I don't think so. We used: Amazon EC2 Ubuntu 7.10 gutsy AMI built using code customized by Eric Hammond <[email protected]> For more information: http://ec2gutsy.notlong.com
I love the idea of variable increases in cost as resource requirements increase. I massively over-deployed in the dot-com era. Oops.
How have S3/EC2 affected funding and acquisitions - is anyone aware of an example of either? (I know jtv uses aws)
How have S3/EC2 affected funding and acquisitions - is anyone aware of an example of either? (I know jtv uses aws)
I would recommend developing your application so that it can easily scale horizontally. Run the main application on a box at a regular hosting provider and have an account on Amazon EC2 which you can use to rapidly create hosts when your load increases (and then shut them off when the load is back down). This way you can keep your server up 24x7 without paying the price of EC2 but still have the option to scale easily.
There are two things that might swing us into having everything entirely hosted at Amazon.
1) If we need to store large amounts of data at S3 and our app constantly needs to manipulate it, sitting between the user and the data rather than simply linking to it.
2) If we launch an app that has a high dependency on Amazon Simple DB.
In either of these cases we would get a significant performance increase from having always on EC2 instances for the app servers. In the meantime SliceHost continues to serve us well for a fraction of the cost.
1) If we need to store large amounts of data at S3 and our app constantly needs to manipulate it, sitting between the user and the data rather than simply linking to it.
2) If we launch an app that has a high dependency on Amazon Simple DB.
In either of these cases we would get a significant performance increase from having always on EC2 instances for the app servers. In the meantime SliceHost continues to serve us well for a fraction of the cost.
It depends on your growth prospects. If you're not going to have 10,000 active users from day one, I'd stick with something cheap and simple like slicehost. It should last you a surprisingly long time and save you a lot of annoying configuration.
I just went through scaling my app for EC2 and it took a good week to get it all worked out. By comparison, setting up one instance on slicehost takes just a couple hours.
I just went through scaling my app for EC2 and it took a good week to get it all worked out. By comparison, setting up one instance on slicehost takes just a couple hours.
Modulus (http://www.mogulus.com/) runs completely off EC2 IIRC. I know at least if you ping their www server it brings pack an EC2 IP.
http://scobleizer.com/2007/11/16/the-serverless-internet-com...
http://scobleizer.com/2007/11/16/the-serverless-internet-com...
We've been going back and forth on whether or not to use S3 and EC2.
At the moment we're only planning on using EC2 for computationally expensive tasks like generating thumbnails, PDFs, and that sort of thing.
We might use S3 for storage of large files, but probably not small frequently accessed ones because of the per-request charges and latency.
We're going to try to build an architecture that's not tied to one particular hosting method... we could start out on EC2 and if it doesn't work out then move to dedicated boxes. And even then if there's a particularly heavy load we could fire up additional EC2 instances.
The nice thing about EC2 is it's pretty easy to experiment. There's no big up-front costs with setting it up, so there's not too much to lose by testing.
At the moment we're only planning on using EC2 for computationally expensive tasks like generating thumbnails, PDFs, and that sort of thing.
We might use S3 for storage of large files, but probably not small frequently accessed ones because of the per-request charges and latency.
We're going to try to build an architecture that's not tied to one particular hosting method... we could start out on EC2 and if it doesn't work out then move to dedicated boxes. And even then if there's a particularly heavy load we could fire up additional EC2 instances.
The nice thing about EC2 is it's pretty easy to experiment. There's no big up-front costs with setting it up, so there's not too much to lose by testing.
maybe
Clearly the proper answer. Depends what you're doing.
highest karma-per-word comment ever :)
close, but I think this one wins the per-word and per-letter contest.
http://news.ycombinator.com/item?id=110034
http://news.ycombinator.com/item?id=110034
lol
Yes, ynot?
the only reason i have seen to avoid amazon as your hosting platform is if you are heavily database dependent. you can design around that if you can think a little bit around the lamp stack model.
for us our largest challenge will be how do we webscale the site.
our assumptions are dynamic dns and round robin load balancing will get us to split any dns record to up to 5 machines reasonably amazon now allows us to scale vertically to 8 core VM with 16gb of ram *this should get us to a point where we are either making money, or getting a good funding round out of someone.
the only reason i have seen to avoid amazon as your hosting platform is if you are heavily database dependent. you can design around that if you can think a little bit around the lamp stack model.
for us our largest challenge will be how do we webscale the site.
our assumptions are dynamic dns and round robin load balancing will get us to split any dns record to up to 5 machines reasonably amazon now allows us to scale vertically to 8 core VM with 16gb of ram *this should get us to a point where we are either making money, or getting a good funding round out of someone.
From my (recent) research, I would say 'not primairly' I agree with bmatheny that using it for some things may be a good idea (if video-to-flash conversion is eating up resources, EC2 is good, if you're running out of HD space, S3 is good) But, I would go with somebody else for your primary 'web hosting' needs. EC2 isn't a web-host, and for us, it came down to 'is it worth the time resources to manage DNS and another API and resturcture part of our program for it to work with AWS?' and the answer was no, but We'll probably offload Processing and Storage pretty soon.
Absolutely not as a primary host. EC2 should only be used for handling peak load, so that you don't have to build out more servers just to handle your traffic spikes. That's it. Nothing more.
Flat-fee hosting plans are much more reasonable considering the bandwidth costs. For example, the virtualized hosting plan at Media Temple serves 1Tb of data for $50 per month.
If you can justify the cost of the logs/data upload, EC2 is be a good way to run stats or other sorts of heavy-duty batch processing. So use EC2 only if you keep hitting ceilings on processing / memory consumption on your hosting plan.
If you can justify the cost of the logs/data upload, EC2 is be a good way to run stats or other sorts of heavy-duty batch processing. So use EC2 only if you keep hitting ceilings on processing / memory consumption on your hosting plan.
To start, it's not a bad option but once a company gets some momentum, then they probably need to look for a "real" host.
Yes! We have the production server on EC2 and the development one runs on a virtual server from Spry. From our load tests EC2 can handle about 6 times more site usage.
2 downsides are, you don't own the static ip address, and you have to get your own data and database backed up to S3 instead of the webhost handling backups.
Useless.
There's no answer without describing what you're hosting.
[edit]If the question is should you consider it, absolutely.[/edit]
There's no answer without describing what you're hosting.
[edit]If the question is should you consider it, absolutely.[/edit]
if it works for you then yes, just have a plan b for the future just incase it goes south.
My (minimal, but real-world) experience shows that for an app that runs fairly continuously, EC2 can end up costing 1x-3x as much as a simple dedicated server with similar resources.
EC2 is great if you need the computational power of 2 or 10 machines for a few hours a day (like to chew through and analyze a bunch of log files, or to process some batch of data nightly), or you need a temporary increase in server resources for a day or two (like if you have a popular link from digg).
EC2 isn't such a great fit if you need a 24/7 always-on server that's doing tasks continuously (even at low volume).