Ask HN: Max no. of EC2 instances used by a single web app?
12 comments
Some other data points:
* Animoto was running several thousands of machines in 2008 (http://bit.ly/EDLtt)
* Litmus runs 400 servers (http://bit.ly/d7Hc7y)
* 99Designs runs entirely on EC2 (http://bit.ly/aotKgg)
I've personally had long-running EC2 instances with uptimes in years. You could do it cheaper in terms of hardware, but at the cost of wasting time at the colo while you could be building cool shit.
* Animoto was running several thousands of machines in 2008 (http://bit.ly/EDLtt)
* Litmus runs 400 servers (http://bit.ly/d7Hc7y)
* 99Designs runs entirely on EC2 (http://bit.ly/aotKgg)
I've personally had long-running EC2 instances with uptimes in years. You could do it cheaper in terms of hardware, but at the cost of wasting time at the colo while you could be building cool shit.
[i got permissions from Lachlan Donald at 99 Designs to post his email to me here - jm3]
We [at 99 Designs] actually don't have a huge number of instances, although we are steadily adding more and more.
We have at present:
We also use RightScale for management and are at present looking at using server arrays to scale up and down the App Server layer to deal with load spikes.
The biggest thing that reduced our need for lots of servers was moving as much functionality to asynchronous tasks as possible, which are queued on each appserver via Beanstalk queues and then consumed by the worker servers. Additionally we have as many pages http cache-able as we can, which are served directly from the squid cache at the front of the app.
Hope that helps!
Cheers, Lachlan Donald CTO, 99designs
We [at 99 Designs] actually don't have a huge number of instances, although we are steadily adding more and more.
We have at present:
2 x Caching Proxies (Large)
8 x App Servers (Large)
3 x DB Servers (High-CPU X-Large)
2 x Workers (Large)
We also use RightScale for management and are at present looking at using server arrays to scale up and down the App Server layer to deal with load spikes.
The biggest thing that reduced our need for lots of servers was moving as much functionality to asynchronous tasks as possible, which are queued on each appserver via Beanstalk queues and then consumed by the worker servers. Additionally we have as many pages http cache-able as we can, which are served directly from the squid cache at the front of the app.
Hope that helps!
Cheers, Lachlan Donald CTO, 99designs
Good start - I wasn't aware of the Litmus environment. The Animoto example is great, I love telling that story to people just starting to look into cloud computing.
To add to the large-environment roll call, here are the persistent server counts of some 100% cloud-hosted companies:
- Bizo: 100+ instances (I work here)
- Reddit: 100(?) instances ("256 Virtual CPUs" http://us.pycon.org/2010/conference/schedule/event/148/) - HNer jedberg runs this
- ShareThis: 250 instances (I worked here)
Longest-running instance I've had is at two years and still going strong.
To add to the large-environment roll call, here are the persistent server counts of some 100% cloud-hosted companies:
- Bizo: 100+ instances (I work here)
- Reddit: 100(?) instances ("256 Virtual CPUs" http://us.pycon.org/2010/conference/schedule/event/148/) - HNer jedberg runs this
- ShareThis: 250 instances (I worked here)
Longest-running instance I've had is at two years and still going strong.
reddit currently has 73 instances with this breakdown:
25 c1.xlarge
26 m1.large
22 m1.xlarge
Zynga runs over 12,000 EC2 nodes total. A single app like FarmVille, which runs entirely on EC2, will be a few thousand. It's safe to guess we're Amazon's biggest customer.
You're not their biggest customer, but you're up there. :)
Does your number of instances scale up and down frequently, or does it monotonically increase with growth?
What size instances do you run for different types of functions?
I've got two running now. A large for the webserver and a medium high CPU for the database. Roughly a couple hundred reqests per second.
I’ll go first: we (140 Proof) actually have 20 instances, not 18. How many instances do other people’s apps use?
Curious to hear from the HN community: what’s the most long-running EC2 instances you've seen in a successful web app?