375 Million pageviews for $15 a month(hubcitylabs.org)
hubcitylabs.org
375 Million pageviews for $15 a month
http://hubcitylabs.org/375-million-pageviews-for-15-a-month/
12 comments
In short, if your site is mostly or entirely static after the initial creation of a page, setup Varnish (or similar reverse proxy / web accelerator) to cache and serve your pages at a timeout rate you find acceptable.
This is a more in-depth article with almost the exact same information that was previously on HN a few months ago.
http://www.ewanleith.com/blog/900/10-million-hits-a-day-with...
http://www.ewanleith.com/blog/900/10-million-hits-a-day-with...
Of course the bandwidth bill will be a bit more than $15 for 375MM pageviews.
This is a pretty fluffy article, and a pretty linkbaity title.
500,000 pageviews/hr does NOT equal 375 Million pageviews/month. Traffic comes in bursts, and has peaks and valleys.
500,000 pageviews/hr does NOT equal 375 Million pageviews/month. Traffic comes in bursts, and has peaks and valleys.
Exactly. I am guessing you would need at least 4 - 5 times the pageview/hr to handle a website with 375 Million PV/M load.
People forget how much we did with servers that had less horsepower than that ten years ago. Computers these days are really powerful.
I agree. I think there's a balance between just throwing hardware at a problem versus trying to squeeze every bit of juice out of every box. I'm cheap as hell and actually like a bit of ops, so I'm more towards the squeezing side.
I've seen people install 4-6 webservers where 2 would have easily handled the load because they didn't want to optimize. I realize it's a viable strategy, but it makes me cringe...
I've seen people install 4-6 webservers where 2 would have easily handled the load because they didn't want to optimize. I realize it's a viable strategy, but it makes me cringe...
Honest questions: Don't Micro instances get throttled under sustained demand? And aren't they lowest priority on the host?
I think what gets throttled is the CPU, so if your instance isn't consuming too much CPU then you're fine. I believe that because he's using Varnish and a cache plugin he's not hitting the DB often, thus serving static content most of the time.
It's funny that I'm seeing this today, since just yesterday I was reading about varnish. I've been given the task of scaling up my first big Wordpress site for a client, which has been running pretty slowly -- it's been getting about 3 million uniques per month. I'm thinking the nginx-varnish setup described in the article that appeared on HN a few months ago (linked to elsewhere in this thread) may be the best way to go, along with throwing some more hardware at it. Does anybody have any additional thoughts about this process for a noob like me?
[deleted]
Seeing this (supporting over 10 million hits a day on an Amazon EC2 micro instance), it makes less and less sense to host a site entirely on Amazon S3.
The advantage of hosting a site entirely on S3 is the minimal administrative overhead. You don't have to configure, run, maintain, update and secure a server.
It depends heavily on your needs. Some top-of-my-head pros/cons:
EC2 Pros:
* Cheaper
* Easier to handle infrequently-updated content
* (Possibly) a simpler application pipeline
EC2 Cons:
* Have to administer a server
* Not durable, have to handle backups
* Worse availability (S3 goes down far less frequently than servers)
That said, if I'm reading it right 10M hits/day on S3 will cost you $100/day (edit: as aidos noted, I'm high by a factor of 10 here) in hits alone, so the price difference here obviously is not small.
EC2 Pros:
* Cheaper
* Easier to handle infrequently-updated content
* (Possibly) a simpler application pipeline
EC2 Cons:
* Have to administer a server
* Not durable, have to handle backups
* Worse availability (S3 goes down far less frequently than servers)
That said, if I'm reading it right 10M hits/day on S3 will cost you $100/day (edit: as aidos noted, I'm high by a factor of 10 here) in hits alone, so the price difference here obviously is not small.
I think you're a multiple out - it'll cost you $10 / day; still not to be sniffed at if you're trying to keep costs down. You pay the same for bandwidth with either solution.
We used s3 for a flash sale that even big boxes (with varnish) couldn't cope with. In our testing s3 didn't even blink (35,000 req/second). Real severs can't come close to that for the same price.
We used s3 for a flash sale that even big boxes (with varnish) couldn't cope with. In our testing s3 didn't even blink (35,000 req/second). Real severs can't come close to that for the same price.
Indeed I am, that's what I get for envelope math.
I think there's a price/performance sweet spot where a small server with something like varnish makes a lot of sense. Move outside of it in terms of hits/day or acceptable cost/hit and S3 gets a lot more attractive, especially when you can slap CloudFront in front of it so easily.
Actually, looking at the CloudFront pricing, unless you plan on doing a lot of cache invalidations it comes out much cheaper than S3.
I think there's a price/performance sweet spot where a small server with something like varnish makes a lot of sense. Move outside of it in terms of hits/day or acceptable cost/hit and S3 gets a lot more attractive, especially when you can slap CloudFront in front of it so easily.
Actually, looking at the CloudFront pricing, unless you plan on doing a lot of cache invalidations it comes out much cheaper than S3.
Cloudfront in front of ec2 would be cheaper (you'd potentially pay a lot to have it in front of s3). Also, we discovered the hard way that cloudfront is nowhere near as resilient as s3 under load (and if you need to do a cache invalidation you are pretty much screwed).
Can you expand on the resiliency of cloudfront? We've had great success after switching to it for hosting of map tiles, but possibly haven't pushed it hard enough yet. (I pushed it hard, but defining "enough" is very, very tricky)
Cloudfront in front of S3 isn't particularly expensive. Invalidations aside they're billed the same way and you're paying less per bit|request.
Invalidations, however, are a bitch. We've had a couple of times that we put up a map, found a problem with it, then decided to wait out the CF cache rather than pay for a bajillion invalidations.
Cloudfront in front of S3 isn't particularly expensive. Invalidations aside they're billed the same way and you're paying less per bit|request.
Invalidations, however, are a bitch. We've had a couple of times that we put up a map, found a problem with it, then decided to wait out the CF cache rather than pay for a bajillion invalidations.
I have to be honest, I can't remember how much load we tried putting on cloudfront during our testing. We may have just assumed that as cf is a regular cdn and s3 is a big one of a kind data store, s3 was going to be more resilient. We did load test s3 and it scaled linearly (with some caveats).
For our use case (a single huge burst of requests) we were willing to drop all caching and pay the additional money. It was absolutely worth it (and under non-sustained load it didn't cost much).
We actually used s3 as a sort of cache in front of realtime data. So the client js was accessing objects on s3 that were being refreshed every second. It worked incredibly well where other things (like gae, who disabled our account under load) had failed us in the past.
For our use case (a single huge burst of requests) we were willing to drop all caching and pay the additional money. It was absolutely worth it (and under non-sustained load it didn't cost much).
We actually used s3 as a sort of cache in front of realtime data. So the client js was accessing objects on s3 that were being refreshed every second. It worked incredibly well where other things (like gae, who disabled our account under load) had failed us in the past.
[deleted]
We should sell this bumper sticker: DISQUS, doing the hard work so you can scale your static site.
It's true, but I guess most people here know that.