BBC iPlayer serves nearly 1000 requests per second with Catalyst(bbc.co.uk)
bbc.co.uk
BBC iPlayer serves nearly 1000 requests per second with Catalyst
http://www.bbc.co.uk/blogs/bbcinternet/2008/12/iplayer_day_performance_tricks.html
7 comments
Since someone suggested in the YouPorn job thread that YouPorn might be the largest Catalyst, Django, or RoR site in the world, I figured this one might also be interesting.
These stats are taken before the busiest period for the iPlayer (over Christmas) so I imagine they doubled the statistics for christmas day.
The BBC also created their own framework which they named Perl on Rails to adapt between Ruby on Rails which is used for their internal web apps and the perl code base of the main site. http://www.bbc.co.uk/blogs/radiolabs/2007/11/perl_on_rails.s...
The BBC also created their own framework which they named Perl on Rails to adapt between Ruby on Rails which is used for their internal web apps and the perl code base of the main site. http://www.bbc.co.uk/blogs/radiolabs/2007/11/perl_on_rails.s...
Serving global content is easy to scale, since it's very easy to cache and easy to distribute across a cluster of computers. The real challenge comes when you have to serve and generate different content for different users and you can't use a global cache.
It sounds to me that they are serving global content - - and doing this, 1000 req. pr. sec is not that impressive and almost any framework tied to memcached could do this - - or even better, setup nginx to use the memcached module.
It sounds to me that they are serving global content - - and doing this, 1000 req. pr. sec is not that impressive and almost any framework tied to memcached could do this - - or even better, setup nginx to use the memcached module.
I don't think anybody is suggesting it's a miracle, or only possible with Catalyst. Just an interesting story of scaling a dynamic site using off-the-shelf Open Source components.
You could do this kind of serving in Ruby on Rails, Django or any other framework. Nothing really interesting, other than it's Catalyst - which for me does not make this any more interesting - - especially also since they use memcached caching to take down the load.
> "Our database is flattened, denormalised and heavily indexed to make sql queries as quick as possible"
very interesting quote. at least for me.
very interesting quote. at least for me.
I wonder if its a true STAR schema (not that they are doing data warehouse-type queries, just curious)
> Apache2.2/OOperl/Catalyst/memcached
What is OOperl?
What is OOperl?
I'm not certain, but I think it's their way of saying, "We don't have any of that crufty Perl you used to see on those awful 'hotscripts' type sites. We only use the good stuff around here."
Because Perl is a multi-paradigm language (supporting OO, functional, and procedural, and most Perlmongers use a little of all three in any given program), some folks consider it worth mentioning it when the paradigm they use most is OO. I'm a little suspicious when I see explicit mention of OO Perl, as I have to wonder if that means I couldn't use functional Perl idioms (which are often quite nice) if I worked in their shop.
Because Perl is a multi-paradigm language (supporting OO, functional, and procedural, and most Perlmongers use a little of all three in any given program), some folks consider it worth mentioning it when the paradigm they use most is OO. I'm a little suspicious when I see explicit mention of OO Perl, as I have to wonder if that means I couldn't use functional Perl idioms (which are often quite nice) if I worked in their shop.
Ah, right. I picked up the OO part from them using Catalyst, so I thought mentioning it as part of their stack meant they used something other than mod_perl.
Most Catalyst deployments are on FastCGI nowadays. Its nice that Catalyst doesn't much care how you deploy it: built-in test server/CGI/mod_perl/FastCGI all 'just work.'
Some more interesting info here..
http://www.bbc.co.uk/blogs/bbcinternet/2008/12/introducing_i...
http://www.bbc.co.uk/blogs/bbcinternet/iplayer/
http://www.bbc.co.uk/blogs/bbcinternet/2008/12/introducing_i...
http://www.bbc.co.uk/blogs/bbcinternet/iplayer/
Here is a talk by Cory Doctorow mentioning why the DRM in the BBC iPlayer is retarded: http://www.boingboing.net/2009/04/15/my-drm-and-ebooks-ta.ht...