Youporn.com is now a 100% Redis Site(groups.google.com)
groups.google.com
Youporn.com is now a 100% Redis Site
https://groups.google.com/forum/?fromgroups#!topic/redis-db/d4QcWV0p-YM
132 comments
Yep, i've been a Redis lobbyist for over a year at this company (i use to be the Lead Software Engineer on Youporn/Pornhub), and i'm really happy to see them using it and using it properly.
How do you store the videos?
I remember seeing YouTube storing videos using some sort of specialized hardware from NetApp or something and they still got hit by the limitation of Linux filesystem (number of nodes?)
I remember seeing YouTube storing videos using some sort of specialized hardware from NetApp or something and they still got hit by the limitation of Linux filesystem (number of nodes?)
It was number of files in a directory, and those were the garage days.
You know your software is successful when it's used by porn sites :D
I remember in the 90's having a conversation with my friend. We were discussing Linux vs Windows for web hosting. My friend said "tell me if you can find a single porn site that's hosted on Windows." He was right, I couldn't find any.
Worked for one that was at least in the top 5 porn sites back around 1998-99. All Windows servers running IIS and we were using NetApp appliances for storage.
They do exist. I turned down the chance to interview at one about 6 months ago.
Sure, but IIS in 2011 is a completely different beast than IIS in the 90s.
Spankwire is alexa top400 and runs in .NET
VHS, Linux, and now Redis; the webscale race is over!!
Porn and war drive all innovation. Or so they say.
Sex and war would seem more accurate; porn being just a subset of sex.
And I suppose one could argue that war is often just a manifestation of a desire for sex or power/money as an intermediary to sex.
And I suppose one could argue that war is often just a manifestation of a desire for sex or power/money as an intermediary to sex.
Reading tip for you: Prometheus Rising by Robert Anton Wilson. Especially the part about the first 4 primitive circuits of the mind which govern survival, dominance, sex, stuff like that. War is very centered on Circuit II: dominance, emotions and territorial thinking.
http://www.wikisummaries.org/Prometheus_Rising
It fucked my mind when I read it :) It's a bit far out sometimes though, but it's a cool read.
http://www.wikisummaries.org/Prometheus_Rising
It fucked my mind when I read it :) It's a bit far out sometimes though, but it's a cool read.
It's even less scientific than Freud, and I regard Freud as much as speculation as science. At least Freud had case studies. :/
Which make the concept of war even more obscene.
If you feel that sex is obscene, perhaps.
Edit: Otherwise I don't see what the goal does to make the butchery of your fellow man any more or less obscene than it inherently is.
Edit: Otherwise I don't see what the goal does to make the butchery of your fellow man any more or less obscene than it inherently is.
I assumed he was gesturing at the creation/destruction discord inherent in sex/war with war being the referent for obscene. Of course, a full account of sex more than kisses and pleasantries, it also involves competition.
The (oftentimes sublimated) drive to procreate is one of the most powerful drivers of all human society and action. Perhaps even THE "primus motor" behind it all.
In Prometheus Rising RAW mentions that the reason monks take a vow of chastity is because it instantly frees them from one of the major hamster-wheels of the human condition.
Of course, I'm not planning to become a monk :) But food for thought...
In Prometheus Rising RAW mentions that the reason monks take a vow of chastity is because it instantly frees them from one of the major hamster-wheels of the human condition.
Of course, I'm not planning to become a monk :) But food for thought...
Allow me to explain the joke.
The legal definition of obscenity requires that a work is meant to be arousing and has patently offensive sexual conduct, in addition to having no redeeming artistic value.
Therefore, adding sex into something offensive makes it more obscene.
The legal definition of obscenity requires that a work is meant to be arousing and has patently offensive sexual conduct, in addition to having no redeeming artistic value.
Therefore, adding sex into something offensive makes it more obscene.
Don't forget games!
http://en.wikipedia.org/wiki/Space_Travel_%28video_game%29
I was afraid to see what youporn is (at work now), so i guess it is really a porn site.
The online porn industry were the first people to really deal with scalability (of managing and serving web content, anyway), the first people to deal with online payments, the first people to come under really heavy, sustained hacking attempts, etc. Don't underestimate how much of what you take for granted technologically in 2012, they pioneered in the mid-90s.
A friend of mine built a lot of porn sites in the late 90s using DB2 and Oracle. He says that Microsoft's SQL Server folks came to him to talk about what it would take to get him (and the porn sites) to use SQL Server.
He said that he was confident that DB2 and Oracle could handle the load. They responded that SQL Server was much more capable than he thought and asked him about the workload.
Their response to his answer was "we can't even simulate that, maybe we're not ready".
I assume that they got better.
He said that he was confident that DB2 and Oracle could handle the load. They responded that SQL Server was much more capable than he thought and asked him about the workload.
Their response to his answer was "we can't even simulate that, maybe we're not ready".
I assume that they got better.
It's not just now either, the porn industry was hugely influential in causing the adoption and spread of video (you know, on casette), movies and photography.
Indeed - I've heard it argued that the real reason that VHS won out over the technically superior Betamax format, was that Sony refused to license the Betamax technology to adult video vendors. Never dug into it, but it seems plausible.
Having just one deciding factor seems implausible. Straw that broke the camel's back? Sure. But calling it "the real reason" completely ignores every other relevant factor.
They were also first at SEO - all of these sites competing for the same words - "sex" and "fuck" being their holy grail.
As well as advertising networks and referral schemes.
Though, oddly, they've been late to the game on social.
Though, oddly, they've been late to the game on social.
Scariest button on a porn site is the Like/G+ button.
Social took off when Zuck tricked users into giving out their real names, when meant social networks grew much faster (as old school friends would connect).
Porn does "social", but only anonymous social, which is naturally handicapped.
If someone created a social network which allowed better privacy controls, the porn sites would love it.
Porn does "social", but only anonymous social, which is naturally handicapped.
If someone created a social network which allowed better privacy controls, the porn sites would love it.
We do the same thing, where we populate a mysql store as a secondary storage for business intelligence -- our Founder, PM, AdOps, etc know enough SQL to be dangerous if we build out more traditional, relational models.
It amused me that you were testing a porn site -- the homepage / potentially-misused zrange-issue.
It amused me that you were testing a porn site -- the homepage / potentially-misused zrange-issue.
300k requests/sec? Surely that can't be on a single Redis server, is it? Is Redis cluster out yet, or did they implement their own?
Redis can run anywhere from 50-200K IOPs, on a single core, dependent on CPU; you can find benchmarks at the following:
* http://code.google.com/p/redis/wiki/Benchmarks
* http://redis.io/topics/benchmarks
That's amazing, thank you. This would make me want to use redis for everything, if I weren't already using it for everything.
The OP actually addresses this. They're doing some form of clustering and had to add more nodes due to network saturation.
Just to clarify, they're using regular master-slave, as opposed to "clustering" which is an unreleased feature.
I love how they use sInterStore as a replacement for a fat SQL query.
I love how they use sInterStore as a replacement for a fat SQL query.
100M page views per day is 1157 page views per second. I wonder how that's being blown up by a factor of 300?
I suppose if each video's metadata causes a redis lookup and there are 300 videos displayed per page that would do it, but you'd think you could batch those lookups. It seems a fairly inefficient.
Either that or they have some background jobs causing redis lookups that aren't a result of page views.
I suppose if each video's metadata causes a redis lookup and there are 300 videos displayed per page that would do it, but you'd think you could batch those lookups. It seems a fairly inefficient.
Either that or they have some background jobs causing redis lookups that aren't a result of page views.
There's 30 videos on the front page. Mousing over the image preview of a video displays a series of preview images.
There were 76 requests made as observed from the net tab in Firebug. The majority of these requests seemed to be for those video images. Note that I'm using an adblocker and refuse to check out the site without it.
Note also that they said 300k queries. There could be constantly-running background jobs doing conversion, pulling those preview frames, and crunching numbers on the metadata.
There were 76 requests made as observed from the net tab in Firebug. The majority of these requests seemed to be for those video images. Note that I'm using an adblocker and refuse to check out the site without it.
Note also that they said 300k queries. There could be constantly-running background jobs doing conversion, pulling those preview frames, and crunching numbers on the metadata.
Do they use any sort of wrapper or architecture around it? Is it in Ruby, BTW?
AFAIK, it is Perl project. And I remember seeing it showcased at Catalyst website, so it likely built on top of Catalyst Framework
Thanks.
Tomorrow our site, with over 100 million page views per day, will be relaunching running on Symfony2. This is a complete rewrite. Prior to that it was running the Catalyst Engine written in Perl.Symfony2 rocks
Sounds like Redis is being put to the role of reader boxes in the older split-head MySQL (or *SQL) architecture "write to master, read from slaves" with a conversion shim between MySQL & Redis..
I'm doing something similar with Dirty Hot Productions (one of my clients). We use MySQL on a separate CMS Rails app to store everything, and all of the websites communicate with the CMS via an API. We use Redis to cache all of the API responses, so effectively everything except the first read (or the first "dirty" read) on a particular query is coming straight from Redis. It works amazingly well, and it doesn't use nearly the amount of RAM I thought it would.
Don't travel to Iran.
[deleted]
For some reason I always imagine porn sites to be run by guys in cheap suits and lots of hair gel. Then I realize that in order to run a site with the kind of traffic that some of them have, you really need to have some decent engineers on staff.
You'd be surprised if you went to meetings with porn business companies in Japan. Stylish offices, elegant suits, great manners, no hint of actual porn anywhere. In fact, you could easily confuse the office with an investment bank.
I once had a surreal meeting with a hotel chain where they mentioned they had detailed management reports on stuff like the average time films are watched for and that they had a graph showing how long porn movies are watched for.
This was all being discussed in the context of a fairly formal business meeting.
This was all being discussed in the context of a fairly formal business meeting.
I used to work with a company who provide most of the network/software/hardware for hotels/airports/tran stations/lobbies around the world.
Some very interesting insights from the hotels side of things.
Some very interesting insights from the hotels side of things.
Care to share?
Wish I still had the data on hand - as you would imagine - mostly porn.
[deleted]
Ok, I'm feeling rather naïve and waking up slowly this morning. Why do they care to know that? I mean, what about their business does that help them improve? Because, I think we can take it somewhat for granted that such information probably isn't something which the individuals involved, their customers in this situation, enjoy knowing is being collected about them. "Hey, you know that activity you probably consider rather private, we like to record everything about it we easily can, because umm... customer service."
If I kept track of the porn watching habits of my neighbors, that would be considered strange and maybe rather pervy. And I don't see how it makes a difference in that evaluation whether they are or aren't renting the tv they use from me.
If I kept track of the porn watching habits of my neighbors, that would be considered strange and maybe rather pervy. And I don't see how it makes a difference in that evaluation whether they are or aren't renting the tv they use from me.
It probably lets them know how long the movies they produce should be. If it's too long before any action is in there then people move on. If it's not long enough then the customer doesn't get what they want. Optimizing this means that they can take a single 8 hour shoot, that'll produce maybe an hour and a half of good footage (just pulling numbers out of my ass) and make more than one video of the right length to get the most profit out of it.
Ok, thanks, I'm restored to cynical now... and the business advantage to be found here is that they can go even further than just optimizing length to match what people want, they can make them just a bit shorter than more people would like, so people will buy two instead of just one.
It's probably an extra-cost service to the hotel guest. The hotel wants to know how much/how often the service is used so they can optimize it for their needs. No sense paying for more bandwidth than you need.
In any case, they probably have no need for individual statistics: the aggregated watching habits of guests is enough to provide useful information.
In any case, they probably have no need for individual statistics: the aggregated watching habits of guests is enough to provide useful information.
"For some reason I always imagine porn sites to be run by guys in cheap suits and lots of hair gel."
Pretty much the exact opposite actually. Porn industry guys tend to be 30 - 40 on average and super cool and laid back, though in an LA sort of way. I'm sure there's a lot of crap going on beneath the surface, but at least from the outside it seems like a much nicer work culture than silicon valley.
Pretty much the exact opposite actually. Porn industry guys tend to be 30 - 40 on average and super cool and laid back, though in an LA sort of way. I'm sure there's a lot of crap going on beneath the surface, but at least from the outside it seems like a much nicer work culture than silicon valley.
The owner of YouPorn (and many others) is Fabian Thylmann, and he is actually a geek at heart:
http://nymag.com/news/features/70985/index4.html (article)
http://www.cnbc.com/id/45989405/Meet_the_New_King_of_Porn (photo)
Sure, not only there is huge traffic, they also generate tons of money, so porn sites are definitely not second citizens in the web from a technical point of view.
Some time ago I've read an IAmA on reddit with staff member of some big porn portal. It was quite interesting but I don't remember what porn site it was and don't want to blindly trying to google it as I'm at work. Writing this in case anyone knows and would post it here :)
I did this IAmA couple month back :
http://en.reddit.com/r/IAmA/comments/kf4be/nsfw_iama_former_...
And i am happy that my months of lobbying for Redis at the office paid :D
And i am happy that my months of lobbying for Redis at the office paid :D
http://sl.reddit.com/r/IAmA/comments/htn7x/i_run_porntubecom...
the guy from pinklab http://www.pinklab.com/reddit.html talked around there, but can't find the link. there were also some others, even a guy who ran a 2g1c kind of site.
the guy from pinklab http://www.pinklab.com/reddit.html talked around there, but can't find the link. there were also some others, even a guy who ran a 2g1c kind of site.
An ex-PornHub developer did an AMA a few months ago; they were using Redis as well: http://www.reddit.com/r/IAmA/comments/kf4be/nsfw_iama_former...
[deleted]
[deleted]
First off, I am not saying that youporn guys are not decent but IMHO, people shouldn't get too excited about this.
Youporn is somewhat quite a simple site I'd imagine. It's just categorized videos with a bunch of stats (views, related video...), and users. There's almost no social interaction between users (i've just registered an account just to confirm that).
That's a very clear (and good) reason why 100% redis suits for this website, apart from the goodness of redis like sorted sets are perfect for the "top rated" or "being watched" sections
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others... I'd say 100% redis wouldn't work any more
Youporn is somewhat quite a simple site I'd imagine. It's just categorized videos with a bunch of stats (views, related video...), and users. There's almost no social interaction between users (i've just registered an account just to confirm that).
That's a very clear (and good) reason why 100% redis suits for this website, apart from the goodness of redis like sorted sets are perfect for the "top rated" or "being watched" sections
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others... I'd say 100% redis wouldn't work any more
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others
Follow each other? I'm not sure you know what the point of a porn website is.
Follow each other? I'm not sure you know what the point of a porn website is.
I'm pretty sure i know what I'm looking for in a porn site. There's many things youporn can do to help me watch better stuff.
Say I'm a fan of a porn star XXX and I'd like to see a simple notification that there's recently a new video tagging her. Same thing if I like some particular user's taste for porn and would like to get noticed if he/she has posted a video. Basically if it gets more social, my wall or dashboard or whatever, would be more sorta relevant.
Say I'm a fan of a porn star XXX and I'd like to see a simple notification that there's recently a new video tagging her. Same thing if I like some particular user's taste for porn and would like to get noticed if he/she has posted a video. Basically if it gets more social, my wall or dashboard or whatever, would be more sorta relevant.
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video
As far as I know, Youporn is all about users posting videos (whereby amateur porn is probably only a small part of all uploads, I guess)
As far as I know, Youporn is all about users posting videos (whereby amateur porn is probably only a small part of all uploads, I guess)
You would imagine wrong. I'm running a video sharing page (think FB&youtube mashed together with a dash of blogger) with about 800k unique/month users and it's no picnic.
Sure, but the sharing is what makes it hard - as soon as your pages are per-user it gets interesting. For youporn, each video page is essentially static (maybe there are comments? Even if there are, I doubt they display a different set of comments to each /viewer/). So video pages are easy to cache, the database layout is simple, and sure there's some effort involved in scaling up but it's nowhere near as hard as e.g. tumblr's architecture that was featured recently (where every viewer sees a page with all the posts from everyone they're following, and so everyone's page is different)
/briefly worked at last.fm
/briefly worked at last.fm
I apparently did NOT say that it's picnic. And I don't see how your reply is related to my comment. Does your site have enough social features? Like follow, walls... And is your site 100% redis too?
I was just stating that building a site like that is not as trivial as you made it out to be. Yes you "only" have some videos, but you also have views of said videos, and comments and rankings. Building different views of videos based on those factors is hard if you have lots and lots of this data points. Not to mention that you then have to (somewhere) display just some user's videos sorted by how they ware ranked, or maybe just some user's videos from some category based on number of views (which is not a simple INT field in the db). Etc etc etc.
What I'm saying is that there is more than it meets the eye.
What I'm saying is that there is more than it meets the eye.
Do you store the video using some sort of distributed file systems? specialized machine? SAN/NAS?
Or the good old' NFS?
Or the good old' NFS?
Nothing fancy actually. We have 3 dedicated storage servers (two in the same DC as our web server, one in germany), with lots and lots of disk space. We then SCP converted videos to them and serve them trough nginx (supports seeking).
Hey, thank you so much for the information. I hope you don't mind to share a little bit more (it's okay if you don't reply due to IP or whatnot, I'll understand).
How do you handle where to put the files on which disks? do you write custom software or there's some sort of off-the-shelf software to do this?
The reason I asked about this is because it looked like a few people I talked to seem to prefer to use Distributed File System such as HBase, GlusterFS or something else so that they don't have to write extra code.
And I'm guessing you back them all up on daily base as well?
How do you handle where to put the files on which disks? do you write custom software or there's some sort of off-the-shelf software to do this?
The reason I asked about this is because it looked like a few people I talked to seem to prefer to use Distributed File System such as HBase, GlusterFS or something else so that they don't have to write extra code.
And I'm guessing you back them all up on daily base as well?
We split files into two levels of folders. For instance if a video has a filename "somethingsomethingXY" it will go into "videos/y/x/" folder. Then it's up to our sys admins to mount those folders to whichever disk they want. So from the app's point of view it doesn't matter where the file actually is. And we backup everything every couple of hours.
Like I said, nothing fancy but it works without any issues (apart from a small delay when you have to copy the file to 2 or 3 different servers (redundancy) before it becomes available, but it's not a big problem but I have few ideas how to solve that if it becomes one :)).
Like I said, nothing fancy but it works without any issues (apart from a small delay when you have to copy the file to 2 or 3 different servers (redundancy) before it becomes available, but it's not a big problem but I have few ideas how to solve that if it becomes one :)).
Thank you very much for sharing this.
Wish you all the best to hit and solve that problem (meaning you'll be successful by that time I'm guessing (^_^)).
Wish you all the best to hit and solve that problem (meaning you'll be successful by that time I'm guessing (^_^)).
[deleted]
"After the switchover we had to add some additional Redis nodes but not because Redis was overworked but because the network cards couldn't keep up with Redis."
Wow!
Wow!
That doesn't seem very special or unique to me though. In almost all settings where some in-memory data is pushed on the network stack, you can expect stuff to be network bound.
I'm currently looking into redis for similar needs. I'd be using it from perl and wondered how will it would scale... (IIRC (from people I know who have worked there) youporn/pornhub runs on Catalyst, an MVC-based web framework written in perl.)
Their certainly regarded as probably being one of the busiest Catalyst sites and they were employing Perl/Catalyst developers back in 2009: http://news.ycombinator.com/item?id=565152
NB: The HN article link hits an expired job post. Here is original job ad via WayBackMachine: http://web.archive.org/web/20090418113245/http://jobs.perl.o...
NB: The HN article link hits an expired job post. Here is original job ad via WayBackMachine: http://web.archive.org/web/20090418113245/http://jobs.perl.o...
Youporn used to work on Catalyst, it's been rewritten using Symfony 2.
As for Pornhub it's running a custom PHP framework.
Probably a better question for the Redis group, but I'll throw it here anyway.
Can someone explain why there's no Ordered Map in Redis? Basically to achieve and ordered index, one has to use Sorted Set and a classic Hash Map, and pair these two when ordered data are needed, which is an extra level of indirection and wasted space. Am I missing something?
Can someone explain why there's no Ordered Map in Redis? Basically to achieve and ordered index, one has to use Sorted Set and a classic Hash Map, and pair these two when ordered data are needed, which is an extra level of indirection and wasted space. Am I missing something?
Porn industry is generally among the first to adopt new technologies and promising architectures.
Among other things, this indicates Redis' impending adoption as a standard infrastructure component across the industry in general and as a replacement to standard databases in some of those places.
Among other things, this indicates Redis' impending adoption as a standard infrastructure component across the industry in general and as a replacement to standard databases in some of those places.
Damn society, why can't you let these guys have a company blog where they can blog about this.
The perfect alibi for watching porn at work: investigating redis.
How are you getting the actual content?
If you look at the http headers you'll see that they use varnish to cache the content and nginx as webservers. They probably have a pretty decent CDN to deliver the content to users across the world.
Side-note: I also heard that the founders of Akamai started up after seeing how the porn industry handled all the traffic from geographical different places.
Side-note: I also heard that the founders of Akamai started up after seeing how the porn industry handled all the traffic from geographical different places.
does anyone know anything about their $ #s??
revenue model, revenue/profits/costs #s ??
[deleted]
And you thought people were spending time on Facebook eh....
When will they unleash the Python?
The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily.
(Relational DBs are also good to do a zillion more things of course.)