We Experiment On Human Beings(blog.okcupid.com)
blog.okcupid.com
We Experiment On Human Beings
http://blog.okcupid.com/index.php/we-experiment-on-human-beings/?utm_source=rss&utm_medium=rss&utm_campaign=we-experiment-on-human-beings
104 comments
> my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site.
This is not what people mean when they say someone looks terrible in their pictures.
The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
This is not what people mean when they say someone looks terrible in their pictures.
The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
According to this research done by the okcupid guys here: http://blog.okcupid.com/index.php/page/7/
"The flash adds 7 years"
"The type and brand of camera you use has a huge effect on how good you look in your pictures."
"There are peak times of the day to take a good picture." ala you are more attractive in photos taken at sunrise and sunset.
Between that blog entry and the other ones on the topic the conclusion really is that how attractive you are in real life and how attractive you are in your photo can have very little correlation.
"The flash adds 7 years"
"The type and brand of camera you use has a huge effect on how good you look in your pictures."
"There are peak times of the day to take a good picture." ala you are more attractive in photos taken at sunrise and sunset.
Between that blog entry and the other ones on the topic the conclusion really is that how attractive you are in real life and how attractive you are in your photo can have very little correlation.
Previous OKTrends blog posts and their MyBestFace feature have shown that putting up poor-quality photos has the same effect as being ugly. People just assume you're ugly.
Heh heh. I recently got a message from okcupid saying "Because of a diagnostic test, your match percentage with <user> was misstated as 31%. It is actually 91%. We wanted to let you know!" I had a feeling something like this might be involved. (I messaged that person anyway, because they had a funny username, so I guess I supported their point.)
Hey OkCupid – How about some SSL Love?
"For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network."
https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-...
The Heartbreaking Truth About Online Dating Privacy: https://www.eff.org/press/releases/heartbreaking-truth-about...
"For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network."
https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-...
The Heartbreaking Truth About Online Dating Privacy: https://www.eff.org/press/releases/heartbreaking-truth-about...
Is this really still the case?
Oh, yeah :/ The login page is over SSL, but everything after that is not (!!) This protects your password but not your OkCupid session cookies. It's an improvement over no SSL at all (which was the case for a long time) but still leaves a lot to be desired.
I thought Experiment 2 was a great example of the pain of trying to fit your users' feelings into an "ideal" model.
When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing.
But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept.
Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively.
So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly.
So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it.
PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.
When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing.
But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept.
Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively.
So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly.
So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it.
PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.
This phenomenon is actually quite common. In retail, it's called the customer funnel: http://en.wikipedia.org/wiki/Purchase_funnel. The idea is you want to capture as much awareness with something really simple, usually it's a brand name, e.g. Apple. When customers express more interest, by coming in Apple store/website, they are presented with more design/technical details. Finally, only when they show purchase intent, they are shown price/configuration/delivery options.
OKC experimented with the matching process -- the express purpose of their site. Facebook experimented with mood manipulation -- something they have no permission to do -- regardless of the fine print. Facebook is a very bad landlord, but people don't want to move.
But Facebook already displays posts based on an algorithm; it doesn't do it chronologically, and it's actually almost impossible for me to find a specific post in my timeline. Especially from the mobile site, it's very common that I see something in the moment between opening the app and my timeline updating, and never being able to see the update again.
Facebook is already making the decision as to whether to show you a post or not, and they make that decision based on what they think you will want to see and what you will engage with, all of which is fuzzy and subjective. They already manipulate things you see to encourage engagement; for example, if their algorithm shows that your engagement is dropping and you're about to leave, they'll show other people your profile and say 'do you know this person?', because someone adding you as a friend boosts your engagement.
So if you think that showing you posts which are happy or sad is manipulative, be aware that Facebook is already filtering your potential-friends list and showing you people solely to boost engagement (either yours or theirs). Looked at another way, this means that it's possible their algorithm isn't currently showing you people you might know because it's not as beneficial for them to make that connection for you yet.
So the purpose of Facebook's site is to get people to interact and generate behaviour, and now they're experimenting with that; which posts do we show? which do we hide? They haven't shown them all for ages, so this is just a tweak to their algorithm that they were testing.
What's really interesting here is how this could actually be used for good. Someone feeling crappy? Show them fewer negative posts and more positive posts. Maybe that will help. Show negative posts less often, and make society in general a little more positive.
Facebook is already making the decision as to whether to show you a post or not, and they make that decision based on what they think you will want to see and what you will engage with, all of which is fuzzy and subjective. They already manipulate things you see to encourage engagement; for example, if their algorithm shows that your engagement is dropping and you're about to leave, they'll show other people your profile and say 'do you know this person?', because someone adding you as a friend boosts your engagement.
So if you think that showing you posts which are happy or sad is manipulative, be aware that Facebook is already filtering your potential-friends list and showing you people solely to boost engagement (either yours or theirs). Looked at another way, this means that it's possible their algorithm isn't currently showing you people you might know because it's not as beneficial for them to make that connection for you yet.
So the purpose of Facebook's site is to get people to interact and generate behaviour, and now they're experimenting with that; which posts do we show? which do we hide? They haven't shown them all for ages, so this is just a tweak to their algorithm that they were testing.
What's really interesting here is how this could actually be used for good. Someone feeling crappy? Show them fewer negative posts and more positive posts. Maybe that will help. Show negative posts less often, and make society in general a little more positive.
Facebook didn't experiment with "mood manipulation" any more than OKC. Facebook experimented with changing the site layout and which content and how much to show. Exactly the same stuff as all sites test.
Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation.
But OKC's changes also affected people's moods. More than Facebook's, I would guess.
All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.
Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation.
But OKC's changes also affected people's moods. More than Facebook's, I would guess.
All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.
I'd also note that the proxy they are using ("apparent mood of subsequent content") hasn't been shown to correlate with the subjects'/users' actual mood.
All the FB experiment actually shows is that by manipulating the mood of content seen, you can affect the mood of content produced.
Here's a couple of contrarian hypotheses: "when some users see more 'happy' content, they feel worse about themselves in comparison, but post more 'happy' content to pretend that isn't so." Or, "when some already-sad users see more 'sad' content, this does not affect their mood directly, but does give them tacit permission to share how they are already feeling. Subsequently, their mood actually improves."
All the FB experiment actually shows is that by manipulating the mood of content seen, you can affect the mood of content produced.
Here's a couple of contrarian hypotheses: "when some users see more 'happy' content, they feel worse about themselves in comparison, but post more 'happy' content to pretend that isn't so." Or, "when some already-sad users see more 'sad' content, this does not affect their mood directly, but does give them tacit permission to share how they are already feeling. Subsequently, their mood actually improves."
They experimented through misrepresentation. It's the latter that's the problem, not the former.
If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
'match percentage' is a fuzzy subjective measure. To a large extent it's asking of OkCupid thinks two people having a conversation is a good idea, which is basically impossible to be a lie.
And they're doing it to fight against users being hideously misrepresentative, which is hilarious.
And they're doing it to fight against users being hideously misrepresentative, which is hilarious.
The percentage is insanely fuzzy.
I often endure rage fits from one of my buddies who shows me example after example of cases where, for example, he answered "often" to a question a lady answered "usually," resulting in a mismatch.
The only good thing about it are key individual questions that let you judge someone's intelligence and determine if they're racist. The scalar number is a crock.
I often endure rage fits from one of my buddies who shows me example after example of cases where, for example, he answered "often" to a question a lady answered "usually," resulting in a mismatch.
The only good thing about it are key individual questions that let you judge someone's intelligence and determine if they're racist. The scalar number is a crock.
If your pal is having "rage fits" about an online dating site, he shouldn't be on there in the first place.
I feel very "bearish" towards dating sites. People convince themselves they are saving time meeting more possible "matches" in less time but they forget that meeting online is not the same as in person, "turn-based" chat is different than "real-time" chat.
Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again.
edit: Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.
Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again.
edit: Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.
For the record not every site does these type of experiments. I've been programming websites since the 90s and I've never experimented on my users like this. The most I've done is swap in and out user interface features to see which one has a better response in a typical A/B test. It's not the same as telling a lie to a user no matter how you split hairs about it. They're overstating their case because they happen to be in the business of matching people up.
Is anyone surprised that only 1/5 people who are 90% matches have a "conversation"? I don't use dating sites, but I'd think that people who do want to converse, and who better to converse with than 90% matches?
But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me.
Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.
But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me.
Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.
Here's a screengrab of my okcupid inbox. http://imgur.com/jsKVyCi
One-liners and no picture are instant 'don't bother replying', and most people with more details tend to get weeded out for other reasons.
One-liners and no picture are instant 'don't bother replying', and most people with more details tend to get weeded out for other reasons.
It is kinda fascinating how awful most guys are at pickup lines.
Be brief, be funny, and always as a question. That's how you do it folks.
Be brief, be funny, and always as a question. That's how you do it folks.
Yeah, and spend hours and hours coming up with funny, witty, engaging prose, individually tailored, for people who are 90% unlikely to even respond. After awhile you start to feel like a putz, switch to simple one-liners, and discover to your surprise that it doesn't really affect the response rate by too much.
As a guy, this... this is pretty much it. I tried OKC for around half a year, starting out doing just that. After the first half I just said "Fuck it" and switched to bullshit two sentence intros that were fill in the blank types. My response rate was practically the same.
Thinking about it now, it seems like the numbers game recruiters play. Send out a lot of semi-decent introductions and hope for a response or two. Your best bet is getting out there and "networking", but in lieu of that, sending out a deluge of messages is the next best, albeit shitty, thing.
I'd be interested in hearing how women deal with the deluge of introductions and what makes them decide to respond to someone. Is it the intro? The profile picture? Do they read the person's bio or use that compatibility meter?
Thinking about it now, it seems like the numbers game recruiters play. Send out a lot of semi-decent introductions and hope for a response or two. Your best bet is getting out there and "networking", but in lieu of that, sending out a deluge of messages is the next best, albeit shitty, thing.
I'd be interested in hearing how women deal with the deluge of introductions and what makes them decide to respond to someone. Is it the intro? The profile picture? Do they read the person's bio or use that compatibility meter?
This is simple to come up with a couple tests for: switch a great profile picture for a bad one, switch a profile text you put effort into with an empty or horrific one, and switch thought out messages with really shitty copypasta.
I've done this, and the profile picture is what matters. Text matters a bit--empty hurt, but the half-assed one, if anything, outperformed the one with effort, though they were basically the same.
As computer types, we like to focus on the text, because the text is all and controllable. Good pictures are more difficult, because they take soft skills. Note that, contrary to expectations, it's probably easier to get great pictures of an average looking person than a great profile from an average writer: just using a well-constructed photograph taken with a decent camera as your main photo puts you ahead of most people, even those significantly more attractive than you.
I've done this, and the profile picture is what matters. Text matters a bit--empty hurt, but the half-assed one, if anything, outperformed the one with effort, though they were basically the same.
As computer types, we like to focus on the text, because the text is all and controllable. Good pictures are more difficult, because they take soft skills. Note that, contrary to expectations, it's probably easier to get great pictures of an average looking person than a great profile from an average writer: just using a well-constructed photograph taken with a decent camera as your main photo puts you ahead of most people, even those significantly more attractive than you.
Does photo quality really matter that much? I have never found that the quality of the photo makes a difference in how attractive I find the person. It can be a cheap cameraphone or a DSLR, but it makes no difference, at least to me.
If it does to other people, then I should put more effort into my photographs...
If it does to other people, then I should put more effort into my photographs...
Someone else posted this in another comment: http://blog.okcupid.com/index.php/page/7/
It turns out photo quality, and even which camera you used, matters a lot.
It turns out photo quality, and even which camera you used, matters a lot.
Are you saying that a picture is worth at least a thousand words? That's good to know in future.
I can't speak for all women who use OKC but here's my general method:
1. Generic one-liners get rejected out of hand. Especially generic one-liners with no icon.
2. Message + picture. Did the message amuse me? What's the massively parallel snarkbeast in the back of my brain tell me about you based on your picture? Or preferably, what's she tell me about you based on the multiple pictures you've uploaded? If you only have one you're probably winnowed out here unless it is one hell of a good picture.
3. Skim profile, consider OKC's match estimate, then I start reading the questions you've answered. Some answer conflicts are instant dealbreakers, some are not - someone who's aggressively monogamous has no chance with my poly self, for instance.
4. Consider my current workload: does this candidate feel worth a couple hours at a coffee shop, or more, to experiment with? This is the hardest filter to pass: are you more interesting than "working on my comic" or "getting other art out of the way so I can work on my comic".
Note that a reply is no guarantee of me actually having a desire to do anything; when I checked OKC after this essay reminded me I have an account there, I found a message from a 60% match who turned out to give me a much more conservative vibe than I like, so I did a probe for my own amusement: he said I seemed adventurous and like someone who has 'a whole lot of soul', so I replied with 'Actually I don't have a soul; one dark night in Los Angeles I went to a certain crossroads and traded it to the Devil for serious drawing skills.'. Which is at once a joke, an ironic metaphor about what it was like to work in the animation industry, and a test to see if the conservative vibe I got was correct.
1. Generic one-liners get rejected out of hand. Especially generic one-liners with no icon.
2. Message + picture. Did the message amuse me? What's the massively parallel snarkbeast in the back of my brain tell me about you based on your picture? Or preferably, what's she tell me about you based on the multiple pictures you've uploaded? If you only have one you're probably winnowed out here unless it is one hell of a good picture.
3. Skim profile, consider OKC's match estimate, then I start reading the questions you've answered. Some answer conflicts are instant dealbreakers, some are not - someone who's aggressively monogamous has no chance with my poly self, for instance.
4. Consider my current workload: does this candidate feel worth a couple hours at a coffee shop, or more, to experiment with? This is the hardest filter to pass: are you more interesting than "working on my comic" or "getting other art out of the way so I can work on my comic".
Note that a reply is no guarantee of me actually having a desire to do anything; when I checked OKC after this essay reminded me I have an account there, I found a message from a 60% match who turned out to give me a much more conservative vibe than I like, so I did a probe for my own amusement: he said I seemed adventurous and like someone who has 'a whole lot of soul', so I replied with 'Actually I don't have a soul; one dark night in Los Angeles I went to a certain crossroads and traded it to the Devil for serious drawing skills.'. Which is at once a joke, an ironic metaphor about what it was like to work in the animation industry, and a test to see if the conservative vibe I got was correct.
I suspect it would be the profile picture.
I know that my method for sifting through the hundreds of profiles given by the matching algorithm is to first discriminate based on the profile pic. From there, I'll go on to read the profile text to see if she's someone I'd want to be around. After that, it's the compatibility list which can show any major areas where we would have a difference of opinion (stuff like kids vs no kids, for example).
If those three hurdles are cleared, I'll write a message. I wouldn't be surprised if she's using a similar mechanism to decide whether to respond or not (thus, the low response rate).
I know that my method for sifting through the hundreds of profiles given by the matching algorithm is to first discriminate based on the profile pic. From there, I'll go on to read the profile text to see if she's someone I'd want to be around. After that, it's the compatibility list which can show any major areas where we would have a difference of opinion (stuff like kids vs no kids, for example).
If those three hurdles are cleared, I'll write a message. I wouldn't be surprised if she's using a similar mechanism to decide whether to respond or not (thus, the low response rate).
Wouldn't it be great to get feedback on why you werent answered?
I feel like this post makes it pretty obvious that the profile profile is the most important factor. It even says that "the text is less than 10% of what people think of you".
My understanding, from the article, is that it was the text on the bio. I suppose the text in an introduction could be similarly weighted in the receiver's mind. So does an incredibly shitty introduction with a great picture warrant a response? How about a mediocre intro with a good picture? Where is the tipping point? How does one sift the wheat from the chaff?
>I'd be interested in hearing how women deal with the deluge of introductions and what makes them decide to respond to someone. Is it the intro? The profile picture? Do they read the person's bio or use that compatibility meter?
If you ever figure this out go ahead and write a book. It will be the most successful, widely read book in human history.
I've always gone with 'minor perturbations in the local electromagnetic field'. I've found this hypothesis about as successful at predictions of how women choose mates as anything else I have ever read or heard or thought of, ever.
This is why kstenerud's approach is the correct one.
If you ever figure this out go ahead and write a book. It will be the most successful, widely read book in human history.
I've always gone with 'minor perturbations in the local electromagnetic field'. I've found this hypothesis about as successful at predictions of how women choose mates as anything else I have ever read or heard or thought of, ever.
This is why kstenerud's approach is the correct one.
I guess it depends on how many users there are. Of course if you are in a big area you can just spam everybody (or everybody below 90%) but unfortunately when I was an OKC user a few years ago there weren't enough users to do that around me.
Anyway it takes a few minutes per user to check them out and write them a quick message.
Anyway it takes a few minutes per user to check them out and write them a quick message.
If it's taking you hours to come up with a short personalized first message, you're probably doing it wrong.
Always works for me. But I'm usually too shy to actually send a message, so I end up with an inbox full of boring. Need to work on that.
What about just saying something along the lines of: "I think I'd like you because ______"?
"overbearing and clingy, judgmental from the get go. expects me to fit to some kind of mould. next."
cooldood_69 "hey! you like icecream?"-- --"yeah!"
cooldood_69 "hey! you like icecream?"-- --"yeah!"
Thanks for being honest.
The key word is "think", but I guess most people don't use "think" to actually mean "think".
The key word is "think", but I guess most people don't use "think" to actually mean "think".
Not meaning to be mean at all - just pointing out how subjective the whole thing can be. There's really no silver bullet - if there was, it would probably be evolved out of the system pretty quickly (through overuse)..
This.
Read the profile, find something they mention you don't know about, ask about it, make a joke.
You can even use the same shitty joke over and over--all told, shouldn't take more than a few minutes.
It's like starting up a conversation at a bar or wherever with a stranger: listen for a second, ask nicely about whatever they're talking about or bring up a random thing, and go from there. Practice on people in elevators if you need to overcome your anxiety. :)
Read the profile, find something they mention you don't know about, ask about it, make a joke.
You can even use the same shitty joke over and over--all told, shouldn't take more than a few minutes.
It's like starting up a conversation at a bar or wherever with a stranger: listen for a second, ask nicely about whatever they're talking about or bring up a random thing, and go from there. Practice on people in elevators if you need to overcome your anxiety. :)
Someone should make a dating site where the men can't send the first message.
It comes down to a classic problem: every guy is best off shooting off as many messages as they can, but doing so screws up the signal-to-noise ratio and causes everyone to do worse, in turn rewarding the spammers.
So, the women (naturally) fall back to a heuristic of "Well, does the profile pic at least look attractive?", in turn reinforcing the idea that they are shallow people only caring about looks at the end of the day (somewhat true given the context). It's just not good for anyone.
I'd thought about requiring a proof-of-work to enable sending of messages--failing that, maybe some kind of system whereby you get a token every time you message and get a message back which you can then use to send additional messages.
Think flow-control for dating sites.
So, the women (naturally) fall back to a heuristic of "Well, does the profile pic at least look attractive?", in turn reinforcing the idea that they are shallow people only caring about looks at the end of the day (somewhat true given the context). It's just not good for anyone.
I'd thought about requiring a proof-of-work to enable sending of messages--failing that, maybe some kind of system whereby you get a token every time you message and get a message back which you can then use to send additional messages.
Think flow-control for dating sites.
On a date with a nice young lady and had the dubious pleasure of seeing some of the messages she'd been getting--we'd been trading funny stories about dating.
The bar is set really rather low. :(
The bar is set really rather low. :(
Women don't respond to most messages, even 90%+. 90%+ match is not that special.
90% matches are common because of the huge number of people on the site. And they seem common because those are the kinds of people you're likely to be shown.
Either that or they're usually rare and I'm just so generic I get along with everyone.
Either that or they're usually rare and I'm just so generic I get along with everyone.
It might be different for the gay side of the site. It's rare that I get a percentage above 60.
I'm sure the gay dating pool is pretty much always smaller than the straight, and beyond that the size of the pool depends a lot on the size of the city you're in— there are tons of people in NYC/Chicago/LA/SF.
I'm in a metro of 5.5 million (Atlanta). Very sparse. That's probably why.
I always heard that Atlanta had a huge gay scene. (But, admittedly, I'm from the South and it's probably true by definition compared to almost any other Southern city.)
Metro Atlanta and Atlanta are two different things. The metro is like Atlanta culturally, but it's not dense at all. It's 20-30 minutes to anything, unless you live in one of the bigger cities.
If you want to understand this better I'd recommend going back through that Blog's archives, they talk extensively about this (TL;DR: Women are the minority on the site, and only respond to a small subset of messages, plus matches are personalty not looks and as this article indicates that is only 10% of people's opinions).
It's quite possible to have 100 90% matches - can't contact them all!
90% of the messages I get from 90%ers are dull one-liners.
I don't think the sticky point is whether or not we should be experimenting with people to find what works and what doesn't. It's about respecting users as humans and not simply data points. How does OkCupid know that conversations "went deeper" or how often emails and numbers were exchanged? Were the participants aware of that level of scrutiny or is that tucked away in the ToS? Most people I know aren't even comfortable talking about their OkCupid, they use it as a means to an end and place trust in the service to be discrete. Do these experiments fall in line with user expectations?
I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.
I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.
For the emails and numbers, they can probably search for those in the messages quite effectively without many false-positives.
I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
Sure, my comment wasn't to say they're doing anything nefarious. I'm actually not taking a position here either way. I'm just pointing out that people aren't upset that services use data to improve their services which seems to be the tone and premise of OkCupid's post.
[deleted]
> I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.)
It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.
I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.)
It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.
Basically, people are exactly as shallow as their technology allows them to be.
Just a quote that resonates ... The design and architecture of our online environments will affect our happiness as much as anything else.
Just a quote that resonates ... The design and architecture of our online environments will affect our happiness as much as anything else.
I'm surprised no one has commented on the ethics of "Experiment #3". Feeding false data to users is not something that should be taken lightly.
I think the premise you are assuming is wrong: You assume that it is sending false data to them to say "this person (which our algorithm has deemed is a bad match), is a good match". The problem is that until the experiment is done, you don't how good a match that person actually is. So you aren't sending them "false data," because you don't know that it is "false data".
If it is being reported as known, but is actually unknown, that still would be considered as 'false information'. For instance, if I claim to be clairvoyant, and predict that the next coin you toss will come up 'heads', that would be a false claim, regardless of the outcome of the coin toss.
But you never did so with the intention of eliciting a negative psychological response.
That to me is a distinct difference, both in practicality and morality.
That to me is a distinct difference, both in practicality and morality.
TL;DR; people are shallow idiots.
Cool article though.
#clickbait
Doesn't load completely in IE11.
What a bunch of pointless data and hubris to prove that, yes, people are predominately fueled by the laws of physical attraction.
It might be hard to accept, but people are shallow. Everybody has an attractiveness threshold that must be reached before someone will spark their interest. It might be lower or higher, depending on the person, but everybody has one.
Obviously, sustained interest and the development of romance is more complicated and involves aspects of personality, shared interests, &c. However, that doesn't change the fact that if they weren't good looking enough for you in the first place, you never would have made the effort to find out about the stuff you actually care about.
Obviously, sustained interest and the development of romance is more complicated and involves aspects of personality, shared interests, &c. However, that doesn't change the fact that if they weren't good looking enough for you in the first place, you never would have made the effort to find out about the stuff you actually care about.
Being able to relate attractiveness to a linear scale doesn't mean attraction operates on linear terms. In reality, there is no threshold nor is there a higher or lower. I agree with you in the sense that people will disregard personality or shared interests if they don't find the person physically attractive but there's no answer that's more correct than another.
Nice to see a new blog post from OkCupid.
They haven't updated their blog since April 2011.
They haven't updated their blog since April 2011.
Interesting that the author has a plug at the bottom for his new book, which he says he's been working on for the past 3 years. I guess his book writing got in the way of blogging.
> I guess his book writing got in the way of blogging.
There were a number of factors (which I've explained elsewhere in my comment history). That's part of it, but another factor was that, in 2010, there were 2.5 people working full-time[0] on doing research for OkTrends.
The blog posts took a lot of work. "The Real Stuff White People Like"[1] took almost two months of my time, plus some from Max and Christian as well. (Much like the product design process, since we didn't start each post off with a clear end result in mind, not all the work was visible in the final product).
I left to go back to school. Max ended up taking on more responsibility for other data/stats work, which slowed the pace a bit, and he left at the beginning of 2012 to do his own stuff.
People asked me for the last three years whether the reason OkTrends hadn't posted since 2011 was because of the Match.com acquisition and whether Match shut them down and I had to tell everyone "No, trust me, they're still around! It's just a coincidence!". Thankfully I no longer have to. :)
[0] 2.5 full-time means: Two of us full-time, as well as Christian, though he split his work time between OkTrends (the blog) and other stuff.
[1] http://blog.okcupid.com/index.php/the-real-stuff-white-peopl...
There were a number of factors (which I've explained elsewhere in my comment history). That's part of it, but another factor was that, in 2010, there were 2.5 people working full-time[0] on doing research for OkTrends.
The blog posts took a lot of work. "The Real Stuff White People Like"[1] took almost two months of my time, plus some from Max and Christian as well. (Much like the product design process, since we didn't start each post off with a clear end result in mind, not all the work was visible in the final product).
I left to go back to school. Max ended up taking on more responsibility for other data/stats work, which slowed the pace a bit, and he left at the beginning of 2012 to do his own stuff.
People asked me for the last three years whether the reason OkTrends hadn't posted since 2011 was because of the Match.com acquisition and whether Match shut them down and I had to tell everyone "No, trust me, they're still around! It's just a coincidence!". Thankfully I no longer have to. :)
[0] 2.5 full-time means: Two of us full-time, as well as Christian, though he split his work time between OkTrends (the blog) and other stuff.
[1] http://blog.okcupid.com/index.php/the-real-stuff-white-peopl...
I had only heard about the studies but never took time to read the blog posts. I think I've spent the better part of the past two hours on there. Great stuff and always interesting to see people doing experiments like this (and making them open and 'fun'). A blog post every few months with as much in-depth research is something to look forward to.
It seemed to me the company being acquired by Match.com got in the way of the blogging.
Yeah, I just started reading all the older blog posts and they are really interesting!
I'm still mad they removed the blog post on the success rates of paid dating services a few days after being acquired by Match.com (I think it was)
In case you wanted to read it again, here's a Wayback Machine link [1].
[1]: http://web.archive.org/web/20101006104124/http://blog.okcupi...
[1]: http://web.archive.org/web/20101006104124/http://blog.okcupi...
It is still true, and even before they wrote the blog post, that used to be my answer to friends' question, "which is better?". I would say, think about it: Match makes money the longer you stay in the system! Then it would dawn on them that they were the product.
I hope it sticks around! They're so fascinating.
> They haven't updated their blog since April 2011.
I think that's around the time they were bought out by Match.com .
I think that's around the time they were bought out by Match.com .
I suspect that the cavalier sentiment OKCupid has approached the discourse with will not go well for them.
Will be interesting to observe the sociological takedowns of this post.
Will be interesting to observe the sociological takedowns of this post.
Unlikely. Facebook went overboard with it after hiding it from their users. OKC has always done this with their data, although I am more than a little disturbed by them showing deliberately bad data to their users.
Also who the hell though it was a good idea to have a dating site without pictures (even for 24 hours?)
Also who the hell though it was a good idea to have a dating site without pictures (even for 24 hours?)
Unfortunately, while showing what we would consider "intuitively bad data" may seem like a bad idea, only actually empirically testing it can we measure and quantify the exact impact that bad data (either shown purposefully or accidentally, or unknowingly) will have on users.
It may have in fact turned out that what we intuitively think as bad data results in better matches or better experiences. I think experimenting is worthwhile, so long as it is done in the open as they have been doing.
It may have in fact turned out that what we intuitively think as bad data results in better matches or better experiences. I think experimenting is worthwhile, so long as it is done in the open as they have been doing.
I remember reading about this concerning Netflix. They used to have a competition on who can come up with a better recommendation algorithm. They eventually decided not to implement the best one because what they were getting was good enough, and since most people were streaming instead of getting a dvd, the more recommendations the better. They could just try and if they don't want to see the whole movie, rate it and instantly pick another.
I'm pretty sure they didn't implement it because it was an extremely impractical solution.
Edit:
"We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment." - https://www.techdirt.com/blog/innovation/articles/20120409/0...
Edit:
"We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment." - https://www.techdirt.com/blog/innovation/articles/20120409/0...
>Streaming has not only changed the way our members interact with the service, but also the type of data available to use in our algorithms. For DVDs our goal is to help people fill their queue with titles to receive in the mail over the coming days and weeks; selection is distant in time from viewing, people select carefully because exchanging a DVD for another takes more than a day, and we get no feedback during viewing. For streaming members are looking for something great to watch right now; they can sample a few videos before settling on one, they can consume several in one session, and we can observe viewing statistics such as whether a video was watched fully or only partially.
I wouldn't say that this makes you wrong, but I'd say you are partially right.
I wouldn't say that this makes you wrong, but I'd say you are partially right.
> I am more than a little disturbed by them showing deliberately bad data to their users.
I actually find this reassuring. It means they're sanity checking their algorithm to make sure that it works better than random pairings, or isn't outright wrong.
I actually find this reassuring. It means they're sanity checking their algorithm to make sure that it works better than random pairings, or isn't outright wrong.
I'm willing to bet that Facebook has always done this with their data as well. You have to, in order to determine not just if your algorithms work, but whether they should work and what they should do.
Also, Facebook didn't 'hide it from their users' so much as 'not tell anyone they were doing it'. And for the average OkCupid user, I'm willing to bet that 99% of them have no idea this sort of thing happens. They log onto the site, do some stuff, and then leave.
Personally, I was as surprised by the idea that OkCupid outright lied about compatibility scores to see what was up as I was about the results, but it doesn't bother me, and I'm the type of person who's read all their blog posts anyway.
Also, Facebook didn't 'hide it from their users' so much as 'not tell anyone they were doing it'. And for the average OkCupid user, I'm willing to bet that 99% of them have no idea this sort of thing happens. They log onto the site, do some stuff, and then leave.
Personally, I was as surprised by the idea that OkCupid outright lied about compatibility scores to see what was up as I was about the results, but it doesn't bother me, and I'm the type of person who's read all their blog posts anyway.
And, probably more to the point, Facebook is Facebook and OKCupid is OKCupid. And even if, as OKCupid says in their post "But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work." the fact that Facebook did it ignited the Internet's collective anti-Facebook sentiment. (Of course, it's also the case that Facebook's experiments were more oriented toward their own advertising than improving the site experience for users.)
I really enjoyed reading this.
On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error.
For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore.
[1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them.
Edit: tweaked phrasing as per comments.
On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error.
For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore.
[1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them.
Edit: tweaked phrasing as per comments.
I believe you are supposed to get notified when rated 4 or 5 regardless of whether you rated the other person at all.
Didn't they recently remove the 4-5 star system in favor of just a like button? I've been using the mobile version lately but it just has a star now for like and no longer shows a 1-5 rating.
>But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
Hm, that is unexpected. Wonder how long it will last.
Hm, that is unexpected. Wonder how long it will last.
Nah, we like people who like us. So if a trusted source tells us someone else either likes or will like us, we tend to like them.
From Modern Seinfeld [0]:
George swipes right for every woman on Tinder. E:"What if you're not attracted to her?" G:"If she's attracted to me, I might be!"
[0] https://twitter.com/SeinfeldToday
George swipes right for every woman on Tinder. E:"What if you're not attracted to her?" G:"If she's attracted to me, I might be!"
[0] https://twitter.com/SeinfeldToday
I think it's one more example of when people's emotions and desires lead them to suboptimal outcomes. Most of the cues we associate with beauty and sexual desire evolved back in the savanna when health and fertility were very real risks; being able to pick up on which potential mates would be able to carry healthy offspring to term and nurture them until adulthood was very important then. Nowadays, the far greater risk is that you'll hate each other and fight all the time, but this has only been a concern since people started living long enough and in close enough proximity to care.