The difference betwen skill and luck.(blogs.hbr.org)
blogs.hbr.org
The difference betwen skill and luck.
http://blogs.hbr.org/cs/2011/02/untangling_skill_and_luck.html
13 comments
Exploitations of people's failure to understand statistics are a lot older than email.
Even in my grandmother's time there would be somebody who would offer to predict the gender of your unborn child - "if I am wrong, I'll give you your money back!".
Even in my grandmother's time there would be somebody who would offer to predict the gender of your unborn child - "if I am wrong, I'll give you your money back!".
Reminds me of http://xkcd.com/628/
Warren buffet says what if all that predict successfully come from the same village, wouldnt that tells you something?
The same rules of probability apply, so it depends on the number of villages making predictions divided by the improbability of successfully matching that village's predictions. If there were only 10 or 20 people buying lottery tickets, and one of them actually won, I'd pay very close attention to that one person's methods.
That scam is an old one going back at the least to newspapers in the early 1900s. Specifically, sports prediction services did it in classified ads (and continue to do so in some places). I think most con-artist scams haven't really changed, just adopted new mediums and new technologies.
> Always compare results with a null model that reflects luck.
It's possible to win lots of money based on making exceedingly poor bets and getting very lucky. There's one card left to come, there's a 5% chance an Ace will come, but if it does you'll have an unbeatable hand, if not, you have a hand that beats 66% of the possible hands. Do you bet? What happens if the Ace doesn't fall?
A poor player who gets lucky will attribute his success to his obvious skill.
A skillful player who gets lucky will chide himself for making a poor decision.
If nothing else, playing poker has taught me this: The correct answer is always "it depends".
You are now 2/3rds of the way to being a successful lawyer, accountant or economist!
The best single comment I've read about luck was written right here on HN. I've detailed it here: http://techneur.com/post/1321568428/luck-ii
In summary, I made a comment about Zuckerberg and the value of Facebook. Someone responded that he was lucky. To which someone else responded with this: (Can't find original HN comment)
"This attitude bugs me a lot. There is probably a path for every single person in this country to make a million dollars in the next year if they were just “savvy enough” to take advantage of it. It seems everyone wants to diminish Zuck’s success by ascribing luck to it, but let me tell you something. Zuck executed the shit out of Facebook. Did the community and mindshare just come out of thin air?
There’s no good reason to believe that Zuckerberg is lucky at all. Saying he was in the right place at the right time has this tacit assumption that if he were somewhere else at the wrong time he would have tried the same thing and fell flat on his face.
It seems because Facebook is an outlier, people feel safe talking about the luck factor, but that’s meaningless because we all exist with individual circumstances, and by that measure everything every one of us does is based on luck. Instead, I prefer to ascribe luck to things that the individual actually had no control over, such as winning the lottery."
In summary, I made a comment about Zuckerberg and the value of Facebook. Someone responded that he was lucky. To which someone else responded with this: (Can't find original HN comment)
"This attitude bugs me a lot. There is probably a path for every single person in this country to make a million dollars in the next year if they were just “savvy enough” to take advantage of it. It seems everyone wants to diminish Zuck’s success by ascribing luck to it, but let me tell you something. Zuck executed the shit out of Facebook. Did the community and mindshare just come out of thin air?
There’s no good reason to believe that Zuckerberg is lucky at all. Saying he was in the right place at the right time has this tacit assumption that if he were somewhere else at the wrong time he would have tried the same thing and fell flat on his face.
It seems because Facebook is an outlier, people feel safe talking about the luck factor, but that’s meaningless because we all exist with individual circumstances, and by that measure everything every one of us does is based on luck. Instead, I prefer to ascribe luck to things that the individual actually had no control over, such as winning the lottery."
Zuck executed the shit out of Facebook.
In comparison to what? For all we know, there were dozens, maybe hundreds of other similar sites being built at the same time. Without being able to compare them, we really can't say how well he executed Facebook. Having used both Friendster and MySpace, I'd say he did better than either of them. But by the time Facebook was big enough to be mentioned in the same breath as those two, success to some large degree was already assured.
Instead, I prefer to ascribe luck to things that the individual actually had no control over, such as winning the lottery.
This is precisely the point: there may very well have been key moments in Facebook's history, over which Zuckerberg had not control whatsoever, yet which ultimately determined the company's fate. We know the economy is a nonlinear dynamical system, and in such systems small perturbations in the input can lead to large changes in the output. There's no reason to suppose that similar effects don't play a role in the life cycle of individuals or businesses.
In comparison to what? For all we know, there were dozens, maybe hundreds of other similar sites being built at the same time. Without being able to compare them, we really can't say how well he executed Facebook. Having used both Friendster and MySpace, I'd say he did better than either of them. But by the time Facebook was big enough to be mentioned in the same breath as those two, success to some large degree was already assured.
Instead, I prefer to ascribe luck to things that the individual actually had no control over, such as winning the lottery.
This is precisely the point: there may very well have been key moments in Facebook's history, over which Zuckerberg had not control whatsoever, yet which ultimately determined the company's fate. We know the economy is a nonlinear dynamical system, and in such systems small perturbations in the input can lead to large changes in the output. There's no reason to suppose that similar effects don't play a role in the life cycle of individuals or businesses.
"In comparison to what? For all we know, there were dozens, maybe hundreds of other similar sites being built at the same time. Without being able to compare
them, we really can't say how well he executed Facebook."
Yes we can. If there were tons of social networking sites out there at the time, and Facebook came out on top, there must be a reason.
Yes we can. If there were tons of social networking sites out there at the time, and Facebook came out on top, there must be a reason.
Luck? :)
Zuckerberg is not particularly unique in terms of his intelligence, ambition or connections. Men and women like Zuckerberg have come into and passed out (or dropped out) of the USA's elite universities for centuries.
Yes. Luck played a big role.
Yes. Luck played a big role.
"Zuckerberg is not particularly unique in terms of his intelligence, ambition or connections. Men and women like Zuckerberg have come into and passed out (or dropped out) of the USA's elite universities for centuries.
Yes. Luck played a big role."
The reason I say luck didn't play a big role, is because not anyone could do what he did.
Can anybody from a 5 year old to a grandmother build Facebook? If not, there must be skill required to accomplish the task.
To build the application behind Facebook alone takes skill (taking out a large percentage of the population). To build the company up to what it is (and make the right decisions) takes another set of skills. I think the luck factor is around 10%.
Of course there is a certain amount of luck with anything that we do. I have always felt that there are lucky situations all around us. Without the proper skills to recognize it, most people will just let it pass by.
The reason I say luck didn't play a big role, is because not anyone could do what he did.
Can anybody from a 5 year old to a grandmother build Facebook? If not, there must be skill required to accomplish the task.
To build the application behind Facebook alone takes skill (taking out a large percentage of the population). To build the company up to what it is (and make the right decisions) takes another set of skills. I think the luck factor is around 10%.
Of course there is a certain amount of luck with anything that we do. I have always felt that there are lucky situations all around us. Without the proper skills to recognize it, most people will just let it pass by.
I am reminded of an anecdote of a famous physicist talking to Army personnel during WW2. Unfortunately I cannot find a reference with various keywords on Google, so take it as you will.
Somehow conversation turns to the great generals -- the 4 and 5 star generals.
"How many great generals are there?" asks the physicist.
"No more than about 1 in 20 is a truly great general", they answer.
"How do you decide who is a 4- or 5- star general?" the physicist asks.
The army men decide that you need to have won 5 battles to be a great general.
The physicist then explains how it can easily be luck. Suppose that battle success is randomly determined. A general has a 0.5^5 chance of winning 5/5 battles out of sheer luck -- 3.125% in fact. Therefore the majority of "Great Generals" are just lucky.
Somehow conversation turns to the great generals -- the 4 and 5 star generals.
"How many great generals are there?" asks the physicist.
"No more than about 1 in 20 is a truly great general", they answer.
"How do you decide who is a 4- or 5- star general?" the physicist asks.
The army men decide that you need to have won 5 battles to be a great general.
The physicist then explains how it can easily be luck. Suppose that battle success is randomly determined. A general has a 0.5^5 chance of winning 5/5 battles out of sheer luck -- 3.125% in fact. Therefore the majority of "Great Generals" are just lucky.
That makes about as much sense as saying: pick two sports teams to play each other, the odds of one of the team winning is 50%. The point is, that probability doesn't hold, and thus nullifies the anecdote.
I didn't explain it very well.
The point of the anecdote is that the null hypothesis -- that 5-star generals are there basically because somebody "wins" each battle and that a fixed percentage of these will win 5 in a row -- is difficult to distinguish from genuine tactical skill.
That's the point of the article too. We too often attribute to skill what can just as reasonably be luck. Partly because of survivor bias: Zuckerberg et al are the one group out of the hundreds of contenders who succeeded in getting runaway network effects. But if contenders had non-zero chances of success, somebody was bound to succeed. The null hypothesis is that Zuckerberg was just lucky and it is basically not easy to disprove, given that as I said above there's nothing particularly unusual about him or Facebook going into the fight.
The point of the anecdote is that the null hypothesis -- that 5-star generals are there basically because somebody "wins" each battle and that a fixed percentage of these will win 5 in a row -- is difficult to distinguish from genuine tactical skill.
That's the point of the article too. We too often attribute to skill what can just as reasonably be luck. Partly because of survivor bias: Zuckerberg et al are the one group out of the hundreds of contenders who succeeded in getting runaway network effects. But if contenders had non-zero chances of success, somebody was bound to succeed. The null hypothesis is that Zuckerberg was just lucky and it is basically not easy to disprove, given that as I said above there's nothing particularly unusual about him or Facebook going into the fight.
You are falling prey to a fallacy. Suppose there is a competition (say a marathon) where one thousand people participate. The winner gets to choose a number between 1 and one million and see if it matches a randomly generated one.
If you win, it is true that not everyone could do what you did (you were the best runner out of a thousand). It also true that luck played a HUGE role (one in a million odds after putting yourself in the right place based on skill).
If you win, it is true that not everyone could do what you did (you were the best runner out of a thousand). It also true that luck played a HUGE role (one in a million odds after putting yourself in the right place based on skill).
The comment is here: http://news.ycombinator.com/item?id=1791800
Searchyc.com is your friend: http://searchyc.com/This+attitude+bugs+me+a+lot.+There+
Searchyc.com is your friend: http://searchyc.com/This+attitude+bugs+me+a+lot.+There+
You can also just google it with quotation marks.
I don't want to be a dick, but you do have some amount of control over the probability of you winning the lottery. If you don't play, it becomes significantly more difficult to win.
The difficulty in separating skill and luck is unconsciously illustrated by the author himself in his basketball example. We're asked to consider how much of a 'hot hand' is attributable to chance, and how much to skill. But what is chance in this context? Clearly he doesn't mean the likelihood of a ball launched in an arbitrary trajectory finding its way to the basket. More likely, he means the shooting average of an average basketball player in the type of game being observed (pickup game, college, etc). Clearly skill contributes to such a baseline average.
With basketball we have a skill that can easily be measured for nearly anyone: we just have them toss a basketball a few dozen times and we have a good gauge of how skillful they are compared to others. But what about, say, a venture capitalist? The game that they are supposed to be skilled at has so much overhead that very few are ever played, and not all those that want to play can (you've got to have a lot of money to even sit at the table). It's quite possible that success as a VC is entirely attributable to luck, with no contribution from skill at all. For these kinds of activities we have no way to generate a meaningful statistical baseline. We can only engage in a thought experiment: if hypothetically the success in question was actually due solely to chance, would the world we see look any different?
With basketball we have a skill that can easily be measured for nearly anyone: we just have them toss a basketball a few dozen times and we have a good gauge of how skillful they are compared to others. But what about, say, a venture capitalist? The game that they are supposed to be skilled at has so much overhead that very few are ever played, and not all those that want to play can (you've got to have a lot of money to even sit at the table). It's quite possible that success as a VC is entirely attributable to luck, with no contribution from skill at all. For these kinds of activities we have no way to generate a meaningful statistical baseline. We can only engage in a thought experiment: if hypothetically the success in question was actually due solely to chance, would the world we see look any different?
"In the mid-1970s, a man sought a ticket that ended in 48. He found a ticket, bought it, and won the lottery. When asked why he was so intent on finding that number, he replied, 'I dreamed of the number seven for seven nights. And seven times seven is 48.'"
Something tells me this guy did well by not taking math in grade school too seriously.
Something tells me this guy did well by not taking math in grade school too seriously.
If he'd paid attention to maths, he wouldn't be playing the lottery.
Well, he was close enough.
> He found a ticket, bought it, and won the lottery. When asked why he was so intent on finding that number, he replied, "I dreamed of the number seven for seven nights. And seven times seven is 48."
Need say no more.
Need say no more.
> Reversion to the mean. Any system that combines skill and luck will revert to the mean over time. This means that an extreme outcome, good or bad, will be followed by an outcome that has an expected value closer to the mean.
So if I toss a coin 10 times and it comes up heads, the next toss is more likely to come up tails? I smell a rat.
So if I toss a coin 10 times and it comes up heads, the next toss is more likely to come up tails? I smell a rat.
No, it means that in the next 10 tosses, you still expect to get a 5/5 split, so over these 20 tosses, you'll probably have 15 heads, which is closer to the mean.
I've seen this analysis in sports. In baseball, for example, you generally expect teams to win one-run games as often as they win games in general (that is, there's no "great closer" effect). So if you see a team winning 80% of their one-run games in the first half of the season, you can expect their record to be worse in the second half.
I've seen this analysis in sports. In baseball, for example, you generally expect teams to win one-run games as often as they win games in general (that is, there's no "great closer" effect). So if you see a team winning 80% of their one-run games in the first half of the season, you can expect their record to be worse in the second half.
This is still wrong, since coin flips are independent events, and events that are totally independent can't affect each other's probability. I understand your reasoning, but it is flawed.
Ah, I see, thanks.
No dude- you were right and he was wrong.
No, he's saying that if you have a few outliers to begin with you expect to get more normal-looking data later on, drowning your outliers in them. If you get 5 heads initially and then another 5 heads and 5 tails, it's still closer to the mean than when you began.
That can't be right, since the each point in the data is independent. In a 50/50 coin-flip game where you get 9 coins come up heads, the probability that the next coin flip results in tails is still 50%. What you need to understand is that probability deals with uncertain events. So instead of thinking about the final count needing to be around 50% heads and 50% tails, you should expect the count of the yet undecided part to converge around 50% heads, 50% tails.
I just noticed this thread continued, but I'm not sure what you're worried about. The two things you said:
* the final count will be about 50/50
* the remaining part will be about 50/50
are actually the same thing (since we're talking about a limit). That is, we could flip a fair coin a billion times, get all heads, and still, in the long run, expect the total ratio to converge to 1/2 (imagine flipping the coin forever - that initial billion flips won't even be a blip on the graph).
* the final count will be about 50/50
* the remaining part will be about 50/50
are actually the same thing (since we're talking about a limit). That is, we could flip a fair coin a billion times, get all heads, and still, in the long run, expect the total ratio to converge to 1/2 (imagine flipping the coin forever - that initial billion flips won't even be a blip on the graph).
"I dreamed of the number seven for seven nights. And seven times seven is 48."
I think his lack of multiplication skills won him the lottery. It was luck indeed.
I think his lack of multiplication skills won him the lottery. It was luck indeed.
It was interesting but I was hoping for some more concrete suggestions than just a 'you should consider randomness vs luck.' What is a better way than ESOs? Give me a concrete example of how to untangle this rather than just saying untangle it. It doesn't have to be universally applicable but an example would be nice.
"I'm a great believer in luck, and I find the harder I work the more I have of it." - Thomas Jefferson
The author has an entire paper dedicated to this [PDF] http://www.cscs.umich.edu/documents/Luck.pdf
Luck = Skill + Opportunity
[deleted]
You find 256 people and email them the details of a 50/50 prediction. 128 people should get the right prediction. You then email them another 50/50 prediction. Of course, 64 of them receive the right prediction twice in a row. You then make a third 50/50 prediction to the 64, another to the 32 positive responses and so on until you find a manageable number of people who now believe you have some amazing insight into the future.
As much as we may want to think things are not random, sometimes they really are, sometimes they're just probabilities.