Why Throwing 92 Heads in a Row is not Surprising [pdf](marcsandersfoundation.org)
marcsandersfoundation.org
Why Throwing 92 Heads in a Row is not Surprising [pdf]
http://www.marcsandersfoundation.org/wp-content/uploads/Stoppard_final.pdf
8 comments
It's very surprising for 92 coin flips to have a tiny Kolmogorov complexity :)
What is surprising about 92 heads in a row is the low entropy of the outcome, not the sheer unlikeliness of it. Yes every other outcome is just as unlikely. But almost very other outcome has MUCH higher entropy.
If we are flipping a fair coin, the "surprise" we have at the result of any individual flip should be the same for all individual flips. Therefore, the entropy of the event, call it "A", of 92 fair coin tosses each resulting in heads is the same as the entropy of any other 92 fair coin tosses, "B". The sum of the individual event entropys must be the same so entropy(A) = entropy(B).
If you are talking about information theory entropy and saying a message of a billion bits composed of all "ones" has lower entropy than a "random" billion bit message, then sure. This is like saying we can compress the billion bits of ones and send less bits but the same amount of information. But I don't think this is synonymous with the above. It would be like saying for event A, each proceeding individual event has less entropy than the previous -- we aren't dealing with a fair coin anymore.
If you are talking about information theory entropy and saying a message of a billion bits composed of all "ones" has lower entropy than a "random" billion bit message, then sure. This is like saying we can compress the billion bits of ones and send less bits but the same amount of information. But I don't think this is synonymous with the above. It would be like saying for event A, each proceeding individual event has less entropy than the previous -- we aren't dealing with a fair coin anymore.
It is true, if you do not have memory (memoryless chanel with a memoryless observer;)
But if you do have memory, then you are free to chose to see "92 consecutive throws with a fair coin" as one event.
In that case the Kolmogorov complexity describes perfectly well why you should be suprised at low entropy outcomes - simply because those are rare events (using our new definition of 'event').
But if you do have memory, then you are free to chose to see "92 consecutive throws with a fair coin" as one event.
In that case the Kolmogorov complexity describes perfectly well why you should be suprised at low entropy outcomes - simply because those are rare events (using our new definition of 'event').
Isn't it part of the issue that he conflates sequences with totals?
As he notes there are trillions of ways to get between 40 and 50 heads. Our surprise isn't in the sequence but the distribution of outcomes?
I would absolutely be "surprised" if someone accurately predicted a coin flip 92 times regardless of the distribution of heads and tails.
As he notes there are trillions of ways to get between 40 and 50 heads. Our surprise isn't in the sequence but the distribution of outcomes?
I would absolutely be "surprised" if someone accurately predicted a coin flip 92 times regardless of the distribution of heads and tails.
i disagree with the author's logic here - yes any individual result is disconnected, yes the gambler's fallacy is wrong, but if we simply expand it to say, a 100 sided die landing on 1, 92 times in a row, then we can see the flaw here.
or to put it another way - expand his argument to 1 million coin flips, and all of them heads.
at some point probabilities should be evinced in results. it is surprising when they are not.
or to put it another way - expand his argument to 1 million coin flips, and all of them heads.
at some point probabilities should be evinced in results. it is surprising when they are not.
He addresses this in the article if you read a little further down. Briefly, any sequence of 1 million coin flips is equally unlikely, so why are most of them not surprising?
He addresses this in the article if you read a little further down. Briefly, any sequence of 1 million coin flips is equally unlikely, but because most fall within our expectations, they are not surprising. 1 million heads in a row does not fall within our expectations and is unlikely, and hence is surprising.
Surprising is a poor word choice.
A: I just threw 92 heads in a row.
B: I’m surprised!
A: No you’re not.
B: The hell you say?
A: I just threw 92 heads in a row.
B: I’m surprised!
A: No you’re not.
B: The hell you say?
"Surprising" is fine. The title isn't "Why you're not surprised at throwing 92 heads in a row". There's no agent in the title, so he's referring to an objective idea of an event being surprising. I'm not sure I agree with his concept of "surprising", but it's at least objective.
Does the author actually define surprising anywhere other than "George Shackle's definition"?
Yes, in the last section:
> Roughly speaking, it’s rational to be surprised by an event if and only if that event requires investigation and explanation.
> Roughly speaking, it’s rational to be surprised by an event if and only if that event requires investigation and explanation.
If that happened to you, wouldn't you investigate it and try to explain it?
In practice, if you saw this happen you'd have probably notice that something was off. The coin is probably weighted, or some sleight of hand tricked you.
In practice, if you saw this happen you'd have probably notice that something was off. The coin is probably weighted, or some sleight of hand tricked you.
Well yeah because it's a weighted coin.
Or, from Terry Pratchett:
People have reality-dampers. It is a popular fact that nine-tenths of the brain is not used and, like most popular facts, it is wrong. Not even the most stupid Creator would go to the trouble of making the human head carry around several pounds of unnecessary gray goo if its only real purpose was, for example, to serve as a delicacy for certain remote tribesmen in unexplored valleys. It is used. And one of its functions is to make the miraculous seem ordinary and turn the unusual into the usual. Because if this was not the case, then human beings, faced with the daily wondrousness of everything, would go around wearing big stupid grins, similar to those worn by certain remote tribesmen who occasionally get raided by the authorities and have the contents of their plastic greenhouses very seriously inspected. They'd say "Wow!" a lot. And no one would do much work.