Randall Munroe Responds to Gelman's Criticism of XKCD(andrewgelman.com)
andrewgelman.com
Randall Munroe Responds to Gelman's Criticism of XKCD
http://andrewgelman.com/2012/11/16808/#comment-109366
32 comments
He's not apologizing for a comic -- he's apologizing for mischaracterzing people through his comic. He's not saying the point is wrong -- just that the labels are :)
I think it's a thoughtful, solidly nice and totally stand-up (and mature) thing for him to have done.
I think it's a thoughtful, solidly nice and totally stand-up (and mature) thing for him to have done.
Yeah, the point is apparently just that unthinkingly accepting p = 0.05 as your cutoff for significance is moronic (this point is utterly uncontroversial among thinking humans), whereas the labels make it seem as if he's trying to say something about the controversy between frequentists and bayesians, which he didn't even know existed.
> (this point is utterly uncontroversial among thinking humans)
Are you sure about this? Most people don't know any statistics, and the vast majority of people who know some statistics never took it further than Stats 101. So a lot of people do blindly just apply p tests like this.
Randall's point about cancer statistics just serves to give a real world example of this, and why it's so important people better understand it.
Are you sure about this? Most people don't know any statistics, and the vast majority of people who know some statistics never took it further than Stats 101. So a lot of people do blindly just apply p tests like this.
Randall's point about cancer statistics just serves to give a real world example of this, and why it's so important people better understand it.
Well, some people might, indeed, be educated stupid, though I hope to god no actual Stats 101 class teaches people that p < 0.05 = significant, other than as a convention that people might want to be aware of when they encounter the phrase "statistically significant".
I would be legitimately shocked if even a single doctor were willing to defend that p-test as in principle what it is for something to be statistically significant no matter what thing is at issue, rather than as a decent enough heuristic.
Nevertheless, the parenthetical you highlight is, indeed, probably literally false. I was employing a rhetorical device to convey the degree to which I found the point that, evidently, is the one Randall actually intended to be trivial.
I would be legitimately shocked if even a single doctor were willing to defend that p-test as in principle what it is for something to be statistically significant no matter what thing is at issue, rather than as a decent enough heuristic.
Nevertheless, the parenthetical you highlight is, indeed, probably literally false. I was employing a rhetorical device to convey the degree to which I found the point that, evidently, is the one Randall actually intended to be trivial.
I would be legitimately shocked if even a single doctor were willing to defend that p-test as in principle what it is for something to be statistically significant no matter what thing is at issue, rather than as a decent enough heuristic
Unless things have changed in the last twenty years, when I did a series of interviews with doctors about Bayesian probabilities in relation to an expert system project, you're going to be unpleasantly surprised at both the number who do - and the number who aren't really very sure what a p-test is at all...
Unless things have changed in the last twenty years, when I did a series of interviews with doctors about Bayesian probabilities in relation to an expert system project, you're going to be unpleasantly surprised at both the number who do - and the number who aren't really very sure what a p-test is at all...
I share your sensibilities. But I work with consumers of statistical analysis on a daily basis, and it is an unfortunately common to put stars next to parameters that are statistically significant at the 5% level. And many people treat that star or lack thereof as a very serious indication of whether the parameter is "meaningful."
I have even seen a slide presentation by consultants with a business practice of dropping all variables with p>.05, and re-estimating the model (doing this repeatedly until all variables have p>0.05).
The reliance on p-value thresholds is heartbreaking.
I have even seen a slide presentation by consultants with a business practice of dropping all variables with p>.05, and re-estimating the model (doing this repeatedly until all variables have p>0.05).
The reliance on p-value thresholds is heartbreaking.
[deleted]
He's thanking someone for taking the time to correct a possible mistake he made; In turn he's pledging to attempt to avoid further similar mistakes.
All in all, I think this was a wonderful exchange.
All in all, I think this was a wonderful exchange.
A sincere thank you for the gentle corrections; I’ve taken them to heart, and you can be confident I will avoid such mischaracterizations in the future!
At least, 95.45% confident.
Its partly a setup for his parting joke.
At least, 95.45% confident.
Its partly a setup for his parting joke.
Oh! I didn't realize this was Gelman's blog. He is, at least, a dude who seriously knows what he's talking about. http://en.m.wikipedia.org/wiki/Andrew_Gelman
Yes, I originally started http://news.ycombinator.com/item?id=4769667 with title "Andrew Gelman: I don't like this cartoon", but for some reason mods decided to get rid of "Andrew Gelman" in the title. I mean, the whole point is that it's a very famous statistician, not just another blogger ranting.
His name is spelled "Munroe", not "Monroe". Interestingly, the correct spelling of his name is literally the first thing you see when you click that link.
LW discussion of the comic: http://lesswrong.com/r/discussion/lw/fe5/xkcd_frequentist_vs...
Wait, hold on. Isn't the joke that the Bayesian statistician knows that paying back 50 dollars is meaningless if the sun exploded and we're all going to die? If we live, he gets 50 dollars. That's what I got out of it.
>More importantly, now that both you and perhaps Scott Adams have commented on my blog, I am very happy. If only I could think of some way of getting Berke Breathed to comment here, I think I could just retire!
Could someone point me to the "perhaps Scott Adams" comment? (is it in response to another blog entry?)
Could someone point me to the "perhaps Scott Adams" comment? (is it in response to another blog entry?)
Thanks! :)
(Now I see why it's perhaps Scott Adams)
(Now I see why it's perhaps Scott Adams)
I'm suspicious of his reason for bringing up Scott Adams and Berke Breathed.
[deleted]
AFAIK, the breakdown is this:
Frequentist is a special case of Bayesian. The Frequentists have much more mature tools, because they've been working on them since Gauss's day. Bayesians (especially less experienced ones) may claim that that Frequentists are old school, outdated, and don't teach undergrads the new Bayesian way of doing things.
Bayesian methods are more flexible and general, but are often slow (computationally), and can be too flexible. A Bayesian can prove anything. Frequentists have trouble eliminating some biases (because their tools aren't as flexible), but also have trouble purposely (or subconsciously) biasing their results.
I'm not going into the specifics of the methods here, just the source of their disagreements.
Frequentist is a special case of Bayesian. The Frequentists have much more mature tools, because they've been working on them since Gauss's day. Bayesians (especially less experienced ones) may claim that that Frequentists are old school, outdated, and don't teach undergrads the new Bayesian way of doing things.
Bayesian methods are more flexible and general, but are often slow (computationally), and can be too flexible. A Bayesian can prove anything. Frequentists have trouble eliminating some biases (because their tools aren't as flexible), but also have trouble purposely (or subconsciously) biasing their results.
I'm not going into the specifics of the methods here, just the source of their disagreements.
Almost everything written here is incorrect.
First of all, Bayes predates Gauss. It is inaccurate to suggest frequentist statistics predate Bayesian statistics.
Second, neither is a special case of the other.
Third, neither Bayesian methods nor frequentist methods are inherently "more flexible."
Bayesian statistics set up a model and infer model parameters by applying Bayes' rule to the data. Bayes' rule is an indisputable rule of probability.
For any model parameter b, the result of applying a bayesian model is a probability distribution for b given the available data. Criticisms of bayesian models typically center around the fact that you must use a "prior distribution" indicating the modeler's beliefs about b before seeing the data. Bayesian statisticians have a number of responses to this criticism (that some people find compelling, and others do not).
Frequentist methods build models that are justified by their properties in repeated resampling. For instance, a frequentist method is "unbiased" if, given multiple hypothetical samples, it would on average produce the correct parameter b. Frequentist hypothesis testing reports the probability of observing specified data given some assumption about b.
A standard criticism of frequentist methods is that a modeler wants a probability distribution for an unknown parameter given the known data... rather than knowing the probability of observing the realized data given some assumption about the parameter.
First of all, Bayes predates Gauss. It is inaccurate to suggest frequentist statistics predate Bayesian statistics.
Second, neither is a special case of the other.
Third, neither Bayesian methods nor frequentist methods are inherently "more flexible."
Bayesian statistics set up a model and infer model parameters by applying Bayes' rule to the data. Bayes' rule is an indisputable rule of probability.
For any model parameter b, the result of applying a bayesian model is a probability distribution for b given the available data. Criticisms of bayesian models typically center around the fact that you must use a "prior distribution" indicating the modeler's beliefs about b before seeing the data. Bayesian statisticians have a number of responses to this criticism (that some people find compelling, and others do not).
Frequentist methods build models that are justified by their properties in repeated resampling. For instance, a frequentist method is "unbiased" if, given multiple hypothetical samples, it would on average produce the correct parameter b. Frequentist hypothesis testing reports the probability of observing specified data given some assumption about b.
A standard criticism of frequentist methods is that a modeler wants a probability distribution for an unknown parameter given the known data... rather than knowing the probability of observing the realized data given some assumption about the parameter.
I would describe this breakdown as misleading at best.
Frequentists are not a special case of anything. Standard frequentist arguments make absolutely no sense from a Bayesian perspective. For instance there is no prior in which such a thing as repeated significance testing errors can possibly exist. Conversely frequentists can legitimately point to a lot of things that Bayesians do which are at best highly questionable. Such as picking a default prior that gives massively high probability to clearly unlikely scenarios. (Yes the Bayesian replies, but we could pick better priors. But, the frequentist retorts, in practice you don't.)
I think that it is best to learn both, then do whatever makes most sense for your circumstances.
Frequentists are not a special case of anything. Standard frequentist arguments make absolutely no sense from a Bayesian perspective. For instance there is no prior in which such a thing as repeated significance testing errors can possibly exist. Conversely frequentists can legitimately point to a lot of things that Bayesians do which are at best highly questionable. Such as picking a default prior that gives massively high probability to clearly unlikely scenarios. (Yes the Bayesian replies, but we could pick better priors. But, the frequentist retorts, in practice you don't.)
I think that it is best to learn both, then do whatever makes most sense for your circumstances.
I love this line.
The truth is, I genuinely didn’t realize Frequentists and Bayesians were actual camps of people—all of whom are now emailing me.
The truth is, I genuinely didn’t realize Frequentists and Bayesians were actual camps of people—all of whom are now emailing me.
Tribalism is so cute when it's not political.
I once saw David Cox interrupt an argument between the two camps at a conf to say something along the lines of Frequentist/Bayesian, whatever gets the job done for the problem at hand. I can't do it justice, but it was very dudely.
It sounds like Randall got bitten by some variant of Poe's law.
He should be doubly cautious of Rule 34 in this case.
http://xkcd.com/305/
http://xkcd.com/305/
How viewer must recognize who are Frequentists and Bayesian in porn? Of course it possible to place tattoos with formulas on bodies or put some blackboards in background (conventional plot about student and teacher?), but can it be more elegant? May be some joke about contraception?
Why Randall should almost apologize for a comic is a mystery to me.