The Easiest Data Analysis Mistake to Make(blog.statwing.com)
blog.statwing.com
The Easiest Data Analysis Mistake to Make
http://blog.statwing.com/easiest-data-analysis-mistake-to-make/
8 comments
I believe the "dumbbell" effect you are referring to is Simpson's paradox: http://en.wikipedia.org/wiki/Simpson's_paradox
As you point out obviously you can see it in a 2D scatter plot, but you have to select the correct two variables.
As you point out obviously you can see it in a 2D scatter plot, but you have to select the correct two variables.
I think, typically, if you've gone to the trouble of calculating variance and correlation, you would have also calculated the median and mode of these datasets. The differences would have been obvious with those basic analyses.
How do you define the mode of a data set where all the values are different?
So a concept you find in a beginner stats textbook is now 'news'? Definition of blogspam surely...
I would agree with you. Anyways I hope people were not making such errors and if we were, then at least someone would benefit from reading this!
My favourite part of this (aside from the message) is that it links back to this exact discussion page.
An equally common mistake is to visualize without analyzing :-)
It's also worth mentioning "dumbbell" data sets. Two clusters of data, each of which have a independent, meaningful correlation in them, can easily leverage a linear regression into a meaningless line passing through the two clusters. That's a pretty common issue with high dimensional data (obviously you can see it in a 2D scatter plot), and it's not easily caught short of looking at regression diagnostic statistics.