Which Polls Fared Best (and Worst) in the 2012 Presidential Race(fivethirtyeight.blogs.nytimes.com)
fivethirtyeight.blogs.nytimes.com
Which Polls Fared Best (and Worst) in the 2012 Presidential Race
http://fivethirtyeight.blogs.nytimes.com/2012/11/10/which-polls-fared-best-and-worst-in-the-2012-presidential-race/
4 comments
It seems like taking 3 weeks worth of polls would lead to larger error sizes, especially towards Romney this year, since it did seem like Romney was faring better (although still trailing) 1-3 weeks out.
Take the current value of the dow jones, round it to the nearest dollar. Then create one hundred letters, each of which says "The dow jones will be at X, one week from now." where X is the current value plus or minus increments of 5 dollars up to +250 and -250. Then add some sort of hook to the end of the letter such as "If you would like to learn my secrets of predicting the stock market you can take my course for the reasonable cost of Y dollars." Then print out 100 copies of each letter and send them to random folks, perhaps some number of people who receive letters that end up accurately predicting the stock market will fall victim to your scam.
That's the same problem we're dealing with here. Presidential votes only happen once every four years, so it can be difficult to validate any sort of model. A hundred pundits make a hundred different predictions. And dozens of polls make different predictions as well. Does that mean that the pundits and the polls which ended up being closer to the truth are more accurate or does it just mean that they were lucky?
That's the same problem we're dealing with here. Presidential votes only happen once every four years, so it can be difficult to validate any sort of model. A hundred pundits make a hundred different predictions. And dozens of polls make different predictions as well. Does that mean that the pundits and the polls which ended up being closer to the truth are more accurate or does it just mean that they were lucky?
"Perhaps it won’t be long before Google, not Gallup, is the most trusted name in polling."
What is the deal with Gallup?
But how could you/they be so wrong? Are you a subsidiary of Fox? It only makes sense to me if there was money involved.
Polling is hard and easy to get wrong. That’s all there is to it. There is no need for crazy conspiracy theories.
Step 1: Ask people if they want to share who they voted for in as many places as possible.
Step 2: Add results.
Other than simple math, I'm not seeing what there is to mess up.
Step 2: Add results.
Other than simple math, I'm not seeing what there is to mess up.
Wow! You have just blown my mind! And to think that thousands of psychologists, sociologists and market researchers couldn’t figure that out for decades. What were they thinking?
The math is the easy part here. All the rest is hard.
How can you reach as many people as possible? How can you make sure that you reach a random sample of people? If you give up on that because it’s pretty hopeless, how can you make sure that the characteristics you pick to build your sample (for example age, gender, race, etc.) actually matter, are correctly chosen and that you have the right information about how your sample should look when it comes to those characteristics? (This problem can be rephrased as: How do you make sure your sample is representative? It is, however, important to unpack the complexity involved in that.) How do you know that those who tell you they will go vote will actually vote?
That’s not a complete list of problems. As you can see there are more than enough opportunities for error here.
Now, given Gallup’s poor performance compared to other polls I do not want to excuse them. They screwed up, there is not question about that. I do not think, however, that there is any need to make up conspiracy theories.
The math is the easy part here. All the rest is hard.
How can you reach as many people as possible? How can you make sure that you reach a random sample of people? If you give up on that because it’s pretty hopeless, how can you make sure that the characteristics you pick to build your sample (for example age, gender, race, etc.) actually matter, are correctly chosen and that you have the right information about how your sample should look when it comes to those characteristics? (This problem can be rephrased as: How do you make sure your sample is representative? It is, however, important to unpack the complexity involved in that.) How do you know that those who tell you they will go vote will actually vote?
That’s not a complete list of problems. As you can see there are more than enough opportunities for error here.
Now, given Gallup’s poor performance compared to other polls I do not want to excuse them. They screwed up, there is not question about that. I do not think, however, that there is any need to make up conspiracy theories.
I wonder if it actually works in that political direction. If I knew my preferred candidate was down perhaps I'd be more determined to get down the polling station. Conversely if he was clearly ahead, perhaps I'd be more inclined to skip voting. ( I'm talking theoretically, I always vote, it's the law here)
My understanding is that as a non-incumbent campaign, you ideally want voters to believe that your candidate is behind but gaining and that the result will be close.
That link (if you read it) is directly from Gallup -- I just posted it because it seemed relevant to your query.
Whether or not Gallup are delusional (or not) is up for debate.
Whether or not Gallup are delusional (or not) is up for debate.
"Our final estimate of registered voters was an unallocated 49% for Obama, 46% for Romney. The transition to likely voters moved that to the unallocated 49% Romney, 48% Obama."
Ouch. So their "transition to likely voters" estimation is just completely off.
Ouch. So their "transition to likely voters" estimation is just completely off.