US behavioural research studies skew positive(nature.com)
nature.com
US behavioural research studies skew positive
http://www.nature.com/news/us-behavioural-research-studies-skew-positive-1.13599
9 comments
Off topic, but talking about truth in advertising: on your second link, there's a link at the bottom talking about a mummy made from McDonald's burgers that will never deteriorate. I'm tired of seeing this incredibly misleading claim that this is the reason the burgers are unhealthy.
Cooked and salted meat is, in essence, incredibly well preserved. This concept has been known since ancient times. Bacteria requires moisture to cause decomposition, and cooking/salting meat causes it to dry faster than it will rot. Once it's dried, it pretty much won't deteriorate any more. This is the concept behind pemmican and beef jerky.
The point is, there may be misleading advertisements, but this argument isn't any less disingenuous than the Big Mac that's too big to fit in a box.
Cooked and salted meat is, in essence, incredibly well preserved. This concept has been known since ancient times. Bacteria requires moisture to cause decomposition, and cooking/salting meat causes it to dry faster than it will rot. Once it's dried, it pretty much won't deteriorate any more. This is the concept behind pemmican and beef jerky.
The point is, there may be misleading advertisements, but this argument isn't any less disingenuous than the Big Mac that's too big to fit in a box.
First, criticizing a link on a link, really?
Second, I think you missed my point that a set of well-produced pictures of unknown provenance (really, could you figure out the source? I gave up after several clicks) is a more effective way of selling my point than providing substantive information would be.
Second, I think you missed my point that a set of well-produced pictures of unknown provenance (really, could you figure out the source? I gave up after several clicks) is a more effective way of selling my point than providing substantive information would be.
Oh no, I wasn't criticizing you or your link. I was just trying to add to the discussion that misleading advertisements are a problem, but there's plenty of misleading propaganda on the other side as well.
Nothing against your point at all. Sorry for the confusion.
Nothing against your point at all. Sorry for the confusion.
Professor Uri Simonsohn wasn't mentioned in this particular article, but he could have been. He has published several peer-reviewed papers on "p-hacking" and "researcher degrees of freedom" and other tricks that are used (and can be detected after the fact with careful statistical analysis of published papers) to inflate the significance of behavior research results. His faculty website
http://opim.wharton.upenn.edu/~uws/
includes links to many of his papers. Not only are Simonsohn's papers a good lesson in how to use statistics to detect sloppy research practices, they are also often laugh-out-loud funny.
http://opim.wharton.upenn.edu/~uws/
includes links to many of his papers. Not only are Simonsohn's papers a good lesson in how to use statistics to detect sloppy research practices, they are also often laugh-out-loud funny.
We should be skeptical of this study, though. From my experience, behavioral researchers tend to exaggerate their findings.
Half expected to find Douglas Hofstadter credited on the paper abstract. A US Scientist publishing a paper making the claim that US scientists exaggerate their findings is on the same continuum as guards who always tell lies and gramophones that play records that destroy themselves.
Can confirm. Just like with many other types of research. It's part of our culture to exaggerate and be deceptive. It is how our politics, our government, our culture, our business, and how our economy as a whole function. Before I get a bunch of "everyone does it", the reality is that those who we should aspire to compare ourselves to are quite remarkably different regarding that characteristic. But go right ahead, compare America to all the dysfunctional, corrupt, fraying societies of the world. I choose to look up, rather than down.
Edit: I would like to point to the following story in the HN feed regarding Germany's election as a point of reference that things don't have to be the way that we think they are. https://news.ycombinator.com/item?id=6283280
Edit: I would like to point to the following story in the HN feed regarding Germany's election as a point of reference that things don't have to be the way that we think they are. https://news.ycombinator.com/item?id=6283280
It's interesting how the title of the article is an example of the phenomenon itself. Study finds US research is more likely to be wrong, and a US based journal reports this as "skew positive": they are just more optimistic, that's all.
I do think it's important to note that they're generally wrong in the same direction.. they tend to be overly positive, not overly negative (or an even mix of both). That particular skew is more likely to implicate the "publish or perish" incentive system as the culprit than the other two.
My first thought on seeing the title was, "There must be a behavioral basis to this."
I'm interested in how much of this is ethics, a push to get more drastic results, or what is accepted by journals. The last case can be very true too - if there are too many journal submissions, it may be puffery on the referees instead of the publishers.
One way to test this is to see if there is a difference in the results from tenured versus untenured professors, or even in individuals post-tenure. If puffery shrinks post-tenure, then it really is the publish or perish mindset.
I'm interested in how much of this is ethics, a push to get more drastic results, or what is accepted by journals. The last case can be very true too - if there are too many journal submissions, it may be puffery on the referees instead of the publishers.
One way to test this is to see if there is a difference in the results from tenured versus untenured professors, or even in individuals post-tenure. If puffery shrinks post-tenure, then it really is the publish or perish mindset.
In my experience as a european ex-PhD researcher, this is not only US and not only behavioural studies.
Authors are under pressure to publish as much as they can, and positive results are not only easier to publish on journals but also provide good deliverables for research grant review committees and help show that funding was well spent. Not to mention a good reference for the next research proposal, showing that money will be well invested.
Authors are under pressure to publish as much as they can, and positive results are not only easier to publish on journals but also provide good deliverables for research grant review committees and help show that funding was well spent. Not to mention a good reference for the next research proposal, showing that money will be well invested.
<tinfoilhat>
I've never believed otherwise. This is why when I'm debating an issue with someone and they throw up research papers and claim "See? science says so!", I don't really know what to say. I don't think issue is specific to behavior-related studies either. I highly suspect USA's science data when it's related to something that makes lots of money or promotes a certain predominant view point. Not to say it doesn't happen in other countries, but we're talking about USA right now. Specifically things like:
- Studies show corn syrup not harmful
- Studies show cellphones don't cause cancer
- Studies show 1 in 2 children need meds for ADD
These are not exact quotes, just giving a general idea. I suspect these things because if they turn out not to be true, millions of dollars to some big corp entity somewhere would be lost. Or maybe they need science-data to justify their funding. There's just too much money tied up in scientific results to trust they haven't been tampered with. What I like to see, is when many countries do studies and they all come up with similar results.
</tinfoilhat> EDIT: Yeah, I see this getting downvoted but I stand by what I said. Science is influenced by money a great deal these days; you should reserve a little bit of skepticism for scientific results that have close ties to supporting a belief required for bigcorp to keep making money.
I've never believed otherwise. This is why when I'm debating an issue with someone and they throw up research papers and claim "See? science says so!", I don't really know what to say. I don't think issue is specific to behavior-related studies either. I highly suspect USA's science data when it's related to something that makes lots of money or promotes a certain predominant view point. Not to say it doesn't happen in other countries, but we're talking about USA right now. Specifically things like:
- Studies show corn syrup not harmful
- Studies show cellphones don't cause cancer
- Studies show 1 in 2 children need meds for ADD
These are not exact quotes, just giving a general idea. I suspect these things because if they turn out not to be true, millions of dollars to some big corp entity somewhere would be lost. Or maybe they need science-data to justify their funding. There's just too much money tied up in scientific results to trust they haven't been tampered with. What I like to see, is when many countries do studies and they all come up with similar results.
</tinfoilhat> EDIT: Yeah, I see this getting downvoted but I stand by what I said. Science is influenced by money a great deal these days; you should reserve a little bit of skepticism for scientific results that have close ties to supporting a belief required for bigcorp to keep making money.
You're getting downvoted because
- You just made up article titles to prove your point.
- You call out the USA for no particular reason, because your arguments apply equally to other countries.
- You don't provide any alternative to "science", which according to you is inherently flawed, which makes pretty much any discussion with you pointless.
- You just made up article titles to prove your point.
- You call out the USA for no particular reason, because your arguments apply equally to other countries.
- You don't provide any alternative to "science", which according to you is inherently flawed, which makes pretty much any discussion with you pointless.
Okay, fair enough. let me clarify:
- Those points I used are real arguments being made today. I don't have exact quotes/links, but they do exist. There are people who believe those 3 items are perfectly correct. I admit item#3 is a bit of an exaggeration, but there is an alleged issue of over-diagnosis of ADD. I'm not saying they're wrong, but just saying to keep in mind that there is a lot of money that would be lost if those 3 things were to be discovered false.
- I called out the USA because this article is about the USA. Also I noted in italics that I wasn't saying it was only USA.
- Indeed, I don't have any alternative. Though I did mention I like the idea of multiple countries doing the study to see if everyone reaches the same conclusion. I'm just saying there is something to be said for a bit of skepticism. We know how long it took for science to decide tobacco was bad for you. It wasn't like they didn't know before, it just took a long time to push past the greed & corruption that kept the data hidden. Just saying to always keep a critical mind with these things. Afterall, this article here is calling out exactly what I'm claiming.
EDIT: And I see my original comment has been upvoted out of the graveyard during my commute to work, so I guess there are people who agree with what I'm trying to say.
- Those points I used are real arguments being made today. I don't have exact quotes/links, but they do exist. There are people who believe those 3 items are perfectly correct. I admit item#3 is a bit of an exaggeration, but there is an alleged issue of over-diagnosis of ADD. I'm not saying they're wrong, but just saying to keep in mind that there is a lot of money that would be lost if those 3 things were to be discovered false.
- I called out the USA because this article is about the USA. Also I noted in italics that I wasn't saying it was only USA.
- Indeed, I don't have any alternative. Though I did mention I like the idea of multiple countries doing the study to see if everyone reaches the same conclusion. I'm just saying there is something to be said for a bit of skepticism. We know how long it took for science to decide tobacco was bad for you. It wasn't like they didn't know before, it just took a long time to push past the greed & corruption that kept the data hidden. Just saying to always keep a critical mind with these things. Afterall, this article here is calling out exactly what I'm claiming.
EDIT: And I see my original comment has been upvoted out of the graveyard during my commute to work, so I guess there are people who agree with what I'm trying to say.
I am skeptical, which is why I don't put much value in your comment. Want to convince me that corn oil is bad or that cell phones cause cancer?
Cite proper studies on these things, and don't cherry pick results you like, give me a fair sampling of all research. If the data supports your claim, I'll believe it.
Cite proper studies on these things, and don't cherry pick results you like, give me a fair sampling of all research. If the data supports your claim, I'll believe it.
"What I like to see, is when many countries do studies and they all come up with similar results."
Leaving aside the fact that different countries are basically irrelevant here, what you are describing is a scientific consensus. These are always founded not on a single or even a handful of studies, but rather on many, many studies and mountains of evidence, meta-studies analyzing the studies, independent verification of results, and so forth. Science doesn't move forward by individual studies - that's a common misconception - it moves forward when enough of those studies accrue that a clear consensus emerges and shifts the prevailing paradigm. This is also why the lone brilliant genius who overturns everything with their piercing insight is such a stupid meme - even Einstein had to verify his results and wait for the rest of the community to double check his work.
And that's why it's a fallacy to place too much weight on a single study, and an even bigger fallacy to disregard places where there is, in fact, a clear consensus - e.g. cellphones don't cause cancer, global warming and evolution are real, etc...
Leaving aside the fact that different countries are basically irrelevant here, what you are describing is a scientific consensus. These are always founded not on a single or even a handful of studies, but rather on many, many studies and mountains of evidence, meta-studies analyzing the studies, independent verification of results, and so forth. Science doesn't move forward by individual studies - that's a common misconception - it moves forward when enough of those studies accrue that a clear consensus emerges and shifts the prevailing paradigm. This is also why the lone brilliant genius who overturns everything with their piercing insight is such a stupid meme - even Einstein had to verify his results and wait for the rest of the community to double check his work.
And that's why it's a fallacy to place too much weight on a single study, and an even bigger fallacy to disregard places where there is, in fact, a clear consensus - e.g. cellphones don't cause cancer, global warming and evolution are real, etc...
US research findings are America Strong
We also take it for granted that marketing involves a certain amount of exaggeration and deception, and that anyone taken it by dishonest marketing only has themselves to blame, as long as the dishonesty is kept within certain culturally accepted limits that keep getting stretched further and further [2]. Maybe some advertising can get away with honesty, but that's the exception, and it's arrogant to think you're exceptional [3].
In other words, lying about ourselves is now culturally expected.
Perhaps in the next generation, an unwillingness to aggressively sell yourself will be seen as just as sad and self-defeating as not completing high school. How do these poor kids not see how dropping out of school affects their life? It's so tragic, we think now. Just sit in the classroom and do your work. Perhaps in twenty years we'll be lamenting the stubborn inability of disadvantaged kids to exaggerate their achievements. How can these poor kids just sit around doing their schoolwork and not sell themselves aggressively? It's so tragic. Just smile manically and say you're an awesome, passionate rock star. Is that so fucking hard to understand? What are parents teaching these days? Shit, we should really be funding Head Start.
[1] http://www.danpink.com/books/to-sell-is-human/
[2] This link might not have much substance, but it's slick and vivid: http://geekologie.com/2012/01/bogus-burgers-fast-food-false-...
[3] Honesty in advertising is such an unthinkable idea that it has been relegated to the place our culture reserves for statements that are so false we cannot imagine anything falser, the bogus trend piece: http://www.steamfeed.com/is-the-next-frontier-in-advertising...