Ego depletion theory may have been debunked(slate.com)
slate.com
Ego depletion theory may have been debunked
http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.single.html
24 comments
Planet Money did a podcast that was basically about this: http://www.npr.org/sections/money/2016/01/15/463237871/episo...
There seems to be a few issues at play. First is the "file drawer effect". If the experiment doesn't show a positive (or surprisingly negative) outcome, then it's most likely to get filed into a drawer and forgotten.
The second issue is just random luck. Suppose you flip a coin 10 times, then it's possible that you might get a run of nine heads and one tail. Based on that experiment you could publish a paper that the coin has a 90% chance of flipping heads. That would be a surprising result, and it's likely that it would be debunked if anyone tried to reproduce it.
The third issue is that it's exceedingly rare for anyone to try reproducing an experiment. There's no grant money for reproducing experiments and regardless of the outcome it's unlikely to advance a researcher's career.
Together these mean that most published papers could very well be the product of random luck that will never be verified.
There seems to be a few issues at play. First is the "file drawer effect". If the experiment doesn't show a positive (or surprisingly negative) outcome, then it's most likely to get filed into a drawer and forgotten.
The second issue is just random luck. Suppose you flip a coin 10 times, then it's possible that you might get a run of nine heads and one tail. Based on that experiment you could publish a paper that the coin has a 90% chance of flipping heads. That would be a surprising result, and it's likely that it would be debunked if anyone tried to reproduce it.
The third issue is that it's exceedingly rare for anyone to try reproducing an experiment. There's no grant money for reproducing experiments and regardless of the outcome it's unlikely to advance a researcher's career.
Together these mean that most published papers could very well be the product of random luck that will never be verified.
Obligatory xkcd: https://xkcd.com/882/
To me, it feels on the face of it, the experiments are really really reductionist and fail to recognise the role of human subjects and social cues in the equation.
For instance, in the first experiment outlined they had one set of subjects eat radishes instead of cookies before trying to solve an impossible puzzle. Now, if you are a subject and the first thing you are asked to do is eat radishes, isn't this a pretty good signal that you are probably being messed with? The logical reaction to that is to conclude that there's a greater chance that the puzzle is also some sort of trickery and you are much less likely to spend a long time trying to solve it.
For instance, in the first experiment outlined they had one set of subjects eat radishes instead of cookies before trying to solve an impossible puzzle. Now, if you are a subject and the first thing you are asked to do is eat radishes, isn't this a pretty good signal that you are probably being messed with? The logical reaction to that is to conclude that there's a greater chance that the puzzle is also some sort of trickery and you are much less likely to spend a long time trying to solve it.
I was compelled to participate in this sort of nonsense when meeting my general education requirements in college. It was time-consuming and the experiments were ridiculous, so I always behaved erratically as a "screw you" gesture to the university.
The mix of students in PSY 101 were people like me stuck with taking it and people who wanted to enter the field. I imagine both populations skewed the results.
The mix of students in PSY 101 were people like me stuck with taking it and people who wanted to enter the field. I imagine both populations skewed the results.
The best (and only) thing to do now is to come completely clean and develop new and much more rigorous standards for determining both correlation and causality.
The practice of science is a very human affair. That means politics and clan thinking. Science, like the press, develops a popular "narrative" about how things work. New research tends to magically confirm that narrative.
I suspect this has been going on for a very long time.
I love science. And I understand that there are a lot of demagogues that want to create these false battles between science and the ignorant. For many of those folks, they will want to circle the wagons. Don't let folks know how bad things are!
To those folks, I'd ask them to remember that the strength of science is its tentative nature and ability to adapt. Sure, people aren't so good at that, but science is. Stick to that and do your best to toss emotions out of it.
The practice of science is a very human affair. That means politics and clan thinking. Science, like the press, develops a popular "narrative" about how things work. New research tends to magically confirm that narrative.
I suspect this has been going on for a very long time.
I love science. And I understand that there are a lot of demagogues that want to create these false battles between science and the ignorant. For many of those folks, they will want to circle the wagons. Don't let folks know how bad things are!
To those folks, I'd ask them to remember that the strength of science is its tentative nature and ability to adapt. Sure, people aren't so good at that, but science is. Stick to that and do your best to toss emotions out of it.
The optimistic side of all this is that what we are seeing is the advancement of science. It certainly didn't happen in one experiment- not even in 100- but taking something that is considered "accepted" and throwing it out on further evidence is a good thing and says ultimately positive things about the field.
Experiments in areas with huge numbers of degrees of freedom are difficult to sort out.
I mean, if we just take just the chocolate cookie experiment, and assume that we can replicate that particular experiment. We run 100 versions of that experiment and find that students will attempt the impossible for about twice as long as those who had to eat radishes.
What is the most general thing that we can conclude?
How about if we find the same result with candy vs celery? Twinkies vs nothing?
What can we possibly be sure we learned from such experiments?
Experiments in areas with huge numbers of degrees of freedom are difficult to sort out.
I mean, if we just take just the chocolate cookie experiment, and assume that we can replicate that particular experiment. We run 100 versions of that experiment and find that students will attempt the impossible for about twice as long as those who had to eat radishes.
What is the most general thing that we can conclude?
How about if we find the same result with candy vs celery? Twinkies vs nothing?
What can we possibly be sure we learned from such experiments?
> The best (and only) thing to do now is to come completely clean and develop new and much more rigorous standards for determining both correlation and causality.
The question then would be who would want to provide funds for that? It is not the financial backers who have problems with the current methods.
I don't think the butter vs margarine or the tea vs coffee debate will be settled within my lifetime ;)
The question then would be who would want to provide funds for that? It is not the financial backers who have problems with the current methods.
I don't think the butter vs margarine or the tea vs coffee debate will be settled within my lifetime ;)
Funding isn't really a barrier to that solution. In fact, it's part of the problem. Besides, I'd wager that most people who do great work don't make as much as they are worth, and so progress is the work of passionate partial-volunteers.
It should be undergraduate work to replicate studies. It would fix all this dumb shit.
Didn't you read the Memo?
Science is a lot like sausage processing. The end result may range from the quite passable to the quite good, but if you actually saw the raw materials and the process, you'd be more discriminant on what you put in your plate.
Science is a lot like sausage processing. The end result may range from the quite passable to the quite good, but if you actually saw the raw materials and the process, you'd be more discriminant on what you put in your plate.
One possible advantage the "harder" sciences have is that another study that builds on the previous one can in some ways act as a replication.
If I publish something claiming bacteria has a gene X that produces phenotype Y, then a subsequent study that transfers gene X into a different bacteria must also observe Y. Otherwise something is fishy, and often the second set of researchers will go back and reexamine the original result.
It's rare that work in science is directly, point for point replicated, it's unsexy, expensive work. The fact that the harder sciences have somewhat of a built in control mitigates but by no means eliminates the risk of bad science surviving.
I can't think of an analogous process in the softer sciences. Seeing if a phenomenon is replicated in a different context does not implicitly re-test the original work in the same way.
If I publish something claiming bacteria has a gene X that produces phenotype Y, then a subsequent study that transfers gene X into a different bacteria must also observe Y. Otherwise something is fishy, and often the second set of researchers will go back and reexamine the original result.
It's rare that work in science is directly, point for point replicated, it's unsexy, expensive work. The fact that the harder sciences have somewhat of a built in control mitigates but by no means eliminates the risk of bad science surviving.
I can't think of an analogous process in the softer sciences. Seeing if a phenomenon is replicated in a different context does not implicitly re-test the original work in the same way.
>If I publish something claiming bacteria has a gene X that produces phenotype Y, then a subsequent study that transfers gene X into a different bacteria must also observe Y.
There is no guarantee (and it often isn't the case) that the phenotype observed in the first organism is purely the result of gene X; its effects may be context-dependent.
To take this a little further, I think it's plausible that a soft scientific question differs from a hard one specifically in terms of how easy it is to separate the phenomenon in question from its context.
There is no guarantee (and it often isn't the case) that the phenotype observed in the first organism is purely the result of gene X; its effects may be context-dependent.
To take this a little further, I think it's plausible that a soft scientific question differs from a hard one specifically in terms of how easy it is to separate the phenomenon in question from its context.
One of the biggest things about the "harder" sciences is that there's usually a whole engineering discipline behind them as well. Nothing reproduces a result as effectively as a host of engineers building real things on the back of it. If it doesn't work, we'll find out pretty soon, and if it does work then we'll have lots of evidence to show it.
The problem with results like "ego depletion" is that their utility is fuzzy at best. With no clear application, we don't have a body of practical results using it.
The problem with results like "ego depletion" is that their utility is fuzzy at best. With no clear application, we don't have a body of practical results using it.
Well, the problem is that the end result was proof of ego depletion. We can move the goalposts and say, "Wait, no this new nullification is the true end result," but those 100 studies ruled the roost for quite some time.
The reality is that the end results are usually terrible and a lot of science out there is straight up junk science designed to chase grants and to lean on journal biases to get published, especially in the social sciences. The problem is we don't really have a better alternative so working with a clock that's right twice a day is much better than having no clocks at all. We might have hit organizational limits on how well we can do science on this level. With funding comes politics, with journals comes bias, with publishing comes perverse incentives, challenging big egos is difficult, unknowns have to 'pay their dues', small universities and institutions don't have influence compared to their larger peers, the sometimes political tenure process, etc.
Hell, the article itself even calls out Carter's relatively young age or for not being tenured in an almost "Can we trust this young nobody," way.
The reality is that the end results are usually terrible and a lot of science out there is straight up junk science designed to chase grants and to lean on journal biases to get published, especially in the social sciences. The problem is we don't really have a better alternative so working with a clock that's right twice a day is much better than having no clocks at all. We might have hit organizational limits on how well we can do science on this level. With funding comes politics, with journals comes bias, with publishing comes perverse incentives, challenging big egos is difficult, unknowns have to 'pay their dues', small universities and institutions don't have influence compared to their larger peers, the sometimes political tenure process, etc.
Hell, the article itself even calls out Carter's relatively young age or for not being tenured in an almost "Can we trust this young nobody," way.
> "Can we trust this young nobody," way.
That's normal science at work. Old sages are biased, like everybody else. Fortunately, they are mortal too [1], and science keeps moving forward 'one funeral at a time', as the saying goes.
The problem is with people who want to substitute faith in the Divine Providence of God, with faith in the (eventual) Unfailibility of Science. We are just a bunch of social primates, after all. We were never meant to peek into the Deep Mysteries of Existence, anyways... which does not mean we cannot use our relatively advanced intelects to figure out a bunch of clever and useful tricks.
======= edit: comment [1]
[1] Something that is nightmare fuel to me is that one day we are going to stumble upon some serious life extending technology, and then we will all end up under the thumb of a bunch of ancient, obstinated fools who cannot/won't accept the limits of their own leadership skills.
That's normal science at work. Old sages are biased, like everybody else. Fortunately, they are mortal too [1], and science keeps moving forward 'one funeral at a time', as the saying goes.
The problem is with people who want to substitute faith in the Divine Providence of God, with faith in the (eventual) Unfailibility of Science. We are just a bunch of social primates, after all. We were never meant to peek into the Deep Mysteries of Existence, anyways... which does not mean we cannot use our relatively advanced intelects to figure out a bunch of clever and useful tricks.
======= edit: comment [1]
[1] Something that is nightmare fuel to me is that one day we are going to stumble upon some serious life extending technology, and then we will all end up under the thumb of a bunch of ancient, obstinated fools who cannot/won't accept the limits of their own leadership skills.
Here's how I console myself: if those old obstinate fools are actually poor leaders, they will make worse decisions as a result, and thus not be as effective at marginalizing error as they should be, so they will be unable to avoid the unforeseen things that will get them killed.
Is "discriminant" a word now? I thought performant had pretty much exhausted the possibilities.
eudox(2)
As someone who was once a bench scientist, I'd love to think I was immune to these kinds of pitfalls. After all, we control things carefully in the lab, we can control many more of the inputs to an experiment, test a hypothesis by different avenues, have truly independent controls...
But this is a very large literature, that in effect by sheer volume of work was doing many of those things. This is quite humbling.
That said, I do think that medical science, though massively flawed (see previous discussion on clinical trials no longer revealing positive results https://news.ycombinator.com/item?id=11209778) there is a way to make it so we can trust science. By:
1.) as previously said given a hypothesis, it must be confirmed by 2 independent approaches (if this is true, then this other thing should be true, is it?)
2.) publishing more negative results, removing the incentive for the file-drawer effect mentioned by @jobu
3.) require much larger sample sizes for studies to be published, and take effect size properly into account when judging whether the outcome is valid. One interesting way to do this:
http://www.stat.columbia.edu/~gelman/research/published/retr...
But this is a very large literature, that in effect by sheer volume of work was doing many of those things. This is quite humbling.
That said, I do think that medical science, though massively flawed (see previous discussion on clinical trials no longer revealing positive results https://news.ycombinator.com/item?id=11209778) there is a way to make it so we can trust science. By:
1.) as previously said given a hypothesis, it must be confirmed by 2 independent approaches (if this is true, then this other thing should be true, is it?)
2.) publishing more negative results, removing the incentive for the file-drawer effect mentioned by @jobu
3.) require much larger sample sizes for studies to be published, and take effect size properly into account when judging whether the outcome is valid. One interesting way to do this:
http://www.stat.columbia.edu/~gelman/research/published/retr...
Not to be an ass - but I've got to ask. Does the "reproducibility crisis" have any policy or political implications for the "global warming" threat? I know nothing about climatology, and since I can't judge the "conservative" critiques of the consensus on their own merits, my tendency is to dismiss them as politically motivated and hence unreliable. As folks keep saying, "The science is settled." But if the science was settled in this instance, is it possible that my strategy for sorting through the debate - which I suspect is in fact a pretty common strategy - is in fact premature?
One difference here which might give you an idea of where to place your bets is that the conservative opposition to global warming is based largely on studies financed by the energy industry. The willpower research is being called into question by researchers who don't have a link to any industry with a vested interest in the outcome.
Is this an airtight argument? No. There is still the possibility that global warming research has analogous methodological issues. But the conflict of interest definitely raises the burden of proof.
An additional difference is that climatology research is much better founded than willpower research is. Psychology is notoriously tricky to test since you can't break apart a brain and watch all the parts interacting. Compared to psychology, the systems that climatologists study are downright simple, and we have good models of how the systems behave.
Is this an airtight argument? No. There is still the possibility that global warming research has analogous methodological issues. But the conflict of interest definitely raises the burden of proof.
An additional difference is that climatology research is much better founded than willpower research is. Psychology is notoriously tricky to test since you can't break apart a brain and watch all the parts interacting. Compared to psychology, the systems that climatologists study are downright simple, and we have good models of how the systems behave.
I agree that the conflict of interest definitely raises the bar. But of course, that's what I immediately hear from my conservative friends as well: that papers which support the consensus are more likely to be published, that studies likely to confirm the consensus are more likely to be funded, and researchers who question that consensus are likely to be blackballed by their colleagues. Not being on the inside, I don't have any special insight as to whether or how strong the dynamic operates, of course. Which means that as an non-scientist, I sometimes feel like I'm in a pretty poor state to actually know what's true. Like I said, my instinct is to respect the consensus. But the evidence seems to have increased over the last decade or so that "consensus" can be a pretty poor pointer to the truth. And I want to take that evidence into account as well.
Actually, scientists have a huge incentive to overthrow an existing regime if it's bad but nobody knows it yet. It will make your career if you are able to prove it. The evidence required is high if what you're trying to disprove is well supported, but if you can provide that evidence there will be a wellspring of PhD students ready to ride that wave and build their careers.
For climate deniers, the evidence just isn't there. There's no wave to ride.
For climate deniers, the evidence just isn't there. There's no wave to ride.
Oh yes. The human brain is the most organizationally complex object in the solar system. People outside the brain sciences are bombarded with pretty photos of fMRI blobs, and don't understand how little we still understand about it.
It's a fair question. But, no, there isn't a similar risk.
Psychology is among the social sciences. That field is experiencing the reproducibility crisis, and it's been a long time coming. The subject matter is extremely difficult to study in an objective way, and the smallest of nuances can distort results (e.g. the article mentions taking a test on computer vs on paper). Also, humans aren't passive subjects, they change and react to being studied.
That's just not the case in the physical sciences like chemistry, physics, etc. There isn't any similar crisis there at all.
To be fair, climate science is somewhere in the middle, in that it can't be studied in the fully rigorous way that physics can, with controlled experiments. However, that middle ground contains other very solid fields of science, like evolutionary biology. Massive reproducibility crises just haven't happened in those areas.
The social sciences are just different. Perhaps it's naive to think that a simple law like "ego depletion" could possibly work consistently, given how complex human beings are. That's the core issue, and climate science doesn't suffer from it.
Psychology is among the social sciences. That field is experiencing the reproducibility crisis, and it's been a long time coming. The subject matter is extremely difficult to study in an objective way, and the smallest of nuances can distort results (e.g. the article mentions taking a test on computer vs on paper). Also, humans aren't passive subjects, they change and react to being studied.
That's just not the case in the physical sciences like chemistry, physics, etc. There isn't any similar crisis there at all.
To be fair, climate science is somewhere in the middle, in that it can't be studied in the fully rigorous way that physics can, with controlled experiments. However, that middle ground contains other very solid fields of science, like evolutionary biology. Massive reproducibility crises just haven't happened in those areas.
The social sciences are just different. Perhaps it's naive to think that a simple law like "ego depletion" could possibly work consistently, given how complex human beings are. That's the core issue, and climate science doesn't suffer from it.
Out of curiosity, is the downvote because of my tendency to dismiss conservative critiques? Or because I wonder if that tendency might be wrong?
One of the interesting things that occurs to me when I read this is the potential failure for researchers to remember one great way to test a hypothesis is to test a competing hypothesis.
So for example, in the cookie/radish story, were all participants exposed to the cookie smell? Were they aware some people got cookies and some people got radishes?
If so, the reduced performance of the radish group could be related to a feeling of being pushed down in the social hierarchy, a self-esteem blow, because they were denied a perk other participants received.
I intuitively feel some of these psychological experiments suffer from this. That there is another phenomenon which is really close to the one they think they are studying, which is how they get it wrong. Because the brain/sense of self are both such complex interrelated systems and it is extremely hard to make distinctions between different elements of the process.
So for example, in the cookie/radish story, were all participants exposed to the cookie smell? Were they aware some people got cookies and some people got radishes?
If so, the reduced performance of the radish group could be related to a feeling of being pushed down in the social hierarchy, a self-esteem blow, because they were denied a perk other participants received.
I intuitively feel some of these psychological experiments suffer from this. That there is another phenomenon which is really close to the one they think they are studying, which is how they get it wrong. Because the brain/sense of self are both such complex interrelated systems and it is extremely hard to make distinctions between different elements of the process.
> one great way to test a hypothesis is to test a competing hypothesis.
Isn't it worse than that? A statistical hypothesis test will essentially just let you refute the null hypothesis, it provides no confirmation at all for a specific alternative hypothesis (unless of course it's the only possible alternative hypothesis).
A funny story comes to mind: I studied physics at university but did one semester of psychology. One day in class we did an experiment that was supposed to show that memory is organized into something called schemas, but the experiment seemed severely flawed to me and I couldn't help protest that a much more reasonable interpretation of the outcome was that "people who have actually been told a story will be better at retelling the story, at least when compared to people who have not been told the story." On hearing this criticism my professor became silent for a moment and then said something along the lines of:
"I have a colleague who's son borrows his bike when his own is broken. This colleague sometimes says that his son thinks that his bike will mend itself if left to its own devices. Well, that's one way to interpret the data..."
I thought it was a brilliant way to acknowledge my criticism. But my fellow students unfortunately didn't seem to get it. I guess psychology was never for me. :)
Isn't it worse than that? A statistical hypothesis test will essentially just let you refute the null hypothesis, it provides no confirmation at all for a specific alternative hypothesis (unless of course it's the only possible alternative hypothesis).
A funny story comes to mind: I studied physics at university but did one semester of psychology. One day in class we did an experiment that was supposed to show that memory is organized into something called schemas, but the experiment seemed severely flawed to me and I couldn't help protest that a much more reasonable interpretation of the outcome was that "people who have actually been told a story will be better at retelling the story, at least when compared to people who have not been told the story." On hearing this criticism my professor became silent for a moment and then said something along the lines of:
"I have a colleague who's son borrows his bike when his own is broken. This colleague sometimes says that his son thinks that his bike will mend itself if left to its own devices. Well, that's one way to interpret the data..."
I thought it was a brilliant way to acknowledge my criticism. But my fellow students unfortunately didn't seem to get it. I guess psychology was never for me. :)
I'm in the middle of Thinking Fast and Slow which revolves heavily around ego depletion and this is very surprising. They cite a ton of evidence and the guy won a Nobel Prize for his work. Hard to imagine so much of it could be flat wrong.
He also said there "you cannot argue with the evidence"... I'm glad someone is trying.
The thing is, you can always argue with evidence, you just need newer evidence.
There is only a few special case exceptions where this presents a problem, which are mostly thought experiments in astronomy where the evidence will be lost in the future.
There is only a few special case exceptions where this presents a problem, which are mostly thought experiments in astronomy where the evidence will be lost in the future.
It's a weird coincidence that I just began listening to a podcast he is regularly featured on, Hidden Brains from NPR. I wonder if he'll talk about these events at all. It could be a dark time for him.
He might be surprised, but his whole premise was being evidence-based and upending old ways of thinking with rigorous experimentation. If this new study holds up to scrutiny, I think to think he'd be glad to have better science.
> Just two of the [24] research groups produced a significant, positive effect
If only they had used green jelly beans instead [^1]...
This is not limited to social psychology. Scientific publishing has a strong bias towards positive results. As long as null results are filed away, this kind of thing will continue to happen.
[1]: https://xkcd.com/882/
If only they had used green jelly beans instead [^1]...
This is not limited to social psychology. Scientific publishing has a strong bias towards positive results. As long as null results are filed away, this kind of thing will continue to happen.
[1]: https://xkcd.com/882/
> "It’s easier to publish stuff that tears something down than it is to build something up,” he told me wearily. “It’s not an enjoyable time. It’s not much fun.”
If your proposed effect is real then you should have no problem in proving it. If it doesn't exist, or if it does, but is much smaller and doesn't apply nearly as broadly as you have stated, then of course you will have "not much fun" proving it. Especially if that fun means making tens or even hundreds of experiments, then exhaustively processing data to find some experiments with positive result and discarding the rest, then again looking for explanations why these positive result experiments are right, and null or negative are somehow flawed. Not only that, but you have to avoid contradictions with older studies.
If your proposed effect is real then you should have no problem in proving it. If it doesn't exist, or if it does, but is much smaller and doesn't apply nearly as broadly as you have stated, then of course you will have "not much fun" proving it. Especially if that fun means making tens or even hundreds of experiments, then exhaustively processing data to find some experiments with positive result and discarding the rest, then again looking for explanations why these positive result experiments are right, and null or negative are somehow flawed. Not only that, but you have to avoid contradictions with older studies.
Also, it may be technically "easier" to tear-down a hypothesis, but the rewards (prestige and attention) often come from going the other way.
At least, up until whatever-it-is becomes established enough that tearing it down is novel and surprising.
At least, up until whatever-it-is becomes established enough that tearing it down is novel and surprising.
It should be established that tearing things down that looked solid and accepted before is novel and surprising. Because it is. Especially when so many people have worked on top of that.
Once everything that was tearable is teared down, the remaining picture will be much more reliable representation of what is real and what is not.
Once everything that was tearable is teared down, the remaining picture will be much more reliable representation of what is real and what is not.
I like Baumeister and Tice. They make interesting theories, especially the talk "Is there anything good about men?" But this is good too -- showing that science can be messy. Makes me think of Thom Hartmann's "farmer vs hunter" analogy for ADHD, and Bem's studies on precognition.
Science is a human endeavor and as such is subject to lots of economic and mass psychological effects, but at least there is some constraint on what can be claimed. The truth is out there, in physics and other hard sciences. In everything else, the models are much more handwavy approximations of underlying complexity and variation.
Science is a human endeavor and as such is subject to lots of economic and mass psychological effects, but at least there is some constraint on what can be claimed. The truth is out there, in physics and other hard sciences. In everything else, the models are much more handwavy approximations of underlying complexity and variation.
> "When he and his adviser, Michael McCullough, reanalyzed the 2010 paper’s data using state-of-the-art analytic methods, they found no effect."
Hmm...what was different this time?
Hmm...what was different this time?
Funny thing is that the debunking of ego-depletion theory has some interesting intersection with another article posted on the front page of HN: https://news.ycombinator.com/item?id=11231371
Self control as a static variable is not equally useful across all situations... I find it improbable that humans would have only a single 'algorithm' through which to mediate executive function. Surely the dominant strategy would be greatly affected by the environment?
Psychology is an interesting field, because one really has to wonder just how static human beings actually are over time...It seems almost probable to assume people living today would handle the same experimental setup quite differently than those who lived 50 years ago.
Psychology is an interesting field, because one really has to wonder just how static human beings actually are over time...It seems almost probable to assume people living today would handle the same experimental setup quite differently than those who lived 50 years ago.
> Eating a radish when you’re surrounded by fresh-baked cookies represents an epic feat of self-denial
This is just so much BS. I could bathe in radish. All the chocolate cookies in the world can go hang themselves if there's radish around.
Maybe my ego feels so depleted because I just haven't had any radish yet this year. But I did have mountains of cookies.
This is just so much BS. I could bathe in radish. All the chocolate cookies in the world can go hang themselves if there's radish around.
Maybe my ego feels so depleted because I just haven't had any radish yet this year. But I did have mountains of cookies.
calling psychology a soft science is like calling water a soft solid. Science is based on isolating causality in a reliable, repeatable manner. Human behavior is extremely, deeply context dependant, and in a complex multi layered chaotic way.
Well, science is about doing experiments. It's really difficult to isolate causality with psychology, but possible to at least identify patterns. And then it takes more experiments to peel off those patterns.
This is a really good case of science at work. It's an experiment that specifically targeted ego depletion and couldn't find it.
I find it a bit worrying that so many people immediately agree to the conclusion proposed by these experiments. Ego depletion is one thing, but there are other well-known effects like Pavlov's Dog.
This is a really good case of science at work. It's an experiment that specifically targeted ego depletion and couldn't find it.
I find it a bit worrying that so many people immediately agree to the conclusion proposed by these experiments. Ego depletion is one thing, but there are other well-known effects like Pavlov's Dog.
they say credibility is hard to gain and easy to lose. Physics has as a standard to do predictions reproducible exactly to a something like 8 standard deviations. Psychology, as a domain, appears to be struggling to make predictions that are reproducible "up to 40% of the time". Now, even if we say they are twice that as an approximate totally unscientific show of good faith, we're still really far at 20% reproducible from reliable, credible knowledge as is promised by other fields, and would still make the field pretty murky at best
Absolutely agree. And what is scarier is when reductionist psychology gets armed with a pharmaceutical cudgel in the form of psychiatry. Works for some, sometimes, but one always has the feeling they don't really know why...
Good. I am going to need unlimited will power to wade through the haystack of modern science and find the needles of truth.
It's a little darkly funny how people react to perceived indignities in the sciences with the exact same behavior pattern they're bemoaning.
The narrative of these people is that there are bad people out there doing unreproducible work. They know they're bad, but they're just motivated by grants, low p-values, etc, and eventually they just snap and do bad science all over the place. It's an incentive problem.
So, to prove this, they... go get grants, do a meta-analysis with some nice statistics, and get subjected to the same incentive system. Except this time, they don't even have to do anything original. They just have to badly copy something else.
There's always something like this in the article:
>In his lab, Baumeister told me, the letter e task would have been handled differently. First, he’d train his subjects to pick out all the words containing e, until that became an ingrained habit. Only then would he add the second rule, about ignoring words with e’s and nearby vowels. That version of the task requires much more self-control, he says.
I really hate getting science news from sources like Slate, because you have to decompile the journalese into what the scientist actually said, but the technical terms here are that Baumeister had participants perform a fixed-target character recognition task. This is an incredibly old cognitive task that was used in some of the first cognitive psychology experiments like [http://psych.indiana.edu/tradition/Shiffrin_and_Schneider_19...] that showed that over time with practice, a controlled or manual cognitive process, like "determine if this word contains the letter 'e'" will become an automatic process.
My cognitive psychology professor was one of Shiffrin's students, so he was actually in that experiment, and was very annoyed by it, because in the fixed-target condition, you pretty quickly start noticing your target letter all the time. It'll just "pop" out at you when you're trying to read.
Obviously the exercise here won't cause that sort of behavior, but once you learn a rule like that and start having to react in miliseconds to the presence or absence of an e, even a little bit of training will create a much stronger effect than no training at all.
This is a huge study, and it would have been drastically more expensive if this facet had been included. But it also might have reproduced Baumeister's work much more fully and the answer would have been useful. As it is, this is literally more bad science, for exactly the same reason as the original science may or may not have been bad.
The narrative of these people is that there are bad people out there doing unreproducible work. They know they're bad, but they're just motivated by grants, low p-values, etc, and eventually they just snap and do bad science all over the place. It's an incentive problem.
So, to prove this, they... go get grants, do a meta-analysis with some nice statistics, and get subjected to the same incentive system. Except this time, they don't even have to do anything original. They just have to badly copy something else.
There's always something like this in the article:
>In his lab, Baumeister told me, the letter e task would have been handled differently. First, he’d train his subjects to pick out all the words containing e, until that became an ingrained habit. Only then would he add the second rule, about ignoring words with e’s and nearby vowels. That version of the task requires much more self-control, he says.
I really hate getting science news from sources like Slate, because you have to decompile the journalese into what the scientist actually said, but the technical terms here are that Baumeister had participants perform a fixed-target character recognition task. This is an incredibly old cognitive task that was used in some of the first cognitive psychology experiments like [http://psych.indiana.edu/tradition/Shiffrin_and_Schneider_19...] that showed that over time with practice, a controlled or manual cognitive process, like "determine if this word contains the letter 'e'" will become an automatic process.
My cognitive psychology professor was one of Shiffrin's students, so he was actually in that experiment, and was very annoyed by it, because in the fixed-target condition, you pretty quickly start noticing your target letter all the time. It'll just "pop" out at you when you're trying to read.
Obviously the exercise here won't cause that sort of behavior, but once you learn a rule like that and start having to react in miliseconds to the presence or absence of an e, even a little bit of training will create a much stronger effect than no training at all.
This is a huge study, and it would have been drastically more expensive if this facet had been included. But it also might have reproduced Baumeister's work much more fully and the answer would have been useful. As it is, this is literally more bad science, for exactly the same reason as the original science may or may not have been bad.
In an ideal world, there would be a filter for all news, that contains "may have been". The filter would either:
* throw the article silently away
or
* substitute all occurrences with "may or may not have been", which is identical in meaning. I am sure more readers would notice, that they are being fed information tripe.
* throw the article silently away
or
* substitute all occurrences with "may or may not have been", which is identical in meaning. I am sure more readers would notice, that they are being fed information tripe.
It's examples like this - particular to psychology - which always give me a chance to reflect and chuckle about how I believe the 'science' of psychology to be academically pointless. Not to be confused with psychiatry, which I find to be a worthwhile medical, scientific pursuit (adding subtances/drugs does change the equation quite a bit). But real, authentic psychology is pretty much an observational writing exercise that, in my opinion, can't be broadly applied any more than general dietary advice.
Each individual's physiological composition makes psychology nothing more than a practice bed for bored creative writers. Understanding the psyche is critically important to write convincing characters, absolutely, and that takes a lot of practice and thought to fashion into a functional perspective (or literary craft tool) that enables a person to write "believable" characters. Observation, not causation, as it were.
It's not really a stretch to share this belief, if one were to be familiar with the origins of psychology that trace back to an esteemed writer and philosopher(and turn-of-the-century physician...) by the name of William James, brother of noteworty Henry James.
Each individual's physiological composition makes psychology nothing more than a practice bed for bored creative writers. Understanding the psyche is critically important to write convincing characters, absolutely, and that takes a lot of practice and thought to fashion into a functional perspective (or literary craft tool) that enables a person to write "believable" characters. Observation, not causation, as it were.
It's not really a stretch to share this belief, if one were to be familiar with the origins of psychology that trace back to an esteemed writer and philosopher(and turn-of-the-century physician...) by the name of William James, brother of noteworty Henry James.
"Authentic" psychology is definitely not just an "observational writing exercise." There's this weird…meme…that psychology is just sitting around nattering about feelings.
Some psychology experiments are undoubtedly crap, but there are also a lot of carefully-designed and well-analyzed experiments, many of which actually are casual.
Consider the vast amount of work that has gone into characterizing the visual or auditory system. In a typical experiment, subjects are shown a set of stimuli and asked to make judgements about them (e.g., "Is A brighter than B?" or "Have you seen C before?"). These results are then used to investigate the factors that affect these judgements. These experiments are definitely casual (the experimenter controls the stimulus). Empirically, these results do generalize across subjects and subpopulations and, in many cases, can be tied back to physical properties of the brain. For example, we knew that there were three color sensitive photoreceptors from behavioral data, long before the S, M, and L cones were actually identified.
tl;dr: It hasn't been 1920 for a long time.
Some psychology experiments are undoubtedly crap, but there are also a lot of carefully-designed and well-analyzed experiments, many of which actually are casual.
Consider the vast amount of work that has gone into characterizing the visual or auditory system. In a typical experiment, subjects are shown a set of stimuli and asked to make judgements about them (e.g., "Is A brighter than B?" or "Have you seen C before?"). These results are then used to investigate the factors that affect these judgements. These experiments are definitely casual (the experimenter controls the stimulus). Empirically, these results do generalize across subjects and subpopulations and, in many cases, can be tied back to physical properties of the brain. For example, we knew that there were three color sensitive photoreceptors from behavioral data, long before the S, M, and L cones were actually identified.
tl;dr: It hasn't been 1920 for a long time.
> There's this weird…meme…that psychology is just sitting around nattering about feelings.
Look at the overwhelmingly rubbish depiction of psychology in movies.
It's all people lying on couches; or tenacious psychologists breaking through with unwilling clients; or catharsis.
And it's odd, because when programmers see programming in movies they see how bad that depiction is, so I'm not sure why the meme of "psychology is just sitting around nattering about feelings" is so strong.
Look at the overwhelmingly rubbish depiction of psychology in movies.
It's all people lying on couches; or tenacious psychologists breaking through with unwilling clients; or catharsis.
And it's odd, because when programmers see programming in movies they see how bad that depiction is, so I'm not sure why the meme of "psychology is just sitting around nattering about feelings" is so strong.
>And it's odd, because when programmers see programming in movies they see how bad that depiction is, so I'm not sure why the meme of "psychology is just sitting around nattering about feelings" is so strong.
The Gell-Mann Amnesia effect probably best explains this. I think you might find it interesting:
> Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them.
> In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.”
> — Michael Crichton
Also I promise it is a coincidence I've responded to you twice in such rapid succession!
The Gell-Mann Amnesia effect probably best explains this. I think you might find it interesting:
> Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them.
> In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.”
> — Michael Crichton
Also I promise it is a coincidence I've responded to you twice in such rapid succession!
There's a big gap between research psychologists and therapeutic psychologists. Their daily activities are very different.
But even within therapeutic psychology, you're right. Freudian psychoanalysis (talk therapy) was huge in literature and movies, even though Freud himself has been out of influence inside the field for decades. CBT, exposure therapy, etc are much more common treatments.
But even within therapeutic psychology, you're right. Freudian psychoanalysis (talk therapy) was huge in literature and movies, even though Freud himself has been out of influence inside the field for decades. CBT, exposure therapy, etc are much more common treatments.
> Not to be confused with psychiatry, which I find to be a worthwhile medical, scientific pursuit (adding subtances/drugs does change the equation quite a bit)
You'd be disturbed to find out just how weak some of that evidence is, especially since it's used to deprive people of their liberty and force them to take treatment against their will. There's a slew of medicine that has pretty weak evidence base. (eg knee arthroscopy)
You'd be disturbed to find out just how weak some of that evidence is, especially since it's used to deprive people of their liberty and force them to take treatment against their will. There's a slew of medicine that has pretty weak evidence base. (eg knee arthroscopy)
It's smug comments like this - particular to engineers - which always give me a chance to chuckle about how little computer programmers actually understand scientific practice.
All the same problems happen in biological sciences, and are even worse in some cases, since there's more money at stake. Failures to replicate are rampant in medical/pharmaceutical studies.
If you want to claim biology's not scientific either, go ahead. But science is about seeking knowledge, not having total agreement all the time.
All the same problems happen in biological sciences, and are even worse in some cases, since there's more money at stake. Failures to replicate are rampant in medical/pharmaceutical studies.
If you want to claim biology's not scientific either, go ahead. But science is about seeking knowledge, not having total agreement all the time.
I think it is a problem endemic to all science that requires human experimentation. Human experimentation is extremely expensive in both time and money, and so often the power of experiments is quite low relative to the effect that one wishes to measure.
Then, all of the original problems (expensive, ethical review board &ct.) still exist for those who might want to reproduce the effects, along with the fact that there is not only no incentive for researchers to reproduce an experiment, there are active disincentives.
Then, all of the original problems (expensive, ethical review board &ct.) still exist for those who might want to reproduce the effects, along with the fact that there is not only no incentive for researchers to reproduce an experiment, there are active disincentives.
If the parent mindlessly bashed psychology that doesn't mean you too need to go generalize and label all engineers (programmers).
Fair point. I consider myself both. I'm currently a programmer, but at one point, I was working on a neuroscience PhD, so I've been on both sides of this. The messiness of the data was part of the reason I personally left academia, but I saw people doing good work and trying to sift truth from mud, so these sorts of comments irk me.
Well I'm not wrong that the origin of psychology is simply a blending of philosophy and writing, which are fundamentally outside the scope of the scientific method. I went out of my way to argue that biology-based science is truly science (e.g. psychiatry) but that didn't seem to register. Oh well, audiences miss the point all the time.
Research psychologists are at the forefront of science, the scientific method, and statistical methods.
Haven't you ever wondered why all of these articles by psychologists have been questing the very fundamentals of the way science is practiced? Psychology, as a field, is our day's premiere example of a science that examines itself. The issues with reproducibility are being illuminated and confronted by - who else? - psychologists.
You're wrong on virtually every count. The scientific method is the primary philosophy practiced by psychologists. I would encourage you to read more before making easily falsified statements like "psychology is ... fundamentally outside the scope of the scientific method."
Haven't you ever wondered why all of these articles by psychologists have been questing the very fundamentals of the way science is practiced? Psychology, as a field, is our day's premiere example of a science that examines itself. The issues with reproducibility are being illuminated and confronted by - who else? - psychologists.
You're wrong on virtually every count. The scientific method is the primary philosophy practiced by psychologists. I would encourage you to read more before making easily falsified statements like "psychology is ... fundamentally outside the scope of the scientific method."
To add some specifics to this pile-on:
- The classical example of a randomized experiment is a perception/psychophysics experiment: the Lady Tasting Tea. This introduced the idea of a null hypothesis as well as Fisher's exact test. https://en.wikipedia.org/wiki/Lady_tasting_tea
- Visual and auditory experiments more or less invented signal detection theory and used it to probe visual and auditory processing. Those ROC curves everyone loves predate "machine learning" by decades.
- A lot of the fancy hierarchical and Bayesian modeling work have come from psychologists and social scientists (e.g., Andrew Gelman)
- The classical example of a randomized experiment is a perception/psychophysics experiment: the Lady Tasting Tea. This introduced the idea of a null hypothesis as well as Fisher's exact test. https://en.wikipedia.org/wiki/Lady_tasting_tea
- Visual and auditory experiments more or less invented signal detection theory and used it to probe visual and auditory processing. Those ROC curves everyone loves predate "machine learning" by decades.
- A lot of the fancy hierarchical and Bayesian modeling work have come from psychologists and social scientists (e.g., Andrew Gelman)
Nevermind that perhaps the most prescribed class of psychiatric drugs has been called into question:
http://blogs.scientificamerican.com/cross-check/are-antidepr...
Psychiatry is a really, really hard field for canonical drug research. Mechanisms of disease are for the most part completely unknown, the mechanism of action for the drugs is often unclear in the patient (why stimulants treat ADHD? it's complicated...) and you've got the additional complication that diagnosis for the disease is purely symptomatic (in fact, that diagnosis process is nearly identical to that of psychology) so two people with the same symptoms may have radically different diseases, but hey, they'll both be in your clinical trial!
Drawing a difference between Psychology and Psychiatry doesn't really hold up. Doing research on people is extremely difficult to get right, all medical and psychology research is subject to these problems.
http://blogs.scientificamerican.com/cross-check/are-antidepr...
Psychiatry is a really, really hard field for canonical drug research. Mechanisms of disease are for the most part completely unknown, the mechanism of action for the drugs is often unclear in the patient (why stimulants treat ADHD? it's complicated...) and you've got the additional complication that diagnosis for the disease is purely symptomatic (in fact, that diagnosis process is nearly identical to that of psychology) so two people with the same symptoms may have radically different diseases, but hey, they'll both be in your clinical trial!
Drawing a difference between Psychology and Psychiatry doesn't really hold up. Doing research on people is extremely difficult to get right, all medical and psychology research is subject to these problems.
I guess Einstein wasn't doing science because he used thought experiments and philosophical arguments instead of _real_ experiments.
Nothing wrong with thought experiments and philosophical arguments, as long as you use them properly.
Proper use goes like this:
1) Identify some area where theory does not match experiment or other theories (in Einstein's case, there were known experimental issues involving the speed of light that were the basis for his work on special relativity, for example).
2) Use thought experiments, philosophical arguments, divine inspiration, inspired guesses, random guesses, etc to produce a theory that accounts for your known experimental data. I expect all of these have been used in the past, in fact. Einstein used thought experiments and philosophical arguments for special relativity but an inspired guess for the photoelectric effect.
3) Use your new theory to make new predictions about experiments that have not yet been performed. If there are no such predictions to be made from the theory that step 2 produced, then the utility of that theory is pretty limited, obviously.
4) Perform the experiments whose results you predicted in step 3 and see whether you predicted them correctly. It's important to publish the results of these experiments, especially if the theory did NOT predict the results you got.
5) Keep repeating steps 3 and 4 as much as possible, ideally with a variety of different predictions produced in step 3, to gain more confidence in the new theory.
These are, of course, all "doing science". And insofar as they can be done by different people, one could spend all one's time on step 2 and be "doing science". But the scientific community as a whole is only "doing science" if it actually goes through all five steps. If it stops after step 2, that's not called "doing science".
Proper use goes like this:
1) Identify some area where theory does not match experiment or other theories (in Einstein's case, there were known experimental issues involving the speed of light that were the basis for his work on special relativity, for example).
2) Use thought experiments, philosophical arguments, divine inspiration, inspired guesses, random guesses, etc to produce a theory that accounts for your known experimental data. I expect all of these have been used in the past, in fact. Einstein used thought experiments and philosophical arguments for special relativity but an inspired guess for the photoelectric effect.
3) Use your new theory to make new predictions about experiments that have not yet been performed. If there are no such predictions to be made from the theory that step 2 produced, then the utility of that theory is pretty limited, obviously.
4) Perform the experiments whose results you predicted in step 3 and see whether you predicted them correctly. It's important to publish the results of these experiments, especially if the theory did NOT predict the results you got.
5) Keep repeating steps 3 and 4 as much as possible, ideally with a variety of different predictions produced in step 3, to gain more confidence in the new theory.
These are, of course, all "doing science". And insofar as they can be done by different people, one could spend all one's time on step 2 and be "doing science". But the scientific community as a whole is only "doing science" if it actually goes through all five steps. If it stops after step 2, that's not called "doing science".
Logical positivism died about 100 years ago, you're late to the party.
People need to stop doing meta-analysis * . Have you ever heard of meta-analysis of mathematics or physics papers? No? P = NP meta analysis maybe? Haha.
I can tell you why: Because if it's solid science, you don't need 100 papers to describe an effect. Besides, it is a proven fact that the results of meta-analysis are weaker due to how the underlying statistics works (at least if you do it in a naive way, and I do not know if it can be fixed by altering the procedure. I have to brush up on my statistics knowledge, it's good for machine learning.).
* EDIT: I should have added here 'to investigate the ground-truth of underlying papers.' Of course there's nothing wrong with meta-analysis to investigate researcher behaviour and methodology.
I can tell you why: Because if it's solid science, you don't need 100 papers to describe an effect. Besides, it is a proven fact that the results of meta-analysis are weaker due to how the underlying statistics works (at least if you do it in a naive way, and I do not know if it can be fixed by altering the procedure. I have to brush up on my statistics knowledge, it's good for machine learning.).
* EDIT: I should have added here 'to investigate the ground-truth of underlying papers.' Of course there's nothing wrong with meta-analysis to investigate researcher behaviour and methodology.
Actually: https://www.win.tue.nl/~gwoegi/P-versus-NP.htm
And amusingly, the distribution of result is close to 50/50
And amusingly, the distribution of result is close to 50/50
That's actually pretty cool. The difference is that here people are completely aware that this study does not in any way tell us whether P = NP (even if it was 100/0) as long as there's no solid proof on the table.
A meta-analysis in those fields is usually only used to analyze the behaviour and methodology of researchers, not to investigate the ground truth of the papers.
A meta-analysis in those fields is usually only used to analyze the behaviour and methodology of researchers, not to investigate the ground truth of the papers.
"The behaviour and methodology of researchers" is exactly that thing that will lead to the truths you seek. You're basically saying a bridge-builder needn't waste any time worrying about the engineering of boats.
While I admit it was wrong to claim that all meta-analysis is bad, the main point of my argument has not been refuted. You cannot inspect 100 papers and make a truth claim that is stronger than the quality of the papers, and doing so is a folly. Nobody in maths uses meta-analysis to prove a theorem.
I am a big believer that leaders need to understand social psychology. Unfortunately every one of these debunking studies makes it very hard for the non-expert to know what to apply in the field. It's very frustrating! (I suppose one could say the same about economics)
WRT economics, an understanding of the major concepts is probably more than enough: elasticity of demand, substitutes, complements, barriers to entry, network effect, substitution effect, income effect, (diminishing) returns to scale, altruism vs. greed, monetary vs. fiscal policy, real vs. nominal, normative vs. positive, inflation, time value of money, aligning incentives, setting precedents, and basic game theory (how to draw and play a simple game w/ payoff tables).
I doubt the actual models get used outside of academia and central banks. The frameworks I have studied have been so situation-specific and strong-assumption-driven that it would take much more time and effort to cram reality into a model than to simply use high level concepts, some basic data, a few customer interviews, and intuition to inform strategy.
I doubt the actual models get used outside of academia and central banks. The frameworks I have studied have been so situation-specific and strong-assumption-driven that it would take much more time and effort to cram reality into a model than to simply use high level concepts, some basic data, a few customer interviews, and intuition to inform strategy.
I would argue the leadership of both successful and failed economies have an understanding of the topics you outline. Therefore, the understanding of these topics is only enough to make efficient decisions if your decisions are efficient.
It'd be nice if this from Bertrand Russell was remembered more: "(1) that when the experts are agreed, the opposite opinion cannot be held to be certain; (2) that when they are not agreed, no opinion can be regarded as certain by a non-expert; and (3) that when they all hold that no sufficient grounds for a positive opinion exist, the ordinary man would do well to suspend his judgment."
Of course, most people don't actually even see what the experts say, they just take on the belief of what one person reporting on one expert says... so you get misconceptions like http://danluu.com/dunning-kruger/
I'd prefer if leaders focused on being good leaders and not trying to play expert or even knowledgeable-enough on technical matters that really don't concern them. A good leader will find the domain experts and drill them for critical information needed for their decision making, they don't need to understand the domain details. (And they can verify their trust with other experts.)
Of course, most people don't actually even see what the experts say, they just take on the belief of what one person reporting on one expert says... so you get misconceptions like http://danluu.com/dunning-kruger/
I'd prefer if leaders focused on being good leaders and not trying to play expert or even knowledgeable-enough on technical matters that really don't concern them. A good leader will find the domain experts and drill them for critical information needed for their decision making, they don't need to understand the domain details. (And they can verify their trust with other experts.)
If by social psychology you mean "the way people think and behave socially", i agree. If you mean "the results and theories of the discipline called 'social psychology'", i disagree. It must be the case that each of us has some understanding of our social behavior and the behavior of others. Otherwise we would never form social structures.
i don't know if i've ever gained real insight into human behavior from reading the results of a contrived experiment.
i don't know if i've ever gained real insight into human behavior from reading the results of a contrived experiment.
Yeah. I've actually had a lot better results from Proverbs for example with regards to social behavior: giving gifts to people who are angry with you pacifies. There are other studies I've heard of that are really popular too, like the experiment where people are told that they must continue torturing someone, that when I've looked into it weren't nearly so iron-clad as claimed.
i don't know if i've ever gained real insight into human behavior from reading the results of a contrived experiment.
Therein lies my problem. :-) I want to know more, but most sources are woefully unreliable.
Therein lies my problem. :-) I want to know more, but most sources are woefully unreliable.
I fail to see the significance of this, other than a shift in what sort of self-help books are sold.
Are there any widespread pedagogic or management practices that are built on top of ego depletion?
Are there any widespread pedagogic or management practices that are built on top of ego depletion?
I'd say it has had a profound effect. The thought that you can wear down someone's willpower by making them make too many decisions is pretty scary if your job is to make decisions. One work-around that, for example, the POTUS has espoused is wearing the same clothes every day. (One less decision to make.)
See: http://www.fastcompany.com/3026265/work-smart/always-wear-th...
See: http://www.fastcompany.com/3026265/work-smart/always-wear-th...
Maybe my definitions are off, but clicking or not clicking based on the arrangement of letters in a word seems like an cognitive task, not one of willpower; it seems pretty easy to do without much stress. So this new result seems like a very oblique attack, not a strike at the foundations of ego depletion theory.
For me this is quite interesting. Because some one has said that it has debunked a study doesn´t means that he has done it. There is still the possibility that the new study is the incorrect one.
That a team claims that there is a problem it is a good sign that this needs more studies. But it will be jumping to conclusions telling people that the original study has been debunked.
That a team claims that there is a problem it is a good sign that this needs more studies. But it will be jumping to conclusions telling people that the original study has been debunked.
There can always be more studies, and there can always be the possibility of error.
However, consider if the two studies were reversed in time, so the first study specified what its endpoints were, then did the study, and concluded there was no impact.
Then the second study came along, and during data analysis an interesting correlation appeared which showed there was a impact.
Assuming that both were done by equally competent researchers, which conclusions would you trust more, and why?
Do your conclusions change if you swap the time order?
Personally, I would trust the study with the well-defined endpoints over the other one, but I also feel within myself the desire to trust the first one - even though the order shouldn't change things.
However, consider if the two studies were reversed in time, so the first study specified what its endpoints were, then did the study, and concluded there was no impact.
Then the second study came along, and during data analysis an interesting correlation appeared which showed there was a impact.
Assuming that both were done by equally competent researchers, which conclusions would you trust more, and why?
Do your conclusions change if you swap the time order?
Personally, I would trust the study with the well-defined endpoints over the other one, but I also feel within myself the desire to trust the first one - even though the order shouldn't change things.
If the studies were reversed I think my opinion would be the same. We don't know that much about ego depletion. If it exists or what's driving it.
I think it means ego depletion is much more fragile than we thought or is caused by some other phenomenon. I don't think this study means the thing we called ego depletion doesn't exist and is an artifact of publication bias and chance. It seems to me like there were just too many studies replicating it for that to be true.
I think it means ego depletion is much more fragile than we thought or is caused by some other phenomenon. I don't think this study means the thing we called ego depletion doesn't exist and is an artifact of publication bias and chance. It seems to me like there were just too many studies replicating it for that to be true.
may have been debunked
Baumeister has yet to be steamrolled by the reproducibility crusaders
Baumeister has yet to be steamrolled by the reproducibility crusaders
The framing of this article very much misunderstands the scientific method.
Science is a process. It's a method where you conduct a series of disciplined, reproducible experiments and use this data to build a model of the truth. Science never gives you the truth directly; instead, you get a series of successively more accurate representations of the truth that you can use to make predictions. Those predictions are not guaranteed to always yield a positive result, but when they come up negative, you have more data with which to refine your theories.
Casting this in black and white terms - "this effect has been debunked" - is doing a disservice to science.
Rather, a negative result in this study should be grounds for further investigation. Why did it come up with different results? Is there something different about how they ran the study? Is there something different about the subjects, something that may hold useful lessons in its own right? Was the original data flawed? Does the effect hold only under certain conditions? All of these would be very useful things to know. The point of science is not to say "This is true and this is not", it's to be able to create increasingly refined models of "This is true under these conditions, and if you apply this model in this situation, you will get these results."
By analogy with physics: the ancient Egyptians (the Ptolemaic kings) believed that the sun revolved around the earth. To make this model work, they had to invent increasingly more complex epicycles in the motions of planets. Copernicus came around and showed that the earth revolved around the sun in a circle. Kepler re-examined this data and said no, planets actually move in ellipsis. Einstein re-examined this data and came up with a tensor calculus that even more accurately predicted the orbits of the outer planets. Now, there is strong evidence that Einstein's theory of general relativity is "wrong" too, but we don't have anything to replace it with.
Was the Ptolemaic calculation wrong? Well, technically not - as my physics professor pointed out, they were just doing a Fourier decomposition of the observed data. And for a kid making a sun-dial on a beach or a hiker who wants to know when the sun will set, the first term of that series is perfectly adequate. But the advantages of the heliocentric and relativistic models are that they can predict the data with increasing accuracy, without adding extraneous magic numbers.
Similarly, when a psychological study can't be replicated, it doesn't necessarily mean the basic idea is useless. Rather, it means that we should look further into the situations where that idea is useless, and refine our models so they gain additional predictive power.
Science is a process. It's a method where you conduct a series of disciplined, reproducible experiments and use this data to build a model of the truth. Science never gives you the truth directly; instead, you get a series of successively more accurate representations of the truth that you can use to make predictions. Those predictions are not guaranteed to always yield a positive result, but when they come up negative, you have more data with which to refine your theories.
Casting this in black and white terms - "this effect has been debunked" - is doing a disservice to science.
Rather, a negative result in this study should be grounds for further investigation. Why did it come up with different results? Is there something different about how they ran the study? Is there something different about the subjects, something that may hold useful lessons in its own right? Was the original data flawed? Does the effect hold only under certain conditions? All of these would be very useful things to know. The point of science is not to say "This is true and this is not", it's to be able to create increasingly refined models of "This is true under these conditions, and if you apply this model in this situation, you will get these results."
By analogy with physics: the ancient Egyptians (the Ptolemaic kings) believed that the sun revolved around the earth. To make this model work, they had to invent increasingly more complex epicycles in the motions of planets. Copernicus came around and showed that the earth revolved around the sun in a circle. Kepler re-examined this data and said no, planets actually move in ellipsis. Einstein re-examined this data and came up with a tensor calculus that even more accurately predicted the orbits of the outer planets. Now, there is strong evidence that Einstein's theory of general relativity is "wrong" too, but we don't have anything to replace it with.
Was the Ptolemaic calculation wrong? Well, technically not - as my physics professor pointed out, they were just doing a Fourier decomposition of the observed data. And for a kid making a sun-dial on a beach or a hiker who wants to know when the sun will set, the first term of that series is perfectly adequate. But the advantages of the heliocentric and relativistic models are that they can predict the data with increasing accuracy, without adding extraneous magic numbers.
Similarly, when a psychological study can't be replicated, it doesn't necessarily mean the basic idea is useless. Rather, it means that we should look further into the situations where that idea is useless, and refine our models so they gain additional predictive power.
So then, one could propose any ridiculous hypothesis (e.g. vaccines do not work), and when nobody can reproduce the results, applying your line of argument, it would lead us down a rabbit hole of second guessing and proposing even more complex reasons as to why others are unable to reproduce the results.
I think that in the - day-to-day usage of the word / most commonly understood meaning of the word - 'true', it is fair to say that Science is in the business of determining truth from fantasy. Sure, you are permitted under science to investigate anything and everything that pleases you/(your donors :) ). I think that scientific consensus is very closely associated with 'truth'.
I totally get the larger point you're making, but even the nature of science itself is a bit of a grey area. I think that some ideas should, and have been laughed out of the room. The so called man-on-the-street does not have time for subtlety or scientific hypotheses which are carefully couched in nuanced terminology. I think that its OK for people to think that science has all the answers, every once in a while :)
I think that in the - day-to-day usage of the word / most commonly understood meaning of the word - 'true', it is fair to say that Science is in the business of determining truth from fantasy. Sure, you are permitted under science to investigate anything and everything that pleases you/(your donors :) ). I think that scientific consensus is very closely associated with 'truth'.
I totally get the larger point you're making, but even the nature of science itself is a bit of a grey area. I think that some ideas should, and have been laughed out of the room. The so called man-on-the-street does not have time for subtlety or scientific hypotheses which are carefully couched in nuanced terminology. I think that its OK for people to think that science has all the answers, every once in a while :)
The "vaccines cause autism" scare is actually a good example of science working. It was investigated - as it should have been, because if vaccines did cause autism that would be a massive public health crisis - and then they found that the original author falsified the data. The unfortunate part is that that finding hasn't made it into public consciousness the way the original scare did.
I think that the belief that science = received truth is behind the rejection of science by a large portion of the population, because it puts it in conflict with other sources of "received truth". People discount evolution as "just a theory" because they're looking for absolute truth; if they were habitually trained to look for evidence and then knew the body of evidence behind evolution, and could compare it with the body of evidence behind gospel, they might understand that "theory" is a lot more rigorously proven than "truth".
I think that the belief that science = received truth is behind the rejection of science by a large portion of the population, because it puts it in conflict with other sources of "received truth". People discount evolution as "just a theory" because they're looking for absolute truth; if they were habitually trained to look for evidence and then knew the body of evidence behind evolution, and could compare it with the body of evidence behind gospel, they might understand that "theory" is a lot more rigorously proven than "truth".
Along the same lines, I think this quote from the economist Ludwig von Mises is quite on point:
"Science does not give us absolute and final certainty.
It only gives us assurance within the limits of our mental
abilities and the prevailing state of scientific thought."Reexamining previous studies found them flawed, and their 'positive results' disappeared. That's not the same a 'a negative result'. This doesn't claim one experiment and Boom! the theory is disproved. Its disingenuous to say "a negative result in this study should be ground for further investigation" The issue is, why had so many studied done it wrong?
The idea is clearly not useless, if it is being used to gain additional predictive power, no?
> "For a second paper published last year, Carter and McCullough completed a second meta-analysis that included different studies, including 48 experiments that had never been published. Again, they found “very little evidence” of a real effect."
> "“All of a sudden it felt like everything was crumbling,” says Carter, now 31 years old and not yet in a tenure-track position. “I basically lost my compass. Normally I could say, all right there have been 100 published studies on this, so I can feel good about it, I can feel confident. And then that just went away.”"
So you have at least 100 bullshit studies on a really popular and often citied experiment. If bullshit can hide in plain sight like this, imagine what is lurking out of sight?