Effect of One-Legged Standing on Sleep(quantifiedself.com)
quantifiedself.com
Effect of One-Legged Standing on Sleep
http://quantifiedself.com/2011/03/effect-of-one-legged-standing-on-sleep/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+QuantifiedSelf+%28The+Quantified+Self%29
7 comments
Methodology varies with the terrain. Studies of human genetics are limited to observation, as experimentation is unethical. Studies of history -- archeology, evolution -- don't have a lot going on in the repeatability department. Studies on topics of opinion -- advertising effectiveness, political surveys -- necessarily contend with a host of sampling biases and confounding variables.
Not everything is physics. And I don't think it's intellectually optimal to flag everything that can't be physics as simple nonsense.
I'd rather acknowledge and talk about methodological weaknesses when we discuss how persuasive the experiment is. I mean, it's not like an experiment that does everything "right" is always perfectly persuasive either. Reality is complicated; sometimes you were asking the wrong question. Most things are neither ironclad nor nonsense, but fall somewhere in the middle.
So, two things about subjective measurement. First, just because something's subjective doesn't mean it's totally unreliable. You can fool yourself about small things; large things are harder. Suppose you're trying to demonstrate the presence of a material by the color of a flame. If you're trying to distinguish between pale green and pale pale green, better break out the spectrometer. But calling whether it's yellow or blue? Qualia will suffice.
Applied to this case, I think I would rely on someone to be able to answer the question, "Are you well-rested or not?" with a "yes", "no", or "sorta". On the other hand, a 99 vs a 98 percent measurement -- as done here -- is pretty darn iffy.
The second thing about subjective measurements is that, when you're after subjective results, that's the right sort of measure. For example, if someone says, "When I spend time with friends, I feel like I do better on math tests the next day," you might rightly respond, "We should be measuring the scores, not your feelings. It's possible your friends are making you feel competent when you really aren't." But if someone says, "When I spend time with friends, I feel happier," does it even make sense to say, "It's possible your friends are making you feel happier when you really aren't?" The best you can do is try to define degrees of happiness and measure statistically.
When the feeling is the object of the discussion, the subjective isn't an illusion. A pill which makes you feel competent when you aren't is producing an illusion, but a pill that makes you feel happy when you want to is a pill that works.
(This likewise reduces the point of blinding. If we're trying to figure out whether a chemical makes you happy, it makes sense to run it against a placebo. But if we're trying to figure out whether an activity makes you happy, being aware that you did it is appropriate. Whether it's the physical motion or the awareness that makes you happy is immaterial to whether the activity works.)
All of that is to say, if the guy is trying to prove he is well-rested in the sense that he performs better, he reacts more quickly, his judgement is more sound, all the various things we associate with being well-rested, he should be taking tests. But if he is trying to feel well-rested, he should be asking, "How well-rested do I feel?"
Now, it's hard to say exactly which he's after -- feeling well-rested but possibly performing like crap, or feeling like crap but possibly performing well. I'd think he probably wants both things. I certainly would. So I think he's only measuring half of what he should. The result could certainly be made more complete by doing something like the coffee guy did (http://quantifiedself.com/2009/10/the-false-god-of-coffee/).
It is promising that he measured how long he slept before first waking up -- that's pretty objective -- but I can't tell exactly what he's saying he did. (I can't tell what he means by baseline(3) and random(3), so I can't really critique the results).
It would be nice if he'd chosen the exercise randomly for the random portion of the experiment. By just picking one, there's a serious possibility he was picking based on how tired he already felt due to events of the day (it is exercise he's signing up for, after all.) I don't know that it reduces those results to nonsense, but it certainly does reduce them to possibly nonsense. I think the best you can get is, "Huh, that's interesting enough to investigate--do it again, and use a die this time."
And then we have the obvious criticism: he only tried it on himself. Whether it works only on him for psychological reasons relating to his life history, or whether it only works on some people for genetic reasons, or what . . . remains to be seen.
On the whole, I'd give it about a five out of ten for methodology. I do like that he tried the exercise over a period of months, kept notes, and looked for patterns. That's worth a lot of credit right there. I think it could be improved by stating more clearly what he's trying to prove, by incorporating objective measurements where possible and keeping subjective measurements as clear-cut as possible -- though that sort of thing is really more appropriate for the next iteration, anyway. There's investigating to find interesting patterns, and then there's setting out to demonstrate the truth of what you think you've found. An investigative experiment -- which this certainly was -- is inherently not going to be a demonstrative one.
I wouldn't mind clearer labeling of the charts, and I definitely wouldn't mind access to the raw data. That would bolster the case a lot.
And a repeat would be great -- someone else should try it and see if it works. Ideally several people.
Overall, though, it's far from nonsense, and I certainly wouldn't put it in the "trying to get people to stand on one leg to make a point about how gullible we are when we see graphs" category. I'd mark it investigative/interesting.
Not everything is physics. And I don't think it's intellectually optimal to flag everything that can't be physics as simple nonsense.
I'd rather acknowledge and talk about methodological weaknesses when we discuss how persuasive the experiment is. I mean, it's not like an experiment that does everything "right" is always perfectly persuasive either. Reality is complicated; sometimes you were asking the wrong question. Most things are neither ironclad nor nonsense, but fall somewhere in the middle.
So, two things about subjective measurement. First, just because something's subjective doesn't mean it's totally unreliable. You can fool yourself about small things; large things are harder. Suppose you're trying to demonstrate the presence of a material by the color of a flame. If you're trying to distinguish between pale green and pale pale green, better break out the spectrometer. But calling whether it's yellow or blue? Qualia will suffice.
Applied to this case, I think I would rely on someone to be able to answer the question, "Are you well-rested or not?" with a "yes", "no", or "sorta". On the other hand, a 99 vs a 98 percent measurement -- as done here -- is pretty darn iffy.
The second thing about subjective measurements is that, when you're after subjective results, that's the right sort of measure. For example, if someone says, "When I spend time with friends, I feel like I do better on math tests the next day," you might rightly respond, "We should be measuring the scores, not your feelings. It's possible your friends are making you feel competent when you really aren't." But if someone says, "When I spend time with friends, I feel happier," does it even make sense to say, "It's possible your friends are making you feel happier when you really aren't?" The best you can do is try to define degrees of happiness and measure statistically.
When the feeling is the object of the discussion, the subjective isn't an illusion. A pill which makes you feel competent when you aren't is producing an illusion, but a pill that makes you feel happy when you want to is a pill that works.
(This likewise reduces the point of blinding. If we're trying to figure out whether a chemical makes you happy, it makes sense to run it against a placebo. But if we're trying to figure out whether an activity makes you happy, being aware that you did it is appropriate. Whether it's the physical motion or the awareness that makes you happy is immaterial to whether the activity works.)
All of that is to say, if the guy is trying to prove he is well-rested in the sense that he performs better, he reacts more quickly, his judgement is more sound, all the various things we associate with being well-rested, he should be taking tests. But if he is trying to feel well-rested, he should be asking, "How well-rested do I feel?"
Now, it's hard to say exactly which he's after -- feeling well-rested but possibly performing like crap, or feeling like crap but possibly performing well. I'd think he probably wants both things. I certainly would. So I think he's only measuring half of what he should. The result could certainly be made more complete by doing something like the coffee guy did (http://quantifiedself.com/2009/10/the-false-god-of-coffee/).
It is promising that he measured how long he slept before first waking up -- that's pretty objective -- but I can't tell exactly what he's saying he did. (I can't tell what he means by baseline(3) and random(3), so I can't really critique the results).
It would be nice if he'd chosen the exercise randomly for the random portion of the experiment. By just picking one, there's a serious possibility he was picking based on how tired he already felt due to events of the day (it is exercise he's signing up for, after all.) I don't know that it reduces those results to nonsense, but it certainly does reduce them to possibly nonsense. I think the best you can get is, "Huh, that's interesting enough to investigate--do it again, and use a die this time."
And then we have the obvious criticism: he only tried it on himself. Whether it works only on him for psychological reasons relating to his life history, or whether it only works on some people for genetic reasons, or what . . . remains to be seen.
On the whole, I'd give it about a five out of ten for methodology. I do like that he tried the exercise over a period of months, kept notes, and looked for patterns. That's worth a lot of credit right there. I think it could be improved by stating more clearly what he's trying to prove, by incorporating objective measurements where possible and keeping subjective measurements as clear-cut as possible -- though that sort of thing is really more appropriate for the next iteration, anyway. There's investigating to find interesting patterns, and then there's setting out to demonstrate the truth of what you think you've found. An investigative experiment -- which this certainly was -- is inherently not going to be a demonstrative one.
I wouldn't mind clearer labeling of the charts, and I definitely wouldn't mind access to the raw data. That would bolster the case a lot.
And a repeat would be great -- someone else should try it and see if it works. Ideally several people.
Overall, though, it's far from nonsense, and I certainly wouldn't put it in the "trying to get people to stand on one leg to make a point about how gullible we are when we see graphs" category. I'd mark it investigative/interesting.
Too many people, like you, are hung up on "proof". Yes, we should be aware of the difference between hypothesis exploration and a solid demonstration. But that doesn't mean that hypothesis exploration is pointless!
An experiment like this one is entirely appropriate when trying to decide whether a hypothesis is worth exploring further; setting up a more elaborate study would have been premature.
So when you say "his experiment proves nothing" I think you are judging it by standards it was not, and should not have been, intended to meet. Don't focus only on the results of science; without the process, there would be no interesting results.
An experiment like this one is entirely appropriate when trying to decide whether a hypothesis is worth exploring further; setting up a more elaborate study would have been premature.
So when you say "his experiment proves nothing" I think you are judging it by standards it was not, and should not have been, intended to meet. Don't focus only on the results of science; without the process, there would be no interesting results.
No, hypothesis exploration is entirely valid, it's what science is based on.
My issue is that he proceeds to draw very strongly worded conclusions (using the phrase "beyond doubt") based on very weak evidence and presenting the experiment in a way which both fails to highlight the inherent flaws and indeed hides them behind a veil of pseudo-scientific and statistical language.
Yes explore hypotheses, present anecdotal evidence, but make it clear that's what you're doing and explain why. The next step here isn't to try 5 and 6 leg stands, it's to try it with people who don't understand what is being tested and remove the worst of the flaws in the methodology.
As for being hung up on "proof", yes guilty as charged, but that's really not my issue which is don't claim proof when you don't have it. Which in claiming things "beyond any doubt" (which he does) this is what he is doing.
My issue is that he proceeds to draw very strongly worded conclusions (using the phrase "beyond doubt") based on very weak evidence and presenting the experiment in a way which both fails to highlight the inherent flaws and indeed hides them behind a veil of pseudo-scientific and statistical language.
Yes explore hypotheses, present anecdotal evidence, but make it clear that's what you're doing and explain why. The next step here isn't to try 5 and 6 leg stands, it's to try it with people who don't understand what is being tested and remove the worst of the flaws in the methodology.
As for being hung up on "proof", yes guilty as charged, but that's really not my issue which is don't claim proof when you don't have it. Which in claiming things "beyond any doubt" (which he does) this is what he is doing.
The experimental procedures he went through gave little additional support for his argument than simply stating:
"I noticed I sleep better when I stand on one leg for a while".
The statistical analysis of the study does not provide any additional weight to the statement. In fact, the statistical analysis is more of a distraction because he didn't control for an important influence in self-evaluating health. If a person expects that taking an action will improve their health, it's very likely that the person will think he feels better after taking that action.
The statistical analysis of the study does not provide any additional weight to the statement. In fact, the statistical analysis is more of a distraction because he didn't control for an important influence in self-evaluating health. If a person expects that taking an action will improve their health, it's very likely that the person will think he feels better after taking that action.
While it is not rigorously scientific in the sense that physics is, it is compares very favorably to findings in the fields of medicine and nutrition. (Yes, they are much worse than you imagine).
And while this document cannot reflect this, I've been following Seth Roberts for a few years now -- he is the best kind of scientist, with a remarkable talent for objective measurement of oneself. While this cannot be generalized to other people easily, as far as it refers to himself, it's probably way better than most medical results published in the last 20 years.
And while this document cannot reflect this, I've been following Seth Roberts for a few years now -- he is the best kind of scientist, with a remarkable talent for objective measurement of oneself. While this cannot be generalized to other people easily, as far as it refers to himself, it's probably way better than most medical results published in the last 20 years.
I think we are being awfully hard on this guy just because he attempted to quantify this stuff about his life. If someone shared offhand that standing on one leg helped them sleep better in a Hacker News comment, I can't imagine anyone responding that their methodology was "close to nonsense" or that their experience "proved nothing".
Let's be realistic. The vast majority of evidence we use to update our beliefs is not of scientific caliber. (Just imagine how crippled a reasoner someone would be if they never changed their mind about anything unless they saw scientific evidence that indicated they should. Pivoting a startup would be very difficult, for instance.) There is a continuum of evidence quality, and we should reward, not punish, this guy for collecting higher-quality evidence even if he didn't get all the way to "scientific".
Let's be realistic. The vast majority of evidence we use to update our beliefs is not of scientific caliber. (Just imagine how crippled a reasoner someone would be if they never changed their mind about anything unless they saw scientific evidence that indicated they should. Pivoting a startup would be very difficult, for instance.) There is a continuum of evidence quality, and we should reward, not punish, this guy for collecting higher-quality evidence even if he didn't get all the way to "scientific".
It isn't scientifically sound but it also isn't nonsense. If you find a simple trick that solves a problem you're having it might be useful for others with the same problem to try it, too. The limitations of this should be pretty self-evident to anyone reasonably intelligent. This is on a website that is all about self experimentation, after all. I don't know why are assuming you've stumbled onto a scientific study.
Sorry but with the exception of the basic idea that if you exert yourself physically (in this case by standing on one leg until you're tired) you may sleep better it is complete nonsense but it is also very obviously intended to look as if his experiment (his word) it has scientific and statistical merit which it does not.
If you don't think he's intending this then you need to explain away why he uses the phrases "experiment" and "a randomized experiment" several times (and indeed the language throughout is clearly intended to convey scientific rigour), outlines an experimental methodology (which he contrasts with his mere anecdotal data), fills the pages with graphs showing the mean and error bars and uses language that imply statistical significant ("one-tailed p = 0.02"), uses the "results" to draw a conclusion beyond "any doubt" and then seeks to explain what has happened in scientific terms ("to allow the signaling molecules to be regenerated" - I'm struggling not to giggle at that incidentally).
Normal people don't talk like that unless they're trying to sound scientific.
The intention here is clearly to imply some level of authority in his results based on something which is either intended to be or intended to look like scientific method when it is, in scientific terms utter nonsense and proves nothing at all.
I have no issue with him suggesting his theories to people, I do have an issue with him, either as a result of ignorance or intentional deceit, dressing it up as in any way authoritative.
If you don't think he's intending this then you need to explain away why he uses the phrases "experiment" and "a randomized experiment" several times (and indeed the language throughout is clearly intended to convey scientific rigour), outlines an experimental methodology (which he contrasts with his mere anecdotal data), fills the pages with graphs showing the mean and error bars and uses language that imply statistical significant ("one-tailed p = 0.02"), uses the "results" to draw a conclusion beyond "any doubt" and then seeks to explain what has happened in scientific terms ("to allow the signaling molecules to be regenerated" - I'm struggling not to giggle at that incidentally).
Normal people don't talk like that unless they're trying to sound scientific.
The intention here is clearly to imply some level of authority in his results based on something which is either intended to be or intended to look like scientific method when it is, in scientific terms utter nonsense and proves nothing at all.
I have no issue with him suggesting his theories to people, I do have an issue with him, either as a result of ignorance or intentional deceit, dressing it up as in any way authoritative.
Seth Roberts has made his entire reputation on self-experimentation. (Partly on the strange things he claims to have learned by doing it, and also partly on his defense of it as a methodology.)
He's written some papers that talk about the methodological concerns. Here are two that Google churned up:
http://sethroberts.net/articles/2010%20The%20unreasonable%20...
http://escholarship.org/uc/item/2xc2h866
I can't honestly claim that he has successfully made n=1 experiments respectable, but I think it's safe to say he's very well aware of the criticism on these grounds and is not ignoring it.
He's written some papers that talk about the methodological concerns. Here are two that Google churned up:
http://sethroberts.net/articles/2010%20The%20unreasonable%20...
http://escholarship.org/uc/item/2xc2h866
I can't honestly claim that he has successfully made n=1 experiments respectable, but I think it's safe to say he's very well aware of the criticism on these grounds and is not ignoring it.
This is interesting and I'll have a look at them, though in this case my issue isn't with n=1, it's with the wider design of the experiment and the failure to be clear about it's limitations.
The guy is a prof. of psychology at berkeley, and knows his statistics much better than most researchers in medicine or nutrition.
I don't respect your skepticism, unless you are just as skeptical about advice you get from your Doctor or pharmacist.
Roberts looks for effective, simple treatments that you can try yourself without requiring a large budget or large lab, publishes everything without trying to make money of it -- including a diet-free diet that works remarkably well, that flies in the face of all "nutrition science" common sense. He is not the crackpot you think he is.
I don't respect your skepticism, unless you are just as skeptical about advice you get from your Doctor or pharmacist.
Roberts looks for effective, simple treatments that you can try yourself without requiring a large budget or large lab, publishes everything without trying to make money of it -- including a diet-free diet that works remarkably well, that flies in the face of all "nutrition science" common sense. He is not the crackpot you think he is.
I'm sceptical about things in so far as you can be and actually achieve stuff - obviously certain things have to be taken at face value because the effort in not doing so massively outweighs the benefits you might gain from a more detailed assessment.
But if you read an article (as in this case) which has obvious flaws, not to point them out just because you can't research every drug or treatment a doctor gives me would clearly be absurd.
And I'm not saying he's a crackpot, I'm saying that in this instance he's carried out an experiment with a flawed methodology and drawn overly strong conclusions from it. The can be a (former) professor at Berkeley or a high school drop out, it doesn't change the weaknesses in what he's written in that article.
But if you read an article (as in this case) which has obvious flaws, not to point them out just because you can't research every drug or treatment a doctor gives me would clearly be absurd.
And I'm not saying he's a crackpot, I'm saying that in this instance he's carried out an experiment with a flawed methodology and drawn overly strong conclusions from it. The can be a (former) professor at Berkeley or a high school drop out, it doesn't change the weaknesses in what he's written in that article.
If I stood on one leg “to exhaustion” — until it hurt too much to continue — a few times, I woke up feeling more rested
Wouldn't it be easier to just exercise? Regular physical activity leads to better sleep, and doesn't require you to stand on one leg.
Wouldn't it be easier to just exercise? Regular physical activity leads to better sleep, and doesn't require you to stand on one leg.
It's actually a good alternative to regular physical activity. You can do it anywhere for any length of time and it will dramatically increase the load you can withstand at a given angle.
http://en.wikipedia.org/wiki/Isometric_exercise
"While dynamic exercises are slightly better than isometric exercises at enhancing the twitch force of a muscle, isometrics are significantly better than dynamic exercises at increasing maximal strength at the joint angle. Flexibility may be increased when isometrics are performed at joint range of motion extremes. These isometric contractions recruit muscle fibers that are often neglected in some dynamic exercises. For example, gymnasts are extremely strong at great ranges of motion through the practice of isometric holds."
http://en.wikipedia.org/wiki/Isometric_exercise
"While dynamic exercises are slightly better than isometric exercises at enhancing the twitch force of a muscle, isometrics are significantly better than dynamic exercises at increasing maximal strength at the joint angle. Flexibility may be increased when isometrics are performed at joint range of motion extremes. These isometric contractions recruit muscle fibers that are often neglected in some dynamic exercises. For example, gymnasts are extremely strong at great ranges of motion through the practice of isometric holds."
What he's doing is exercise. He's working a muscle to exhaustion, which is essentially weightlifting without the weights. I guess the value is that he's found a simple workout that gets the results he needs in eight minutes a day with no equipment.
It would be interesting to find out if he would get better or worse results through cardio instead of working a muscle to exhaustion, and whether or not it is important that the legs be that muscle. Some people suffer from restless legs syndrome which keeps them from achieving deep sleep, and maybe exercising the legs is countering this.
(I wish there was some way he could experiment that wasn't subject to the placebo effect, though, which makes the results moot when applied to anyone else.)
It would be interesting to find out if he would get better or worse results through cardio instead of working a muscle to exhaustion, and whether or not it is important that the legs be that muscle. Some people suffer from restless legs syndrome which keeps them from achieving deep sleep, and maybe exercising the legs is countering this.
(I wish there was some way he could experiment that wasn't subject to the placebo effect, though, which makes the results moot when applied to anyone else.)
Well, yeah. It just seems a little strange (and likely less efficient) to do leg raises, when a decent workout of real exercises would be just as easy and more effective.
Even if you stick to the legs, exercises like bootstrappers, squats, and lunges would be significantly better.
Even if you stick to the legs, exercises like bootstrappers, squats, and lunges would be significantly better.
(see above comment) dynamic exercise is only more effective at increasing muscle twitch force. isometrics will be more effective at increasing strength at joint angle. for example, i could do more squats in a minute (up to 90) after i did isometric holds vs before.
Fair enough. I just didn't get the impression that the OP took an exercise approach to his experiment. It seemed less isometrics and more "I'm gonna stand on one leg until I get tired."
Interesting, useless experiment. It was invalid from the start. He went in 'knowing' that standing for long periods 'made' him sleep better. That may or may not be true. It is anecdotal. Not only that, but he may have drawn the same conclusion that standing on one leg also worked and that doing 4 sets was best, etc. I'm not saying that standing's effect on sleep doesn't warrant proper exploration. I am saying that the placebo effect cannot be ruled out with this guy's work.
We seem to be approaching self-satire here on HN.
My Mom always told me a glass of milk would help me sleep better. It isn't scientific, but if I'm stressed out and have a glass of milk before bed I'll relax more and sleep better.
One legged standing could be just a good! I'll definitely try it out.
One legged standing could be just a good! I'll definitely try it out.
The calcium in the milk helps: http://www.medicalnewstoday.com/articles/163169.php
Why is this on Hacker News!?
Do squats, acquire sleep.
His measure of how well rested he is is completely subjective. There's obviously no blinding (as he's subject and experimenter) which makes it almost certain that as he knows the hypothesis he's looking into and the measure is subjective, he's going to either subconsciously or consciously push the data towards proving or disproving it based on his preconceived ideas.
He talks about randomly choosing (possibly sloppy language but if you choose in any way it's not random, regardless it's not clear) and he's failed to adjust for or control any other variables that might influence sleep like, oooo, alcohol or caffeine or stress or a whole bunch of things that will make more of a difference than standing on one leg which given the relatively small data set is a pretty big deal (again some of this may depend on the method of randomisation).
And then the variation is within fractions of one percent (the scale showing 99 and 99.4) which given that the measure is subjective is essentially insignificant.
Please, please tell me that this is a wind up about how people believe anything scientific looking and trying to get people to stand on one leg for lulz rather than something serious?
Given that what he's doing is basically exercise and that linking sleep and exercise makes some sense, it's not a totally stupid hypothesis, but his experiment proves nothing.