Nate Silver on the Leaked Global Warming Emails(fivethirtyeight.com)
fivethirtyeight.com
Nate Silver on the Leaked Global Warming Emails
http://www.fivethirtyeight.com/2009/11/i-read-through-160000000-bytes-of.html
50 comments
I joke about the unfortunate demise of my competitors all the time. It doesn't mean I'm actively working to destroy them.
If you think that Saiers is in the greenhouse skeptics
camp, then, if we can find documentary evidence of this,
we could go through official AGU channels to get him ousted.
(From Tom Wigley).
http://en.wikipedia.org/wiki/Tom_Wigley
And if the AGU accepts his denial of climate change as a legitimate reason to oust him there is nothing wrong with that. He didn't say "we're going to lie about data and make up stuff and get my colleague fired," he said, "We're going to present evidence of what my colleague believes is good science. If this does not meet the standard of our scientific body, we will expel him from it."
He said skeptic, not denier. That's a wide gulf.
Which makes me question the validity of his work. If he's so concerned about an adversary, IMHO it appears as though he believes his work will not stand up to scrutiny.
That assumes a high correlation between validity, acceptance and success.
Peer review itself is conservative, reactionary and political. There is a strong "confirmation" bias in what gets published. This is good, I suppose, but:
It takes a lot more than being right to get your results out there and accepted by your peers. And if your results contradict the existing consensus, god help you! You'd have to be very politically astute to be published at all, and hang on to your funding. Conversely, if your results appear to support the existing consensus, or can be doctored up to do so, then the "validity" requirement is greatly relaxed. Either way, political concerns will always play a role.
Peer review itself is conservative, reactionary and political. There is a strong "confirmation" bias in what gets published. This is good, I suppose, but:
It takes a lot more than being right to get your results out there and accepted by your peers. And if your results contradict the existing consensus, god help you! You'd have to be very politically astute to be published at all, and hang on to your funding. Conversely, if your results appear to support the existing consensus, or can be doctored up to do so, then the "validity" requirement is greatly relaxed. Either way, political concerns will always play a role.
Which is rather depressing, politics has no place in science . . . except maybe political 'science'.
Valid data and good research should always be available, which is one major failing of the peer review. However, the peer review is a necessary demon to keep pure bullshit out of scientific publications.
Valid data and good research should always be available, which is one major failing of the peer review. However, the peer review is a necessary demon to keep pure bullshit out of scientific publications.
Here's more of the same:
"I can't see either of these papers being in the next IPCC report. K and I will keep them out somehow - even if we have to redefine what the peer-review literature is !"
While I agree that much of what I've seen quoted could be explained, I can't see any reasonable way to make this one seem benign.
"I can't see either of these papers being in the next IPCC report. K and I will keep them out somehow - even if we have to redefine what the peer-review literature is !"
While I agree that much of what I've seen quoted could be explained, I can't see any reasonable way to make this one seem benign.
I think what you see are scientists trying to do their job in an incredibly politically-charged climate.
E&E is not a peer-reviewed scientific journal, but a social science publication (or perhaps trade journal?), whose editor (Sonja) has claimed: “I’m following my political agenda — a bit, anyway. But isn’t that the right of the editor?”.
http://en.wikipedia.org/wiki/Energy_and_Environment
I imagine if I were a scientist it would be immensely frustrating to see politicians citing articles published in E&E as if it were a credible scientific journal, knowing that most people are not aware of the difference.
E&E is not a peer-reviewed scientific journal, but a social science publication (or perhaps trade journal?), whose editor (Sonja) has claimed: “I’m following my political agenda — a bit, anyway. But isn’t that the right of the editor?”.
http://en.wikipedia.org/wiki/Energy_and_Environment
I imagine if I were a scientist it would be immensely frustrating to see politicians citing articles published in E&E as if it were a credible scientific journal, knowing that most people are not aware of the difference.
This isn't the first time people have raised questiones about Philip Jones and the Climate Research Unit. HN discussion here: http://news.ycombinator.com/item?id=843517
Obviously, an article in the National Review isn't worth much as evidence by itself. But it quotes reputable scientists who have gone on record as saying they believe he is stonewalling their requests to see his raw data.
The CRU has acknowledged that the files are genuine. In one of the emails, Phil Jones brags how he unethically manipulated the presentation of his results (Nate's words, not mine). And we might not be able to go back and check his results against his measurements.
It seems to me that this would be a good time for an audit of the climate change data. Our first priority should be maintaining the integrity of the scientific process, and in this case that means getting rid of the appearance of impropriety. It's wrong to hold back scientific criticism because of the way we feel about whichever group is currently using the scientific consensus to advantage in the infinite clown fight that is partisan politics.
EDIT: this post went through several versions, most of which were bad. Sorry to those who replied to something that's gone now.
Obviously, an article in the National Review isn't worth much as evidence by itself. But it quotes reputable scientists who have gone on record as saying they believe he is stonewalling their requests to see his raw data.
The CRU has acknowledged that the files are genuine. In one of the emails, Phil Jones brags how he unethically manipulated the presentation of his results (Nate's words, not mine). And we might not be able to go back and check his results against his measurements.
It seems to me that this would be a good time for an audit of the climate change data. Our first priority should be maintaining the integrity of the scientific process, and in this case that means getting rid of the appearance of impropriety. It's wrong to hold back scientific criticism because of the way we feel about whichever group is currently using the scientific consensus to advantage in the infinite clown fight that is partisan politics.
EDIT: this post went through several versions, most of which were bad. Sorry to those who replied to something that's gone now.
CRU's work hardly constitutes an "entire branch" of climate science or meteorology, must less the whole tree (!) of human knowledge.
What happened here is that some players in a controversy were revealed (gasp) to have strong opinions, and even an agenda. And other players, themselves no less possessed of agenda, are using it to attack the subject via guilt by association. Meh. Lots for Fox News to spit about. Not much really interesting here.
What happened here is that some players in a controversy were revealed (gasp) to have strong opinions, and even an agenda. And other players, themselves no less possessed of agenda, are using it to attack the subject via guilt by association. Meh. Lots for Fox News to spit about. Not much really interesting here.
some players in a controversy were revealed (gasp) to have strong opinions
Foul ball. This is not what was revealed.
They're shown to have acted on those opinions, at the very least, unethically manipulating the presentation of study results to exaggerate the conclusion they agreed with.
Any maybe more than that. Depending on the context (which I don't know), it may show that they actually manipulated the data, which calls into question all further research that uses the data coming out of that study.
Foul ball. This is not what was revealed.
They're shown to have acted on those opinions, at the very least, unethically manipulating the presentation of study results to exaggerate the conclusion they agreed with.
Any maybe more than that. Depending on the context (which I don't know), it may show that they actually manipulated the data, which calls into question all further research that uses the data coming out of that study.
[deleted]
I don't understand why conservatives are so happy about this. If an entire branch in the tree of human knowledge is unsound, that's an unambiguously bad thing. It means, at the least, that we have a lot of work to do over again.
Maybe those people are happy because they have been saying for a while that an entire branch of human knowledge is unsound and that there is a lot of work to do over again?
Maybe those people are happy because they have been saying for a while that an entire branch of human knowledge is unsound and that there is a lot of work to do over again?
His post is pure strawman: say the "enemy's" entire argument centers around one post, then downplay that, then announce that there's nothing to see here folks, move along, move along. It's far more devstating than that. Excerpt:
At 09:41 AM 2/2/2005, Phil Jones wrote:
Mike, I presume congratulations are in order - so congrats etc !
Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs [ed- McIntyre and McKittrick, AGW skeptics] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works on precedents, so the first request will test it.We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it - thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it !
http://blogs.news.com.au/heraldsun/andrewbolt/index.php/hera...
Update: Here's a summary of many of the emails. This gives an even stronger sense of what a strawman argument Silver used. Interesting stuff: http://bishophill.squarespace.com/blog/2009/11/20/climate-cu...
From that link: Jones says that UK climate organisations are coordinating themselves to resist FoI [Freedom of Information Act]. They got advice from the Information Commissioner [!](1219239172)
and this:
Jones tells Mann that he is sending station data. Says that if McIntyre requests it under FoI he will delete it rather than hand it over. Says he will hide behind data protection laws. Says Rutherford screwed up big time by creating an FTP directory for Osborn. Says Wigley worried he will have to release his model code.
At 09:41 AM 2/2/2005, Phil Jones wrote:
Mike, I presume congratulations are in order - so congrats etc !
Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs [ed- McIntyre and McKittrick, AGW skeptics] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works on precedents, so the first request will test it.We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it - thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it !
http://blogs.news.com.au/heraldsun/andrewbolt/index.php/hera...
Update: Here's a summary of many of the emails. This gives an even stronger sense of what a strawman argument Silver used. Interesting stuff: http://bishophill.squarespace.com/blog/2009/11/20/climate-cu...
From that link: Jones says that UK climate organisations are coordinating themselves to resist FoI [Freedom of Information Act]. They got advice from the Information Commissioner [!](1219239172)
and this:
Jones tells Mann that he is sending station data. Says that if McIntyre requests it under FoI he will delete it rather than hand it over. Says he will hide behind data protection laws. Says Rutherford screwed up big time by creating an FTP directory for Osborn. Says Wigley worried he will have to release his model code.
Does this somehow disprove the information that has been presented though? Just because the presentation has been manipulated to maximize the statement for his cause–and those scientists were at the very least prejudiced towards their opponents–doesn't change the impact of their data.
To me though, the argument against cap-and-trade and against heavy investment in alt-energy is entirely lost and asinine to continue to chase. These resources are finite and the atmosphere's tolerance is similarly and provably so. Now is as good a time to re-ignite our initiatives in sustainable energy as any, and there are significant benefits to the many post-industrial western economies that are glutted with well-educated but under-employed people. IT and IS can't be the only thing that we have over developing nations willing to exploit themselves to gain true 1st world status.
To me though, the argument against cap-and-trade and against heavy investment in alt-energy is entirely lost and asinine to continue to chase. These resources are finite and the atmosphere's tolerance is similarly and provably so. Now is as good a time to re-ignite our initiatives in sustainable energy as any, and there are significant benefits to the many post-industrial western economies that are glutted with well-educated but under-employed people. IT and IS can't be the only thing that we have over developing nations willing to exploit themselves to gain true 1st world status.
On first glance, it seems pretty bad, but on second thought I read more of a kind of paranoia in it. Almost every set of data can be interpreted in multiple ways, depending on the context in which the data is placed. People can twist and subvert data in ways that are very hard to contradict. It's often easier to sell lies than to sell truths and I sense an extreme fear that they have to spend all of their time defending complicated truths against simple falsehoods.
Edit: by which I do not mean they are right and the skeptics are the ones subverting the data and convincing others with simple falsehoods. You may fear your opponent doing that while doing it yourself. The above can be read with the sides interchanged. I'm merely pointing out that reality is underdetermined by data, that truths are sometimes hard to sell and that that leads all sides to fear the other side undermining them in ways they consider immoral.
Plain human psychology may be enough to explain this, without invoking all kinds of conspiracies. Conspiracies are extremely unlikely, because there will always be someone that blabs. A few people imploring others to do immoral things does not a conspiracy make.
Edit: by which I do not mean they are right and the skeptics are the ones subverting the data and convincing others with simple falsehoods. You may fear your opponent doing that while doing it yourself. The above can be read with the sides interchanged. I'm merely pointing out that reality is underdetermined by data, that truths are sometimes hard to sell and that that leads all sides to fear the other side undermining them in ways they consider immoral.
Plain human psychology may be enough to explain this, without invoking all kinds of conspiracies. Conspiracies are extremely unlikely, because there will always be someone that blabs. A few people imploring others to do immoral things does not a conspiracy make.
You make a good point. But I wonder, why would the scientists be paranoid? Why aren't they releasing their data and analysis algorithms as part of the package?
I mean, shouldn't the process be something like this:
1. Gather data 2. Analyze it 3. Publish results. 4. Make data and analysis methods available for other to scrutinize. 5. Others review data and methods. 6. New insights from others are realized.
Carl Sagan, my hero, went on and on science's strength being being that although mistakes are made, they can be corrected through the scrutiny of others, and the picture of truth improves. That process hinges on data and methods being available for others to review.
Of course, data will fall into the hands of idiots who will misrepresent, lie, and deceive. Also, your data will fall into the hands of someone who will find an error, or legitimately refute your claims. And that's a good thing.
Some years ago, some astronomers who had previously published evidence supporting the existence of a new planet in another solar system, presented at a conference that further analysis of their own data led them to reverse their conclusion. Their presentation was about a minute long. They received a standing ovation. (I believe I saw this on an episode of Nova - I can't cite a source)
That's how it's supposed to work.
I mean, shouldn't the process be something like this:
1. Gather data 2. Analyze it 3. Publish results. 4. Make data and analysis methods available for other to scrutinize. 5. Others review data and methods. 6. New insights from others are realized.
Carl Sagan, my hero, went on and on science's strength being being that although mistakes are made, they can be corrected through the scrutiny of others, and the picture of truth improves. That process hinges on data and methods being available for others to review.
Of course, data will fall into the hands of idiots who will misrepresent, lie, and deceive. Also, your data will fall into the hands of someone who will find an error, or legitimately refute your claims. And that's a good thing.
Some years ago, some astronomers who had previously published evidence supporting the existence of a new planet in another solar system, presented at a conference that further analysis of their own data led them to reverse their conclusion. Their presentation was about a minute long. They received a standing ovation. (I believe I saw this on an episode of Nova - I can't cite a source)
That's how it's supposed to work.
Nate Silver is right. This says absolutely nothing about global warming itself. It says something about how scientific research and academia work.
They are in a gold rush. As long as the climate change debate remains top of the agenda the money will keep on coming. Securing continued funding for their projects is the main and sometimes only concern of scientists. At least that is what I conclude from many years of working with them.
They are funded by politicians and other sources that will dry up once the global global warming hype ends. What politicians need is a clear and unambiguous picture that speaks truth beyond reasonable doubt. That's what politicians will pay climate scientists for and that's what they are struggling so hard to deliver.
All this is completely without connection to the question of climate change proper.
They are in a gold rush. As long as the climate change debate remains top of the agenda the money will keep on coming. Securing continued funding for their projects is the main and sometimes only concern of scientists. At least that is what I conclude from many years of working with them.
They are funded by politicians and other sources that will dry up once the global global warming hype ends. What politicians need is a clear and unambiguous picture that speaks truth beyond reasonable doubt. That's what politicians will pay climate scientists for and that's what they are struggling so hard to deliver.
All this is completely without connection to the question of climate change proper.
That's what politicians will pay climate scientists for and that's what they are struggling so hard to deliver.
Doesn't this imply that there is a major conflict of interest here and that the whole science should be labeled suspect by default? I think most people will give the bosses whatever they want even if it isn't true.
Doesn't this imply that there is a major conflict of interest here and that the whole science should be labeled suspect by default? I think most people will give the bosses whatever they want even if it isn't true.
No doubt there is a major conflict of interest. But there are other forces at play as well that mitigate the issue. Scientists are obviously concerned about their name, so outright lying is probably rare.
Making things appear clearer than they actually are, hiding stuff that is inconclusive, writing project proposals opportunistically according to where the money is, these are the things that happen all the time.
And I'm not talking specifically about climate research, but about the driving forces of research and research funding in europe.
Making things appear clearer than they actually are, hiding stuff that is inconclusive, writing project proposals opportunistically according to where the money is, these are the things that happen all the time.
And I'm not talking specifically about climate research, but about the driving forces of research and research funding in europe.
Securing continued funding for their projects is the main and sometimes only concern of scientists. At least that is what I conclude from many years of working with them.
Data is not the plural of anecdote. In addition I find it insulting that you would attempt to negatively characterize an entire field which you have no knowledge of. In fact, this comment is so unbelievable that I would like to know where and who you worked with.
Data is not the plural of anecdote. In addition I find it insulting that you would attempt to negatively characterize an entire field which you have no knowledge of. In fact, this comment is so unbelievable that I would like to know where and who you worked with.
I'm in europe. Maybe that gives you some peace of mind.
Ha! I doubt it's much different in the US.
[deleted]
It is not a very good post.
Nate takes one email and creates a strawman argument around it, claiming that the email is taken out of context and does not represent a smoking gun. He then ignores all the other 1078 or so emails.
Nate takes one email and creates a strawman argument around it, claiming that the email is taken out of context and does not represent a smoking gun. He then ignores all the other 1078 or so emails.
I believe you're holding Nate Silver up to an unrealistic standard. This is obviously not a complete refutation of all of the stolen data, nor does it claim to be. He is instead taking an instance of what people are complaining about -- a pretty standard example of the things that get pasted about this story, from what I've seen -- and pointing out that people are blowing this particular one out of proportion.
Maybe you're complaining that no one is really complaining about this specific email, but to me, this one looks a lot like the others contained in the stolen data that I've seen. From my perspective, it seems a lot of people are reading what they want into the emails, though I'll readily admit I haven't spent my day pouring through it, nor do I really intend to.
Maybe you're complaining that no one is really complaining about this specific email, but to me, this one looks a lot like the others contained in the stolen data that I've seen. From my perspective, it seems a lot of people are reading what they want into the emails, though I'll readily admit I haven't spent my day pouring through it, nor do I really intend to.
If nothing else, this will make people think twice -- if even the moral crusader Nate Silver admits that it's common to fudge data to make things look more compelling, they'll think about that next time someone tells them what the temperature change over the next thirty years will be.
True. Whichever way that temperature appears to be changing.
Did you read the title of his post?
"I Read Through 160,000,000 Bytes of Hacked Files And All I Got Was This Lousy E-Mail"
It seems to imply to me to be a complete refutation of all the stolen data.
I'm usually a fan of 538 and feel pretty level headed in my opinions on global warming, but Nate could have definitely done better.
"I Read Through 160,000,000 Bytes of Hacked Files And All I Got Was This Lousy E-Mail"
It seems to imply to me to be a complete refutation of all the stolen data.
I'm usually a fan of 538 and feel pretty level headed in my opinions on global warming, but Nate could have definitely done better.
It's far more illustrative to debunk (or in this case, merely explain) a single case--which Nate points out is what "the conservatives are mainly zeroing in on"--than to address all "hundreds or thousands of e-mails and documents." I'm sure Nate COULD have done so, and probably to the same effect, but what explanatory purpose would that serve?
I'd rather see the emails between the guys at the Goldman Sachs carbon trading desk. Also, these guys aren't stupid (see the one guy's interest in destroying emails) and are going to be careful when communicating electronically.
real climate is my goto for climate science - well written by scientists for interested non-scientists (note: this doesn't mean it's dumbed down - be prepared for some detail)
http://www.realclimate.org/index.php/archives/2009/11/the-cr...
From here:
http://www.examiner.com/x-25061-Climate-Change-Examiner~y200...
"Anyway, I wanted you guys to know that you’re free to use RC [RealClimate.org - A supposed neutral climate change website] Rein any way you think would be helpful. Gavin and I are going to be careful about what comments we screen through, and we’ll be very careful to answer any questions that come up to any extent we can. On the other hand, you might want to visit the thread and post replies yourself. We can hold comments up in the queue and contact you about whether or not you think they should be screened through or not, and if so, any comments you’d like us to include."
If the comments are modded like that (keeping only the "favorable ones"), I wouldn't be so sure about RC's quality
"Anyway, I wanted you guys to know that you’re free to use RC [RealClimate.org - A supposed neutral climate change website] Rein any way you think would be helpful. Gavin and I are going to be careful about what comments we screen through, and we’ll be very careful to answer any questions that come up to any extent we can. On the other hand, you might want to visit the thread and post replies yourself. We can hold comments up in the queue and contact you about whether or not you think they should be screened through or not, and if so, any comments you’d like us to include."
If the comments are modded like that (keeping only the "favorable ones"), I wouldn't be so sure about RC's quality
That's not news, though, or at least it shouldn't be. RC's moderation policy of deleting or not approving things they don't want to deal with is well-known, at least to me. I believe it is within the rights of a site owner to do that. If you don't like that, take it into account as you read the site. We didn't need any leaked emails to tell us that comments can be deleted on RC.
(Though it probably is fair to point out that there is nothing neutral about the site, any more than the talk.origins faq is neutral. Doesn't make it wrong; does make it not neutral.)
(Though it probably is fair to point out that there is nothing neutral about the site, any more than the talk.origins faq is neutral. Doesn't make it wrong; does make it not neutral.)
Science isn't even supposed to be neutral; leave that to Wikipedia. I don't even know why anyone would expect a site presenting accurate scientific information to be neutral, wouldn't that completely miss the point?
Looking at the RC thread linked above, it sure doesn't seem as if the moderators are deleting all unfavorable comments.
Humorously conservatives (who somehow became anti-science) are cheering this theft but were appalled by the theft of emails from Sarah Palin that showed she was breaking the law (State business on personal email).
I'm sorry, don't you mean to make fun of liberals for engaging in the Palin email theft (the person who did it was the son of a Democratic politician) and condemning this one?
Or maybe you mean to joke about how tribal humans are. That's probably it.
Or maybe you mean to joke about how tribal humans are. That's probably it.
I'd love to see what kind of emails tend to go back and forth among the climate change denial crew. If only we had some hackers around here - we might be able to find out!
We have many hackers here (pen hackers, not "hacker" hackers). As far as I know, my colleagues and I are unwilling to break the law and privacy to view others' communications.
It would be interesting to find out who performed this crack, and who was the paymaster.
[deleted](1)
You can tell Nate Silver's really definitely right, because he uses lots of exclamation points when he's strawmanning his opponents.
"Of course, if it does get published, maybe the resulting settlement would shut down E&E and Benny and Sonja all together! We can only hope, anyway. So maybe in an odd way its actually win-win for us, not them. Lets see how this plays out..."
This says that Michael Mann hopes to have E&E shut down.. just because they post papers opposing his views.
http://en.wikipedia.org/wiki/Michael_E._Mann http://www.multi-science.co.uk/ee.htm