BBC: Climate e-mail hack 'will impact on Copenhagen summit' (according to Saudi Arabia’s negotiator)

[OKIsItJustMe]

http://news.bbc.co.uk/2/hi/science/nature/8392611.stm

15:25 GMT, Thursday, 3 December 2009

Climate e-mail hack 'will impact on Copenhagen summit'

By Richard Black
Environment correspondent, BBC News website

E-mails hacked from a climate research institute suggest climate change does not have a human cause, according to Saudi Arabia's lead climate negotiator.

Mohammad Al-Sabban told BBC News that the issue will have a "huge impact" on next week's UN climate summit, with countries unwilling to cut emissions.

…

Mr Al-Sabban made clear that he expects it to derail the single biggest objective of the summit - to agree limitations on greenhouse gas emissions.

"It appears from the details of the scandal that there is no relationship whatsoever between human activities and climate change," he told BBC News.

"Climate is changing for thousands of years, but for natural and not human-induced reasons.

…

:banghead:

[rockymountaindem]

1. I thought I heard somewhere those e-mails were a hoax

What's the deal?

[slackmaster]

3. They're apparently real, but out of context they are prone to all kinds of bizarre misinterpretation

I can't tell what any of them really mean without intimate knowledge of the people who wrote them and of their relationships with the people they were intended for.

They mean nothing to me, other than evidence that the people who wrote them had a false sense of security about the privacy of emails.

[OKIsItJustMe]

4. The best propaganda has a germ of truth in it

The e-mails are genuine. They contain the quotes attributed to them.

However, if you take the time to actually read them (rather than just reading 2 sentences, seized upon by the “skeptics”) you’ll find that they really aren’t the smoking gun they’re made out to be.

“The fact is that we can't account for the lack of warming at the moment and it is a travesty that we can't.”

Sounds pretty bad, doesn't it?

How about this:

“The fact is that we can't account for the lack of warming at the moment and it is a travesty that we can't. The CERES data published in the August BAMS 09 supplement on 2008 shows there should be even more warming: but the data are surely wrong. Our observing system is inadequate.”

http://www.eastangliaemails.com/emails.php?eid=1048&filename=1255352257.txt

From: Kevin Trenberth <[e-mail]>
To: Michael Mann <[e-mail]>
Subject: Re: BBC U-turn on climate
Date: Mon, 12 Oct 2009 08:57:37 -0600
Cc: Stephen H Schneider <[e-mail]>, Myles Allen <[e-mail]>,
peter stott <[e-mail]>, "Philip D. Jones" <[e-mail]>,
Benjamin Santer <[e-mail]>, Tom Wigley <[e-mail]>,
Thomas R Karl <[e-mail]>, Gavin Schmidt <[e-mail]>,
James Hansen <[e-mail]>, Michael Oppenheimer <[e-mail]>

Hi all
Well I have my own article on where the heck is global warming? We are asking that here in
Boulder where we have broken records the past two days for the coldest days on record. We
had 4 inches of snow. The high the last 2 days was below 30F and the normal is 69F, and it
smashed the previous records for these days by 10F. The low was about 18F and also a
record low, well below the previous record low. This is January weather (see the Rockies
baseball playoff game was canceled on saturday and then played last night in below freezing
weather).
Trenberth, K. E., 2009: An imperative for climate change planning: tracking Earth's global
energy. Current Opinion in Environmental Sustainability, 1, 19-27,
doi:10.1016/j.cosust.2009.06.001. <1> (A PDF of the published version can be obtained
from the author.)
The fact is that we can't account for the lack of warming at the moment and it is a
travesty that we can't. The CERES data published in the August BAMS 09 supplement on 2008
shows there should be even more warming: but the data are surely wrong. Our observing
system is inadequate.
That said there is a LOT of nonsense about the PDO. People like CPC are tracking PDO on a
monthly basis but it is highly correlated with ENSO. Most of what they are seeing is the
change in ENSO not real PDO. It surely isn't decadal. The PDO is already reversing with
the switch to El Nino. The PDO index became positive in September for first time since
Sept 2007. see
<2>http://www.cpc.ncep.noaa.gov/products/GODAS/ocean_briefing_gif/global_ocean_monitoring_c
urrent.ppt
Kevin
Michael Mann wrote:

extremely disappointing to see something like this appear on BBC. its particularly odd,
since climate is usually Richard Black's beat at BBC (and he does a great job). from
what I can tell, this guy was formerly a weather person at the Met Office.

We may do something about this on RealClimate, but meanwhile it might be appropriate for
the Met Office to have a say about this, I might ask Richard Black what's up here?

mike

On Oct 12, 2009, at 2:32 AM, Stephen H Schneider wrote:

Hi all. Any of you want to explain decadal natural variability and signal to noise and
sampling errors to this new "IPCC Lead Author" from the BBC? As we enter an El Nino year
and as soon, as the sunspots get over their temporary--presumed--vacation worth a few
tenths of a Watt per meter squared reduced forcing, there will likely be another dramatic
upward spike like 1992-2000. I heard someone--Mike Schlesinger maybe??--was willing to bet
alot of money on it happening in next 5 years?? Meanwhile the past 10 years of global mean
temperature trend stasis still saw what, 9 of the warmest in reconstructed 1000 year record
and Greenland and the sea ice of the North in big retreat?? Some of you observational folks
probably do need to straighten this out as my student suggests below. Such "fun", Cheers,
Steve
Stephen H. Schneider
Melvin and Joan Lane Professor for Interdisciplinary Environmental Studies,
Professor, Department of Biology and
Senior Fellow, Woods Institute for the Environment
Mailing address:
Yang & Yamazaki Environment & Energy Building - MC 4205
473 Via Ortega
Ph: 650 725 9978
F: 650 725 4387
Websites: climatechange.net
patientfromhell.org
----- Forwarded Message -----
From: "Narasimha D. Rao" <<3>[e-mail]>
To: "Stephen H Schneider" <<4>[e-mail]>
Sent: Sunday, October 11, 2009 10:25:53 AM GMT -08:00 US/Canada Pacific
Subject: BBC U-turn on climate
Steve,
You may be aware of this already. Paul Hudson, BBC's reporter on climate change, on Friday
wrote that there's been no warming since 1998, and that pacific oscillations will force
cooling for the next 20-30 years. It is not outrageously biased in presentation as are
other skeptics' views.


<5>http://news.bbc.co.uk/2/hi/science/nature/8299079.stm
<6>http://blogs.telegraph.co.uk/news/damianthompson/100013173/the-bbcs-amazing-u-turn-on-cl
imate-change/


BBC has significant influence on public opinion outside the US.


Do you think this merits an op-ed response in the BBC from a scientist?


Narasimha


-------------------------------
PhD Candidate,
Emmett Interdisciplinary Program in Environment and Resources (E-IPER)
Stanford University
Tel: [phone]


--
Michael E. Mann
Professor
Director, Earth System Science Center (ESSC)
Department of Meteorology Phone: [phone]
503 Walker Building FAX: [phone]
The Pennsylvania State University email: <7>[e-mail]
University Park, PA 16802-5013
website: <8>http://www.meteo.psu.edu/~mann/Mann/index.html
"Dire Predictions" book site:
<9>http://www.essc.psu.edu/essc_web/news/DirePredictions/index.html

--
****************
Kevin E. Trenberth e-mail: <10>[e-mail]
Climate Analysis Section, <11>www.cgd.ucar.edu/cas/trenbert.html
NCAR
P. O. Box 3000, [phone]
Boulder, CO 80307 [phone] (fax)

Street address: 1850 Table Mesa Drive, Boulder, CO 80305

References

1. http://www.cgd.ucar.edu/cas/Trenberth/trenberth.papers/EnergyDiagnostics09final.pdf
2. http://www.cpc.ncep.noaa.gov/products/GODAS/ocean_briefing_gif/global_ocean_monitoring_current.ppt
3. mailto:[e-mail]
4. mailto:[e-mail]
5. http://news.bbc.co.uk/2/hi/science/nature/8299079.stm
6. http://blogs.telegraph.co.uk/news/damianthompson/100013173/the-bbcs-amazing-u-turn-on-climate-change/
7. mailto:[e-mail]
8. http://www.meteo.psu.edu/%7Emann/Mann/index.html
9. http://www.essc.psu.edu/essc_web/news/DirePredictions/index.html
10. mailto:[e-mail]
11. http://www.cgd.ucar.edu/cas/trenbert.html

[rockymountaindem]

7. Thanks

That makes sense.

[Nederland]

8. I have read one that is VERY disturbing

http://www.di2.nu/foia/HARRY_READ_ME.txt

[muriel_volestrangler]

10. Which bit is disturbing? (nt)

[Nederland]

12. Did you read it?

I know it's long, but the criticism being laid down here is that these things are being taken out of context. As a result, I am reluctant to cut and paste a small section of the file out of fear that I'll be labelled a denier.

[muriel_volestrangler]

15. No, because it's huge, and I don't know the context

It's summarising someone's work on a dataset for 3 years. You don't have to paste a small section, just give the words with which the section that worries you so much begins. Of course, more context for the readme (eg if Harry's was ever actually used by other people, and if so, how) would mean there might be a reason for reading it. But if you think the part of the readme around the worrying bit explains it suffeiciently, I'll be able to see that.

But as it stands, there's nothing there to be worried about at all. Because we don't know enough about it to worry.

[Nederland]

20. Try this

What is being painted here is a picture of data that is horribly mangled and a programmer trying his best to fix it. When you go into the code and look at what he had to do to fix the data, you quickly realize that assumptions are being made about what the "correct" data should look like, and coding things to make the end result match your desired results.

Here is only one example of the types of problems he had with the data:

It takes time.. time I don't have! Though I'm pleased to see that the second FSM is helpfully
chipping in to pair things up when possible.

getting seriously fed up with the state of the Australian data. so many new stations have been
introduced, so many false references.. so many changes that aren't documented. Every time a
cloud forms I'm presented with a bewildering selection of similar-sounding sites, some with
references, some with WMO codes, and some with both. And if I look up the station metadata with
one of the local references, chances are the WMO code will be wrong (another station will have
it) and the lat/lon will be wrong too. I've been at it for well over an hour, and I've reached
the 294th station in the tmin database. Out of over 14,000. Now even accepting that it will get
easier (as clouds can only be formed of what's ahead of you), it is still very daunting. I go
on leave for 10 days after tomorrow, and if I leave it running it isn't likely to be there when
I return! As to whether my 'action dump' will work (to save repetition).. who knows?

Yay! Two-and-a-half hours into the exercise and I'm in Argentina!

Pfft.. and back to Australia almost immediately :-( .. and then Chile. Getting there.

Unfortunately, after around 160 minutes of uninterrupted decision making, my screen has started
to black out for half a second at a time. More video cable problems - but why now?!! The count is
up to 1007 though.

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as
Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO
and one with, usually overlapping and with the same station name and very similar coordinates. I
know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh!
There truly is no end in sight. Look at this:

[muriel_volestrangler]

21. And how did that problem, about 2 years ago, affect the data?

The job appears to have been to merge data from various countries around the world into one format, and decide whether records were duplicates, dummies, related or unrelated. Unsurprisingly, with so much data to work with, it took time. I don't actually see that data is being changed to produce the result that 'someone' wants, as you allege. Perhaps you could quote the bit you're worried about, rather than a bit about what the data stations are called, and what they're coordinates are. You know, one about data.

[Nederland]

23. You have two choices

1) You can read a summary from someone else about what was done. People around here reject that path because all of the summaries are being written by deniers.

2) You can dig into the data yourself and verify that the summaries are accurate.


I have done #2. I'll help you out by listing out the steps:

1) Download the FOI zipfile to your computer and unzip it.

2) Grep the entire document set for *.pro files containing the following:

"ARTIFICIAL CORRECTION"
"HIDE THE DECLINE"

(be sure to do a case insensitive search cause the capitalization is not consistent throughout.)

3) Look at the code following the comments and judge for yourself what it does.


Let me know what you think. I'd be interested to hear other opinions.

[OKIsItJustMe]

24. So here’s the part I don’t get

Let us assume (just for sake of argument) that a climate researcher (or researchers) falsified data. I’m not talking about massaging it, or cleaning it up, I’m talking about out-and-out falsifying it. (I’m not saying they did, this is just a hypothetical.)

From this, can we logically conclude that all climate researchers have been falsifying their data? (Because there’s a heck of a lot of data out there! and it pretty much all points to the same conclusion.)


If, for a moment, we assume that all climate researchers have been falsifying their data, to indicate that anthropogenic climate change is underway—a grand and glorious conspiracy has been taking place for a couple of decades now…

Why?

(i.e. What is the motive?)

Is it—as has been suggested by a certain US Senator—to boost the TV ratings for the Weather Channel?

It it—as has been suggested by certain cartoon—a pitiful cry for attention?

[Nederland]

25. Grant Money

The simple answer to why is grant money. For example, since 1990, Phil Jones has received 22.6 million dollars in grant money to study climate change. That is no small sum.

I have to say I think you are creating a straw man here. I would unequivocally state that I do not think anyone deliberately and/or knowingly falsified data. I think that all of the people involved are true believers, and see what they did as good science. They just are victims of classic group think syndrome. Don't get me wrong--I still believe that climate change is real and man made. However, I regard this event as akin to what happened at CBS news during the 2004 campaign.

If you recall, CBS news did a report on George Bush's time in the National Guard. In my opinion, what happened was that people at CBS failed to notice that one document in their collection was fake. Obviously fake. So obviously fake that it took people on the other side of the argument about 15 minutes to notice it and prove it. Did that one faked document mean that all the other evidence they had collected to prove their point was fake? No. However, it revealed a mind set that was disturbing. It was a classic example of people that are so convinced they are right about something that they loose the ability to judge things accurately. The saw something that agreed with their preconceived notions and accepted it without question.

The same thing is evidently happening in climate science today. People have become so convinced that climate change is real that they have lost sight of what doing good science looks like. You can spin the email comments a variety of ways, but you cannot spin what I saw in the code. The code deliberately takes portions of data and alters them to match a desired output. My gut tells me that is wrong, because I cannot think of a good reason to do it. I would like nothing more than to have the programmer who wrote that code step up and explain why he did what he did. Until that happens though, I'll say what I said in my original post on this thread: I am very disturbed.

[joshcryer]

34. So pick other datasets and go with them. GISS for one.

CRU is being attacked because the people behind CRU are politically active, they have a website, they debunk claims. The problem here is that people don't understand that science is an evolved process. Some people make mistakes, some people fuck up, it is the entirety of the data that tells the truth.

I personally think CRUs involvement with debunking deniers has clouded their judgment *somewhat*, but I don't think to the point of making their data invalid.

[Nederland]

38. Really?

You don't think adding arbitrary numbers to temperature data makes it invalid?

You don't think deleting specific years of a data set makes data invalid?

Interesting.

[joshcryer]

57. Do you understand the reason for removing the post 1960 data from Biffra's data?

[OKIsItJustMe]

48. Which brings us back to a sort of “Prisoner’s Dilemma”

Once again, we have to establish a conspiracy between many, many researchers, all of them holding out their hands for grant money, all of them agreeing not to point out that their colleagues are lying.

Because, if a few of them were motivated to expose the conspiracy, there would be a lot of grant money freed up…

[Nederland]

51. Exactly

And do you know what would go a long way towards clearing all this up?

Transparency

The thing that evidently has been completely lacking inside CRU.

[OKIsItJustMe]

54. Wait a second!

Are you suggesting that there is a world-wide conspiracy of silence among climate researchers!?

[Nederland]

55. No

I am suggesting the fact that FOI requests have to be issued in order to obtain data that was obtained using public funds is an indicator of a lack of transparency. Do you disagree? Can you give me a good reason why people should have to file FOI requests with the government to obtain data? I mean, this is climate data we are talking about, not nuclear weapons codes. Can you justify the actions of the CRU (and I am talking only about CRU, not a "worldwide" conspiracy). I'd love to hear your explanation but I suspect you won't provide one.

[malakai2]

71. I'm dealing with a FOI request at work right now

I can think of a couple reasons why data and documents produced with public funds are not openly available.

First is the time and effort it would take to make those documents widely available. The originals have to stay on file somewhere, and making them available to the public is out for the sake of protecting the documents themselves. Scanning everything and converting to pdf would be time consuming and a storage nightmare. Responding to requests one at a time isn't great-one of the files that might be relevant in my case is roughly 18 inches thick-but it's really the only way to keep scientists doing science and not administrative work.

Second is the sensitive nature of some of the documents. If it includes personally identifiable information, it might not be releasable. For example, records involving real estate information, or tax information, etc., even only in small part, good luck getting that under a FOI request. Same for documents including trade secrets. Consider asking EPA to tell you what the inert ingredients in a random EPA-registered pesticide are...you'll hear crickets, even if the "inert" ingredients are actually toxic, because they're a trade secret.

Third would be data that is lost or misplaced, because when you're working with thousands of documents totaling hundreds of thousands of pages, produced collaboratively by dozens or hundreds of people over many years, not everything is going to be where it's supposed to be at all times. I've managed to find a couple things at work that are copies of originals that no longer exist, or that I can't find.

There are more, but I think in this case the biggest reason is reason number one. If the scientists working on this problem are struggling with just merging data sets, applying appropriate statistical tests to data sets, and puzzling over deviations between models, raw data sets, and transformed data sets, all the while producing new data and metadata, and they are under time constraints to finish these things they're puzzling over...who in their right mind thinks they'd also have the time, or the budget, to also copy everything into some form of freely available web document? There would be a hell of a lot of data...who'd pay to store it online? Then too, a person or team that has gone to great lengths to collect data and create a useful dataset from it is not going to want that data available to someone else who might be able to scoop them by using that data to publish first. The project may have been publicly funded, but the understanding is often that the project will produce publication by the person(s) receiving the funds, and making the data available before the project is completed would threaten that. Of course, after the project is done, the money has been spent, so unless someone is willing to do a lot of work for free, there's no longer any way to make that data readily available to the public. Or, suppose a scientifically illiterate politician used some raw data, applied inappropriate statistical tests, and "proved" that funding for research on the problem is no longer needed...could happen. I wouldn't put it past James Inhofe to rail against funding EPA because they use a logarithmic transformation every time they produce pH data, and we can't have anyone "transforming" data like that.

I don't know how much of that applies to CRU, but I imagine some of it does.

[OKIsItJustMe]

79. Sorry for the delay

If we were talking about one group of researchers, who were producing all of the data which was being used to document “Anthropogenic Climate Change” then your explanation makes a small degree of sense.

But we aren’t.

Numerous organizations, coming at this from numerous different disciplines are producing data which all point to the same conclusion.
http://www.aaas.org/news/releases/2009/media/1021climate_letter.pdf

October 21, 2009

Dear Senator:

As you consider climate change legislation, we, as leaders of scientific organizations, write to state the consensus scientific view.

Observations throughout the world make it clear that climate change is occurring, and rigorous scientific research demonstrates that the greenhouse gases emitted by human activities are the primary driver. These conclusions are based on multiple independent lines of evidence, and contrary assertions are inconsistent with an objective assessment of the vast body of peer-reviewed science. Moreover, there is strong evidence that ongoing climate change will have broad impacts on society, including the global economy and on the environment. For the United States, climate change impacts include sea level rise for coastal states, greater threats of extreme weather events, and increased risk of regional water scarcity, urban heat waves, western wildfires, and the disturbance of biological systems throughout the country. The severity of climate change impacts is expected to increase substantially in the coming decades.¹

If we are to avoid the most severe impacts of climate change, emissions of greenhouse gases must be dramatically reduced. In addition, adaptation will be necessary to address those impacts that are already unavoidable. Adaptation efforts include improved infrastructure design, more sustainable management of water and other natural resources, modified agricultural practices, and improved emergency responses to storms, floods, fires and heat waves.

We in the scientific community offer our assistance to inform your deliberations as you seek to address the impacts of climate change.

---

¹The conclusions in this paragraph reflect the scientific consensus represented by, for example, the Intergovernmental Panel on Climate Change and U.S. Global Change Research Program. Many scientific societies have endorsed these findings in their own statements, including the http://www.aaas.org/news/press_room/climate_change/mtg_200702/aaas_climate_statement.pdf">American Association for the Advancement of Science, http://portal.acs.org/portal/acs/corg/content?_nfpb=true&_pageLabel=PP_SUPERARTICLE&node_id=1907&use_sec=false&sec_url_var=region1&__uuid=4e1c64b4-4f59-4243-87d1-3b926faa4dbd">American Chemical Society, http://www.agu.org/sci_pol/positions/climate_change2008.shtml">American Geophysical Union, http://ametsoc.org/policy/2007climatechange.html">American Meteorological Society, and http://www.amstat.org/news/climatechange.cfm">American Statistical Association.

American Association for the Advancement of Science 
1200 New York Avenue, NW, Washington, DC 20005 USA 
Tel: [phone]   Fax: [phone]     www.aaas.org

That's why I referred to a “World-Wide” conspiracy.



As for a FOI request, what, exactly was requested?

[joshcryer]

80. FOI requests tend to be generic shotgun sprays. "Please release all data related to X."

So when you release "most data related to X" they call conspiracy.

[muriel_volestrangler]

28. So it's not that file that's "VERY disturbing" at all for you, then?

It's another file.

So perhaps this is about the decline in tree ring growth (compared to temperatures) since 1960 - which is talked about here, and which, as that points out, was talked about in a public paper in 1998. 'Hide the decline' seems to be about the same thing - see http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/ .

[Nederland]

39. That file is a starting point

I gave the HARRY_READ_ME file as a starting point that is more accessible to most people because it is written in plain English. The real damning evidence is in the code, because you cannot argue about what the code does to the data.

[muriel_volestrangler]

43. And yet, the 'disturbing' bit turns out to have been talked about in public for at least 11 years

ie the divergence of tree growth and temperature after 1960.

Is there anything new that disturbs you in the emails and files? The Saudi Arabian negotiator is pretending this has suddenly undermined the entire science. But I just think that sounds like a ridiculous excuse for screwing the world for his own profit.

[Nederland]

46. It is new to me

If I had known about it 11 years ago, I would have been just as concerned.

[joshcryer]

61. So you read the 1998 nature paper describing the details?

If you did I don't think you'd be particularly concerned, since there are *reasons* for it.

[kristopher]

16. Horseshit. You ARE a denier; your history proves it.

You play the role of "concern" skeptic as if no one understands from the actual content of your arguments how that is your way to not get tombstoned.

Take this nebulous claim you are making now. You say are reluctant to cut and paste a small section of the file out of fear that you'll be labeled a denier??? When has something like that ever stopped you?

In this case instead of actually providing a meaningful argument, you argue by insinuation saying "I have read one that is VERY disturbing" and then providing a link to 371 pages of email.

If there is an argument buried in there that you find "VERY disturbing" then you could have laid it out. You didn't have to take it out of context to present the argument; you could have presented what you saw in proper context while posting a link to allow everyone to check the specific claim against the context. If your claim had merit, it would stand.

But if you just want to promote the idea that these emails somehow undermine the science, then you'd not do that; you'd do precisely as you did.

:puke:

[joshcryer]

37. A denier is someone who doesn't accept facts when they are presented.

Nederland is more of a doubter in that respect, because he has on several occasions accepted truth.

[kristopher]

44. Not really.

I suppose you could use that as a definition, however I prefer a different one that actually reflects the facts as we know them.
In this case "denier" is an opposition term. Those holding the beliefs that prompt the use of the term "denier" prefer to refer to themselves with the more palatable term "environmental skeptic" a term coined by Bjørn Lomborg in his book "The Skeptical Environmentalist".

This is well defined by Jacques in his paper "Rearguard of Modernity". I append the conclusion below:

Conclusion
Environmental skepticism presents itself as "speaking truth to power" through contrarian claims they say objectively "debunk" the myths of the environmental movement and environmental science. Yet, the analysis of this literature indicates that environmental skepticism is specifically issued from a conservative ideology supported by a coherent conservative countermovement opposed to environmentalism. This positions the bias of skeptic knowledge claims, and while environmentalists' claims clearly have their own bias, the claim that the skeptical project is generated from a sense of objectivity and value neutrality is flatly rejected as part of an attempt to subvert reflexive interrogation and the implied counter-hegemonic resistance this entails.

In particular, environmental skepticism is opposed to the establishment of global environmental concerns and those related to human sustainability. The substantive arguments of environmental skeptics are guided by a "deep anthropocentrism" which dissolves society from non-human nature. Importantly, a severed nature-human relationship effectively challenges the institutionalization of obligation to environmental changes and the people who are affected by these changes. Skepticism therefore preserves a conservative hope for limited government in the global market while it protects a consumptive elite against responsibilities for these systemic changes. In the process, environmental skepticism defends the structure of dominant social values in world politics such as the state system, expansive resource exploitation under world capitalism, and a hegemonic and consumptive North (and US in particular from where most skeptics hail) to flourish unmolested by the gadflies of the environmental movement.

Therefore, being overly concerned with the contrarian knowledge claims of environmental skeptics misses skepticism's more important political message about duty and the legitimacy of public environmental concern. Environmental skeptics, even if they are conclusively proven wrong on all counts, will succeed in—at least temporarily—guarding a falling hegemonic order if academia, the press, and government become overly interested in Darth Vader and Obi Wan dueling at the bay doors.101 I suspect that skeptics will be happy to continue to create this kind of conflict because it ultimately provides an indefinite defense of the dominant social norms and institutions. They do not need to win the debate about the state of the world to maintain this power and dominance. They only need to establish enough doubt about the environmental epistemic community having the debate to throw public action into doubt as well.

Kysar, in Ecology Law Quarterly, notes that both environmentalists, such as Worldwatch Institute, and skeptics like Lomborg are guilty of hyperbole which they use to focus attention on their own policy agenda through competing Litanies.102 These Litanies are, among other things, struggles over the ability to frame risk, and therefore regulation:
As a result, science becomes a contested space in which competitors
vie for the legal authority to impose costs on other parties, whether
in the form of regulatory compliance, or externalized physical and
environmental harms.103

However, Wildavsky, correctly I think, argues that risk is politically assessed by morality, and this makes the framing of public risk a civic exercise.104 From here, he argues that environmental policies need to be made with a preponderance of evidence, not evidence from probabilities. But skeptical ethics severely limit what counts and is available to create such a preponderance of evidence. Thus, contrary to what Lomborg argues, understanding what is to be done and prioritizing action is not just a simple matter of adding up the costs and benefits. The real struggle is over what can count as a cost or benefit or even whether such a conceptual tool is fair, appropriate, or relevant.105

Environmental skepticism is therefore a struggle over the core values and beliefs that frame who and what risks should count as important. But these are no ordinary historical risks. The state of the world debate centers on what core civic values should organize risk in society regarding human development and progress. Wildavsky's "culture" model is based on core fears and different cultural sets have different core fears of risks, and that environmentalists have a specific culture guided by "radical egalitarianism." I do not disagree that some environmentalism is deeply concerned about the fair distribution of ecological space and change.106 Turning this around though, it is just as plausible to frame skeptics as struggling for a "radical in-egalitarianism" within the core values that already organize world politics.

In conclusion, skepticism's influence in politics and culture presents a dramatic threat to human ability and political will to protect the critical life support systems found in ecological goods and services because they dismiss these systems as important. Many civilizations have actively decided, for one reason or another, to ignore the erosion of this essential relationship between society and non-human nature, only to collapse or find themselves at the mercy of a Dark Age that is defined by misery and suffering.107 Jared Diamond writes, Our world is interconnected and interdependent, like Easter Island's 11 clans. Today, we face the same problems—loss of forests, fisheries, biodiversity, fresh water, and topsoil—that dragged down past societies. But for the first time in world history, we are producing or transporting toxic materials, greenhouse gases, and alien species. All these environmental problems are time bombs. The world is now on an unsustainable course, and these problems will be resolved one way or another, pleasantly or unpleasantly, within the next 50 years.108

Yet, Lomborg shrugs off the matter of accountability to exactly these kinds of changes as "blame" and says our true priorities should be more along the lines of a low-fat diet instead of "focusing on pesticides, oxygen depletion, global warming, forests, wind power, biodiversity, etc.—issues which are more clearly someone else's fault."109

To some, the song of skepticism sounds like a sweet song, laden with the security and power of modernity. Diamond points out with optimism I share, when Easter Island collapsed, it did not have the benefit of knowing that other societies had collapsed by undermining ecological life support systems.

However, taking responsibility for global environmental integrity would be a positive step towards paradigmatic and r/evolutionary changes, one of which could be an incorporation of obligations to human societies commensurate with membership and impact within a larger international and ecological community.110 This directly challenges the way power and wealth are concentrated in the current world system, and environmental skeptics have organized as the rearguard for this system and its globalizing—but beleaguered - paradigm. To be sure, the fact that conservatives have felt the need to rally around the DSP indicates that the ecological position is gaining strength.

Skeptics however wish to postpone this change. Their placations sound good to the elite who are part of the dominant world order. From Diamond's lessons, this skeptical song is like lulling the boiling frog to sleep, ignoring that someone put the frog in the pot to begin with, and then telling the frog that things are, "in fact," getting better all the time.

[joshcryer]

62. I admit that it is obvious he is being informed by sites like WUWT and CA.

Denier probably is a more adequate term.

[muriel_volestrangler]

9. Yeah, the emails don't actually throw any doubt on any of the data

What they may show ('may', because a few emails say "I think it would be a good idea if ...", without confirming whether anything actually happened) is that some emails may have been deleted so that outsiders couldn't read them even if they put in a Freedom of Information request; or that one or two people wanted to exclude some people from being editors on journals, or from having some papers included in discussions, because they felt they were 'deniers'. So a few people in the UEA may have been behaving unprofessionally. There may be repercussions for them.

But the science still seems unchanged.

[guardian]

26. Here's some doubt...

Except that the "believers" will never listen to any heresy that threatens the religion of AGW. The following article talks about this file: Link: http://www.anenglishmanscastle.com/HARRY_READ_ME.txt


‘Botch after botch after botch’
Leaked ‘climategate’ documents show huge flaws in the backbone of climate change science
By LORRIE GOLDSTEIN
Toronto Sun
29th November 2009

…The file — 274 pages long describes the efforts of a climatologist/programmer at the Climatic Research Unit (CRU) of the University of East Anglia to update a huge statistical database (11,000 files) of important climate data between 2006 and 2009.

The computer coding, along with the programmer’s apparently unsuccessful efforts to complete the project, involve data that are the foundation of the study of climate change — recordings from hundreds of weather stations around the world of temperature and precipitation measurements from 1901 to 2006, sun/cloud computer simulations, and the like.
…
The CRU at East Anglia University is considered by many as the world’s leading climate research agency. Here’s how CBSNews.com…’s Declan McCullagh describes its enormous impact on policymakers:

“In global warming circles, the CRU wields outsize influence: It claims the world’s largest temperature data set, and its work and mathematical models were incorporated into the United Nations Intergovernmental Panel on Climate Change’s 2007 report. The report … is what the Environmental Protection Agency acknowledged it ‘relies on most heavily’ when concluding carbon dioxide emissions endanger public health and should be regulated.”

As you read the programmer’s comments below, remember, this is only a fraction of what he says.

- “But what are all those monthly files? DON’T KNOW, UNDOCUMENTED. Wherever I look, there are data files, no info about what they are other than their names. And that’s useless …” (Page 17)

- “It’s botch after botch after botch.” (18)

- “The biggest immediate problem was the loss of an hour’s edits to the program, when the network died … no explanation from anyone, I hope it’s not a return to last year’s troubles … This surely is the worst project I’ve ever attempted. Eeeek.” (31)

- “Oh, GOD, if I could start this project again and actually argue the case for junking the inherited program suite.” (37)

- “… this should all have been rewritten from scratch a year ago!” (45)

- “Am I the first person to attempt to get the CRU databases in working order?!!” (47)

- “As far as I can see, this renders the (weather) station counts totally meaningless.” (57)

- “COBAR AIRPORT AWS (data from an Australian weather station) cannot start in 1962, it didn’t open until 1993!” (71)

- “What the hell is supposed to happen here? Oh yeah — there is no ’supposed,’ I can make it up. So I have : – )” (98)

- “You can’t imagine what this has cost me — to actually allow the operator to assign false WMO (World Meteorological Organization) codes!! But what else is there in such situations? Especially when dealing with a ‘Master’ database of dubious provenance …” (98)

- “So with a somewhat cynical shrug, I added the nuclear option — to match every WMO possible, and turn the rest into new stations … In other words what CRU usually do. It will allow bad databases to pass unnoticed, and good databases to become bad …” (98-9)

- “OH F— THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done, I’m hitting yet another problem that’s based on the hopeless state of our databases.” (241).

- “This whole project is SUCH A MESS …” (266)

[muriel_volestrangler]

31. Again, as in #21, how does that affect the data?

It's a problem to do with identifying whether the data for stations from many different countries comes from the same positions or not. If the comment "the programmer’s apparently unsuccessful efforts to complete the project" is accurate, then this doesn't matter at all, because that means the data was not even used.

[joshcryer]

36. The programmer had two problems, 1) he couldn't read code and 2) he didn't like having to make...

...so many hacks for the code to represent the data fairly. Those hacks are not evidence of data falsification.

[tabatha]

42. The hacks are probably for the code

and not for the data.

I've had to hack code for a slightly different use - that was added later.
I hate hacks, and programmers are always frustrated by having to add code that they wish had been specified at the outset. The BIGGEST frustration of a programmer.

But is does not change the data.

[joshcryer]

32. Comments from an incompetent programmer.

Those comments are entirely unsurprising, he doesn't understand what is going on and is winging it.

[slackmaster]

2. I am appalled at the naivete of people who are supposed to be such learned scholars

I'm referring to the individuals at East Anglia who wrote the stupid emails without thinking about the possibility that they would be published out of context.

You have to work with the assumption that everything you record on a computing system is permanent. I understand idiot corporate execs not getting it, but professional scientists who have been working with the Internet for most of all of their academic careers surely ought to know better.

[villager]

5. well I'm sure the Saudi Arabian rep would want to push this meme as hard as possible

They, of course, want to see the addicts tapping their veins over and over again....

[tabatha]

6. Yes, because they are interested in selling oil.

And, yes, I am disappointed in the scientists that could do something so stupid.

[villager]

11. What, exactly, did scientists do -- specifically - that was "so stupid?"

I think a feedback loop release of methane gas would actually be far "stupider," from a species-wide perspective...

[Nederland]

13. Grafted different datasets together

When you graft two different datasets together and plot them on a graph as if they are one set, you are treading on thin ice. Yes, there are times when it is acceptable, but you need a good reason.

When you do it and don't explain what you did and why, that's a violation of good scientific practices.

When you do it and someone asks if you did it and you say you didn't, that's called LYING.

[kristopher]

17. Please specifically substantiate the claims you just made. nt

[joshcryer]

29. If you are talking about the well known decline which has been accounted for...

...it would be irresponsible to include it in the data, since it simply isn't reflective of reality.

[Nederland]

41. I'm talking about deleting the post-1960 data from the Briffa set

Are we talking about the same thing?

[joshcryer]

58. Yes, we are. So you don't understand why that is removed?

Do you want me to explain it to you?

[tabatha]

14. Write what they wrote in the emails.

Even Mann said it was unfortunate in the clip on the BBC site.

[villager]

18. And what did they write in the emails?

If you provide the whole context of the email, that'd be good, too...

[tabatha]

19. I listened to what Prof Michael Mann said on the BBC interview.

And Phil Jones wrote some unfortunate comments.

Jones:
"I can't see either of these papers in the ICC report. Kevin and I will keep them out somehow even if we have to redefine what the peer review literature is".

Mann:
"In no way reflects my views on the matter."
"In no way endorse what was said in that email."

In relation to the deletion of the emails:

Mann:
"I did not delete those mails".
"I can't justify the action".
"He made some poor decisions, frankly, and I think that is clear".

It is hugely damaging, because it gives false impressions, and AGW is the MOST important issue facing the planet and the whole world needs to be on board for solutions.

The theft was abhorrent, but the fall out is worse, from what I have seen on forums in many places. I have fought back as much as I can, but unfortunately some of the email remarks are not good, and put people on the defensive.

[joshcryer]

30. Yes, Jones should be fired, but it should be understood that FOI requests are bullshit.

What people do is send out massive FOI requests in a way that effectively harasses scientists. Scientists don't know how or what to do with the data, they don't know the laws. This is specifically why it was said to ask the information managers rather than just release the stuff.

FOI requests = conspiracy gold. If you don't release the data they claim you are hiding something. If you release it in a format that they cannot understand, they claim that there is obfuscation and the data is incorrect.

Jones made the wrong statement by being human and being fed up with FOI bullshit, not knowing how to handle it properly. In a private email it is not a big deal, however, since it has been made public, to maintain integrity they should get rid of him. As big of a blow to the AGW community that it might be.

[tabatha]

33. Yep, stuck between a rock and a hard place.

He has stepped down.
I am really hoping, probably unrealistically, that good can come of this.

[joshcryer]

35. Also, when you build a model, that model must be based on empirical evidence.

You can't just sit down and make stuff up, this is precisely why the programmer was frustrated, because he had to add in a whole crapload of hacks to get things to fit the observed results.

*NO WHERE* in any of this is *ANYONE ADMITTING TO FALSIFYING RESULTS*

[tabatha]

40. delete

[Nederland]

49. That is debateable

I have the entire FOI2009.zip file on my computer and have spent hours looking it over. Below is merely one of 29 examples that I have found so far where data is deliberately altered using what the author called a "VERY ARTIFICIAL CORRECTION". (There are an additional 60+ examples of code that comments describe as "hiding the decline", but I haven't figured out exactly what is happening in those examples). While I am a web developer with 18 years of experience who knows several languages, I cannot claim to know IDL very well. However, I know enough to see that line 10 defines an array of arbitrary numbers and line 57 applies them to the data. Beyond that, I do not have enough context to understand what is being done here and why. However, is it such a stretch to ask for an explanation? And at this point, an explanation of "we know what we are doing, trust us" is not going to cut it. When FOI requests have to be filed to pry data from these people's tightly gripped fingers, they have lost the benefit of the doubt as far as I am concerned. I would love for someone to come forward and explain what is going on here, because honestly I believe that in the end climate change is real and happening. However, to say that my faith has been shaken is an understatement.

Am I being unreasonable?

Here is the code:

00001: ;
00002: ; PLOTS 'ALL' REGION MXD timeseries from age banded and from hugershoff
00003: ; standardised datasets.
00004: ; Reads Harry's regional timeseries and outputs the 1600-1992 portion
00005: ; with missing values set appropriately. Uses mxd, and just the
00006: ; "all band" timeseries
00007: ;****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********
00008: ;
00009: yrloc=<1400,findgen(19)*5.+1904>
00010: valadj=<0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,[br />00011: 2.6,2.6,2.6]*0.75 ; fudge factor
00012: if n_elements(yrloc) ne n_elements(valadj) then message,'Oooops!'
00013: ;
00014: loadct,39
00015: def_1color,20,color='red'
00016: plot,<0,1>
00017: multi_plot,nrow=4,layout='large'
00018: if !d.name eq 'X' then begin
00019: window, ysize=800
00020: !p.font=-1
00021: endif else begin
00022: !p.font=0
00023: device,/helvetica,/bold,font_size=18
00024: endelse
00025: ;
00026: ; Get regional tree lists and rbar
00027: ;
00028: restore,filename='reglists.idlsave'
00029: harryfn=<'nwcan','wnam','cecan','nweur','sweur','nsib','csib','tib',$[br />00030: 'esib','allsites']
00031: ;
00032: rawdat=fltarr(4,2000)
00033: for i = nreg-1 , nreg-1 do begin
00034: fn='mxd.'+harryfn(i)+'.pa.mean.dat'
00035: print,fn
00036: openr,1,fn
00037: readf,1,rawdat
00038: close,1
00039: ;
00040: densadj=reform(rawdat(2:3,*))
00041: ml=where(densadj eq -99.999,nmiss)
00042: densadj(ml)=!values.f_nan
00043: ;
00044: x=reform(rawdat(0,*))
00045: kl=where((x ge 1400) and (x le 1992))
00046: x=x(kl)
00047: densall=densadj(1,kl) ; all bands
00048: densadj=densadj(0,kl) ; 2-6 bands
00049: ;
00050: ; Now normalise w.r.t. 1881-1960
00051: ;
00052: mknormal,densadj,x,refperiod=<1881,1960>,refmean=refmean,refsd=refsd
00053: mknormal,densall,x,refperiod=<1881,1960>,refmean=refmean,refsd=refsd
00054: ;
00055: ; APPLY ARTIFICIAL CORRECTION
00056: ;
00057: yearlyadj=interpol(valadj,yrloc,x)
00058: densall=densall+yearlyadj
00059: ;
00060: ; Now plot them
00061: ;
00062: filter_cru,20,tsin=densall,tslow=tslow,/nan
00063: cpl_barts,x,densall,title='Age-banded MXD from all sites',$
00064: xrange=<1399.5,1994.5>,xtitle='Year',/xstyle,$
00065: zeroline=tslow,yrange=<-7,3>
00066: oplot,x,tslow,thick=3
00067: oplot,!x.crange,<0.,0.>,linestyle=1
00068: ;
00069: endfor
00070: ;
00071: ; Restore the Hugershoff NHD1 (see Nature paper 2)
00072: ;
00073: xband=x
00074: restore,filename='../tree5/densadj_MEAN.idlsave'
00075: ; gets: x,densadj,n,neff
00076: ;
00077: ; Extract the post 1600 part
00078: ;
00079: kl=where(x ge 1400)
00080: x=x(kl)
00081: densadj=densadj(kl)
00082: ;
00083: ; APPLY ARTIFICIAL CORRECTION
00084: ;
00085: yearlyadj=interpol(valadj,yrloc,x)
00086: densadj=densadj+yearlyadj
00087: ;
00088: ; Now plot it too
00089: ;
00090: filter_cru,20,tsin=densadj,tslow=tshug,/nan
00091: cpl_barts,x,densadj,title='Hugershoff-standardised MXD from all sites',$
00092: xrange=<1399.5,1994.5>,xtitle='Year',/xstyle,$
00093: zeroline=tshug,yrange=<-7,3>,bar_color=20
00094: oplot,x,tshug,thick=3,color=20
00095: oplot,!x.crange,<0.,0.>,linestyle=1
00096: ;
00097: ; Now overplot their bidecadal components
00098: ;
00099: plot,xband,tslow,$
00100: xrange=<1399.5,1994.5>,xtitle='Year',/xstyle,$
00101: yrange=<-6,2>,thick=3,title='Low-pass (20-yr) filtered comparison'
00102: oplot,x,tshug,thick=3,color=20
00103: oplot,!x.crange,<0.,0.>,linestyle=1
00104: ;
00105: ; Now overplot their 50-yr components
00106: ;
00107: filter_cru,50,tsin=densadj,tslow=tshug,/nan
00108: filter_cru,50,tsin=densall,tslow=tslow,/nan
00109: plot,xband,tslow,$
00110: xrange=<1399.5,1994.5>,xtitle='Year',/xstyle,$
00111: yrange=<-6,2>,thick=3,title='Low-pass (50-yr) filtered comparison'
00112: oplot,x,tshug,thick=3,color=20
00113: oplot,!x.crange,<0.,0.>,linestyle=1
00114: ;
00115: ; Now compute the full, high and low pass correlations between the two
00116: ; series
00117: ;
00118: perst=1400.
00119: peren=1992.
00120: ;
00121: openw,1,'corr_age2hug.out'
00122: thalf=<10.,30.,50.,100.>
00123: ntry=n_elements(thalf)
00124: printf,1,'Correlations between timeseries'
00125: printf,1,'Age-banded vs. Hugershoff-standardised'
00126: printf,1,' Region Full <10 >10 >30 >50 >100'
00127: ;
00128: kla=where((xband ge perst) and (xband le peren))
00129: klh=where((x ge perst) and (x le peren))
00130: ts1=densadj(klh)
00131: ts2=densall(kla)
00132: ;
00133: r1=correlate(ts1,ts2)
00134: rall=fltarr(ntry)
00135: for i = 0 , ntry-1 do begin
00136: filter_cru,thalf(i),tsin=ts1,tslow=tslow1,tshigh=tshi1,/nan
00137: filter_cru,thalf(i),tsin=ts2,tslow=tslow2,tshigh=tshi2,/nan
00138: if i eq 0 then r2=correlate(tshi1,tshi2)
00139: rall(i)=correlate(tslow1,tslow2)
00140: endfor
00141: ;
00142: printf,1,'ALL SITES',r1,r2,rall,$
00143: format='(A11,2X,6F6.2)'
00144: ;
00145: printf,1,' '
00146: printf,1,'Correlations carried out over the period ',perst,peren
00147: ;
00148: close,1
00149: ;
00150: end

[tabatha]

52. Well, the comments inlcude

"fudge factor" and "APPLY ARTIFICIAL CORRECTION"

hmmmmm

[Nederland]

53. Yeah

Not exactly the kind of thing that gives you a warm fuzzy feeling is it?

[kristopher]

76. Have you ever worked with large data sets?

Especially those related to real world monitoring?

Especially those related to real world monitoring of data collected in, for example as much weather data is, 6 minute intervals over years, decades, and even centuries?

Because of breakdowns of equipment, communications storage etc, ANY data set is going to need work to make it something that can be processed. It is absolutely normal to insert a predetermined value into the areas where values are missing. These pre-determined values are usually arrived at by making a first cut analysis of the data and establishing norms based on the frequency of occurrences in the existing data.

You don't just make it up, you find the value that is least likely to disturb any pattern embedded in the data. Depending on the analysis you might use a single value for an entire year or you may fine tune it to season, or month, or time of day. The point is it is an inevitable part of the process and is routinely dealt with as I've as outlined.

When you publish it is normal to hold that data and the way you treated it in a file for a while - there is no established time. Offices like this produce truckloads of product and the emphasis is always on what you're doing now. If something isn't being questioned it inevitably joins a long queue of boxes and boxes of "old stuff" that disappears as time marches on.

That's reality.

[kristopher]

78. Listen to those crickets...

[joshcryer]

59. Gaven from RealClimate has already stated that those were test cases, and not included in any...

...publication whatsoever.

The actual files that were included are published on Biffra's website and that part is commented out. It is a test case.

[Nederland]

63. Link?

I'd be curious to see his full explanation because it begs the question, why? What exactly were they testing? Did someone wake up one morning and say: "Hey, I wonder what the plot would look like if we added some ARTIFICIAL data to it? Wouldn't that be interesting?" And did they say that about 29 different plots? And what about the 60+ other examples that I found that talked about hiding declines? Were those just "tests" too?

Sorry to be so sarcastic, I'm just really amazed at people's ability to take these people at their word given everything we've seen.

[joshcryer]

64. Here:

http://www.realclimate.org/index.php/archives/2009/11/something-is-x-in-the-state-of-denmark/comment-page-2/#comments

(Scroll down to comment #99)

Response: Fair enough. The same code is in briffa_sep98_d.pro but there it is commented out. - gavin

Response: Not at all. The 'correction' was calculated as the PC in an EOF decomposition of the divergence in the associated files (so it isn't arbitrary). I understand that this was done in order to test the sensitivity of certain calculations to the presence or absence of the post 1960 'divergence', but regardless of why it was done, it does not appear in any paper, nor does it impact any published data set. In no way can this be described as evidence for data manipulation in the sense you mean. This, like the junk that litters any researcher's hard drive, is just one of those calculations that didn't go anywhere or add anything particularly useful. That's the thing with stolen files - they don't come with context. - gavin

[Nederland]

66. Yeah

So a 'correction' that was calculated as the PC in an EOF decomposition of the divergence in the associated files, and that is not arbitrary, is commented in the code as an "ARTIFICIAL CORRECTION"?

Right. I totally believe that explanation.

[joshcryer]

67. Do you have evidence that the code in briffa_sep98_d.pro is in fact *not* commented out...

...and is in fact *used* in published data? Why would they do that?

If it was done then recreating the false output should be trivial.

[Nederland]

69. I agree

It should be a trivial task to take the original data and compare it to the published data and see if the relevant portions of the code were or were not commented out. Until that is done, I reserve judgment.

[joshcryer]

74. Since the code has been available for years, it would have happened by now.

I have no doubt people like Macintyre tried to find something. In fact, Macintyre was posting on ClimateAudit about "not having access to the data" for a year there, turns out that he admitted to having the data for some time, he admitted to lying about not having it, a lapse of judgment for sure (he probably forgot that he was leading people along all that time).

[Nihil]

77. I *almost* completely agree ...

> Sorry to be so sarcastic, I'm just really amazed at people's ability to
> take these people at their word given everything we've seen.

The difference is in the definition of "these people".

You are quite happy to take "Group A" at their word despite everything
that they have said & done over the years and are "amazed" that other
people are happy to take the word of "Group B".

I am quite happy to take "Group B" at their word despite everything
that they have said & done over the years and am amazed that other
people are happy to take the word of "Group A".

:shrug:

[Nederland]

45. Jones is already gone

He resigned Wednesday.

Michael Mann is now under investigation.

Your argument about FOI requests is ridiculous. All data, raw and processed, should be placed on a web server and available to anyone who wants it. It was collected using public money and should therefore be available to the public. The data is already on computers connected to the internet, so the only additional cost would be the cost of the increased bandwidth--a small price to pay for transparency. More importantly, doing that would actually mean that scientists would spend less time fulfilling requests for data. Right now they respond to requests on a case by case basis, and as the emails have proven, they often only give their data to people that agree with them, and lie about having it to people that don't agree with them. Don't believe me? Search for the email from Michael Mann in which he gives over some same data to a colleague only weeks after telling Steve McIntyre that he couldn't find it, along with a incriminating note to be careful not to let the data into the "wrong hands". This is how good science is done? I think not.

In sum, there is absolutely no reason all data shouldn't be available to anyone on demand. This is the 21st century for Christ sakes. It's not as if people have to print it out and FedEx it somewhere. The only reason scientists in the past have been 'harassed' with FOI requests is because they weren't being transparent about their data. We now know why.

[tabatha]

47. Do you have a link to Mann's investigation?

[Nederland]

50. Link

http://www.examiner.com/x-11224-Baltimore-Weather-Examiner~y2009m11d30-Climategate-Penn-State-Professor-Mann-under-investigation

Here is the statement from the University:

Professor Michael Mann is a highly regarded member of the Penn State faculty conducting research on climate change. Professor Mann’s research papers have been published in well respected peer-reviewed scientific journals. In November 2005, Representative Sherwood Boehlert (R-NY) requested that the National Academy of Sciences (NAS) convene a panel of independent experts to investigate Professor Mann’s seminal 1999 reconstruction of the global surface temperature over the past 1,000 years. The resulting 2006 report of the NAS panel (http://www.nap.edu/catalog.php?record_id=11676) concluded that Mann’s results were sound and has been subsequently supported by an array of evidence that includes additional large-scale surface temperature reconstructions.
In recent days a lengthy file of emails has been made public. Some of the questions raised through those emails may have been addressed already by the NAS investigation but others may not have been considered. The University is looking into this matter further, following a well defined policy used in such cases. No public discussion of the matter will occur while the University is reviewing the concerns that have been raised.

[joshcryer]

60. In an ideal world, yes. In a capitalist world, no. You pay for data, you don't release it.

And being attacked by FOI requests I know damn fucking well just how asinine they can be. People don't use them to "just get information." You couldn't send an FOI request to the MRO teams and tell the world that they're hiding data because the scientists on the teams want to have lead time to write papers.

[Nederland]

65. We are not talking about the "Capitalist" world Josh and you know it

We are talking about the public sector. Government funded research using tax payer dollars. Moreover, you are deliberately ignoring the fact that the system I proposed eliminates the need for FOI requests entirely. Scientists need lead times to write papers? Fine. Put the data on the server after you've published. What is Mann's excuse for in 2003 refusing to provide data for a paper written in 1998? What is his excuse for lying about having it?

[joshcryer]

68. Well, we know that most of CRUs data is available except for that which is paid for.

Whatever other reasons one doesn't want to release data, I cannot say, but it is not as if the people requesting the data are upstanding individuals who are genuinely curious and who genuinely want to use it for scientific purposes. You look at the Yaml data and how hopelessly stupid MacIntyre's analysis was. But because they are involved in public debunking what happens is that they *know* in advance that their own data is going to be misinterpreted and destroyed by hacks who run PR campaigns rather than actual scientific studies.

Note: it's simple enough to "debunk" his claims. Simply get your own data, do your own analysis, and then publish it. What happens when people look at the data from any given set? Oh well, they note the same trends.

[Nederland]

70. Not sure what you mean

Are you implying that some CRU data was paid for by private funding?

[joshcryer]

75. No, some of CRUs data, the very morsel that they aren't releasing, was bought from a company.

To get the data from that company they had to sign a non-disclosure. ie, capitalism. ;)

If you are aware of the software or data industry, if you get data, from say, a private contractor (in this case weather reporters), you sign a big piece of paper with lots of stuff on it that says "I will not share this data with anyone."

You obey these contracts because getting sued is the last thing you want to happen.

In software, and this is just an aside, not pertinent: there are actually no-compete contracts, that you must sign. They force you into an embargo, so you cannot write the same piece of code twice for two different entities until a period of time has elapsed. This is common practice in the software community, I myself have had to sign many.

[malakai2]

73. Don't know about Mann's reasons, but

The money that paid for the research was spent on the research. The researchers' time that was purchased with that money went into producing the research. Now there's no money or time left to put it on the server, or to maintain it on the server. If there's a lot of data and metadata, that gets to be non-trivial. So unless somebody's kicking out more money to admin techs, who's going to put that data up and maintain it? If it's not maintained, what good is it? Who benefits? It's not like the information, or access to it, is free.

[Dead_Parrot]

22. Because he was totally in favour of cuts 'til that happened.

Honest.

[joshcryer]

27. This is a bullshit copout for the failure that is COP15.

[Nederland]

56. Agreed

COP15 was going nowhere long before this came out.

[tabatha]

72. What would be neat in a thread like this

would be to be able to sort the columns, especially if it has comments from a number of days.
That way, one could see just the latest comments, or the comments by each person.