global-warming curve might be an artifact of bad PCA algorithm

[phantom power]

I certainly don't think this invalidates the overwhelming evidence of global warming, but it needs to be corrected.

And we can expect the usual suspects to bludgeon the environmental community with this any day now.

http://www.technologyreview.com/articles/04/10/wo_muller101504.asp?trk=top

[bemildred]

1. Good coders are hard to find.

And good, mathematically competent coders even harder.

[phantom power]

10. Amen to that.

[WMliberal]

2. great, we'll be hammered for bad science

because of the POSSIBILITY of a flaw in some obscure formula. meanwhile, creationism is "sound science."

[Viking12]

3. M&M are corporate hacks

This "study" has been out for a month or so, it is of interest of no one other than a few hand-waving flat-earthers. M&M have changed their basis for criticism of Mann so many times that it is obvious the have some obsessive compulsive axe to grind.

A reasonable critique of Mann's work has been recently published by Hans Von Storch.

Mann's results have been replicated by other studies employing different methods. The "hockey stick" IS NOT the lynch pin to understanding climate change.

See Tim Lambert's blog, "Deltoid" for some debunking of M&M.

[LongTomH]

5. References?

Could you add some links to your sources?

[Viking12]

9. Welcome to DU

As a corporate hack:

http://www.exxonsecrets.org/html/personfactsheet.php?id=1008
http://www.cei.org/dyn/view_expert.cfm?expert=180

As an incompetent researcher:

http://www.disinfopedia.org/wiki.phtml?title=Ross_McKitrick

It's worth noting a point which TechReview somehow seems to have missed, but which Nature's anonymous reviewer #1 did not: that even after M&M's alleged statistical error is corrected, the resulting graph "still has the upward trend towards the end of the series, so this trend is not just an artifact of MBH's PCA procedure." In other words, the handle of the hockey stick may look quite different, but even in M&M's reanalysis it still has a blade. And it is the blade, after all, that is the issue of current interest.

[phantom power]

11. On that subject, I'm a bit suspicious of their "monte carlo test"

Now, the crux of their argument is that if I run PCA on just a subset of the entire data, that will somehow enhance the curve near the endpoints of the dataset.

I'm having a hard time imagining why that would be true, in general. And furthermore, they say they achieved the same effect with a truly trend-less test set. But if the data are trendless, why would a PCA normalization induce a curve at the endpoints? There's no curve to begin with. There's nothing to enhance, whatsoever.

Kind of makes me want to try it myself, but I say that all the time and never get around to it.

On that subject, I wonder if the original temperature dataset is publicly available.

[Viking12]

13. It's not a temperature dataset per se

it's a temperature data set built from temperature proxies - ice core, tree ring, ocean sediment, etc..

The original dataset is available on line. If i get time this afternoon I'll hunt it down and link it.

[bemildred]

6. Does fit in nicely with the "There is still doubt" PR spin. nt

[Boomer]

4. So does this mean...

that the glaciers all over the world aren't really melting?

[hatrack]

7. Yes, and Arctic sea ice thickness hasn't dropped by 40%

They, uh, miscalibrated those big ice calipers, yeah, that's the ticket!!

[Dogmudgeon]

8. That's only one data model

And I believe it's not a new observation, but one that's been scrutinzed for a few years.

Every data model I've seen has come to similar results; in fact, there is a convergence of results from the models. And nearly every model that has had some flaw corrected trends even more strongly toward supporting the AGW (Anthropogenic Global Warming) hypothesis. A few exceptions to a rule do not invalidate a data set. The science behind it isn't like refuting evidence in a courtroom trial; it's a large number of checkable observations.

Climate change is no longer a theory. We know that it happens, and that it's taking place right now. Denying it is a sure route to disaster.

--bkl

[jpak]

12. Artifact my ass

The 19th and 20th century portion of the curve is based on rigorously vetted empirical data (i.e., direct temperature measurements).

It was NOT the product of a "computer model".

The pre-19th century data were derived from tree ring carbon isotope chronologies. These chronologies are also empirical - direct carbon isotope measurements of actual old-growth tree ring series. The derivation of past temperatures from these chronologies is based on the well known fractionation behavior of carbon isotopes with temperature.

They are not the product of a "computer model".

The author of this article is a hack and anyone that believes this nonsense is a fool.

Classic brownwash bullshit from a right-wing anti-environment ideologue.

[phantom power]

14. You're probably right. But I'll play devil's advocate for a moment

I work with algorithms like this for a living. Although this story doesn't match my intuitions about PCA, I've been doing this long enough to see my intuitions blown out of the water plenty of times.

And I've seen how people grab algorithms off the net, and don't check them. Somebody writes a piece of numeric analysis code with a subtle error in it, and it gets propagaged for years.

For all that people say they don't trust computers, the reality is that people often place a lot of unwarranted trust in computers (witness how gullible people have been regarding Diebold software). Even when you want to sanity-check an algorithm, it can be quite difficult to set up the test. After all, we use these things because they perform computations that are too large or complicated to do by hand.

So anyway, when somebody publishes an article about an algorithm error that influences our interpretation of data, I don't just dismiss it out of hand.

[Dogmudgeon]

15. Defenses against these kinds of errors?

A big deal is made out of Peer Review. The popular scientific press (especially the skep mags) are very keen on Peer Review being the gold standard of pre-publication scrutiny.

Less of a big deal is made out of meta-analysis, but I would think it has a much better chance of teasing out the errors.

I was a "Global Warming Skeptic" for a few years at one point. What brought me around was the fact that the data were supporting the hypotheses regardless of the data abstractions, and across large sets of data from disparate proxies.

As for sanity tests with software code, don't the programmers do any of that? Heck, I don't even trust my OWN code, let alone that of others. If I was coding for high-level scientific analysis rather than office task automation, I think I'd be fanatical about soliciting criticism.

But that's just me.

--bkl

[bemildred]

16. "don't the programmers do any of that?"

Some do, some do not, some don't really even know how.
Sturgeon's Law applies; and the fact that it is amazing what
you can get away with and how long you can get away with it.

http://www.jargon.net/jargonfile/s/SturgeonsLaw.html

[Nederland]

17. Not surprising

The hockey stick curve was always suspect because it produced a historical temperature graph that was inconsistent with every other method of measuring global temperatures. Its not surprising to learn that the cause was a simple math error.