date: Wed, 29 Oct 2003 08:38:54 +0000 from: Phil Jones subject: Fwd: Re: STOP THE PRESS! to: k.briffa@uea.ac.uk >Date: Tue, 28 Oct 2003 19:53:52 -0800 (PST) >From: Stephen H Schneider >To: "Michael E. Mann" >cc: Richard Kerr , Andy Revkin , > David Appell , > , > Mike MacCracken , > Michael Oppenheimer , > "Socci.Tony-epamail.epa.gov" , > , , > , Jonathan Overpeck , > Phil Jones , Scott Rutherford > , > Gabi Hegerl , tom crowley , > Tom Wigley , Tim Osborn , > Stefan Rahmstorf , > Gavin Schmidt , > Rob Dunbar , , > Ross Gelbspan , Ben Santer , > , >Subject: Re: STOP THE PRESS! > >Good, Mike, we scientists need to work hard to find fair and effective >boiled down statments that convey both urgency and uncertainty and explain >complexity with simple methphors--as long as we have back up details in >books, websites, papers etc. > Speaking of available data, I note the USA Today column said you did not >make your data available--please be sure that charge is clarified in your >summary of this affair. Cheers, Steve >PS This is what Schulz wrote about you and data availability--if false it >gives the USA Today the obligation to give you a rebuttal letter: >**************************** >In an interview, McKitrick said, "If a study is going to be the basis for >a major policy decision, then the original data must be disseminated and >the results have to be reproducible. That's why in our case we have posted >everything online and invite outside scrutiny." > >Mann never made his data available online — nor did many of the earlier >researchers whose data Mann relied upon for his research. That by itself >raises questions about the U.N. climate-change panel's scientific process. >********************************* >Hello all, > OK, back to me again. You also need to remind our audiences that IPCC is >not a research agency--IPCC does assessment of others work, and it is not >responsible to put data on websites etc. In fact, governments have >specifically told us NOT to do original research, just assesment of >research. It does not prohibit us from, as individual scientists, >publishing scientific research relevant to what IPCC would like to assess, >but then the IPCC process will subject such work to massive peer >review--with Review Editors watching. So there is no "scientific process" >at IPCC strictly speaking, just a scientific asessment process. This may >seem subtle, but the IPCC--a UN agency, with political baggage at least in >the US--is an assessment, not research, organization by design. Cheers, >Steve > > >On Tue, 28 Oct 2003, Michael E. Mann wrote: > > > Thanks Steve, > > > > I plan to work w/ the staffers to try boil this down to its most basic > > terms... > > > > Of course, the proxy data were available uncorrupted on our anonymous ftp > > site--the authors chose not to use that, and instead requested a > > spreadsheet version from my associated (Scott). Its not his fault that > > there were some problems with that particular file--the authors could > > have done numerous things to confirm the possible sources of the obvious > > problems w/ the file that they note in their 'paper'. > > > > This will be an important point to convey to folks. > > > > This is one of the worst examples yet (and we've had some good onces > > recently) of a disingenuous/deficient/absent peer review coupled with an > > irresponsible editor.. > > > > mike > > > > At 07:10 PM 10/28/2003 -0800, Stephen H Schneider wrote: > > Hello all. Interesting tale--why we have competent peer > > review at > > competent journals, and why professional courtesy is always > > to run > > heterodox results by the orthodox for private comments before > > going > > public--unless the motivation isn't science, but a big > > spalsh. Too bad for > > them--the wrong guys will belly-flop (couldn't have happened > > to a nicer > > bunch of prevaricators!). By the way, I give it a 50% > > (Bayesian priors) > > subjective probability they will accuse you of deliberately > > misleading > > them or deliberately preventing replication by "independent" > > scientists > > and the only reason they did this was to smoke you out. From > > them, expect > > anything. Can you explain this to Senator McCain's folks so > > they > > understand the complexities and professional courtesy/peer > > review issues? > > This stuff is not very sound bite friendly and needs some > > prethinking to > > put it simply and clearly so it can be useful in the debate > > held by > > non-scientist debaters. Good luck, Steve > > > > On Tue, 28 Oct 2003, Michael E. Mann wrote: > > > > > Dear Friends and Colleagues, > > > > > > I've got a story with a very happy ending to tell. I't > > will take a bit > > > of patience to get through the details of the story, but I > > think its > > > worth it. > > > > > > By the way, please keep this information confidential for > > about the next > > > day or so. > > > > > > OK, well its about 48 hours since I first had the chance to > > review the > > > E&E paper by M&M. Haven't had a lot of sleep, but I have > > had a lot of > > > coffee, and my wife Lorraine has been kind enough to allow > > me to stay > > > perpetually glued to the terminal. So what has this effort > > produced? > > > > > > Well, upon first looking at what the authors had done, I > > realized that > > > they had used the wrong CRU surface temperature dataset > > (post 1995 > > > version) to calculate the standard deviations for use in > > un-normalizing > > > the Mann et al (1998) EOF patterns. Their normalization > > factors were > > > based on Phil's older dataset. The clues to them should > > have been that a) > > > our data set goes back to 1854 and theirs only back to 1856 > > and (b) why > > > are 4 of the 1082 Mann et al (1998) gridpoints missing?? > > [its because > > > the reference periods are different in the two datasets, > > which leads to a > > > different spatial pattern of missing values]. So they had > > used the wrong > > > temperature standard deviations to un-normalize our EOFs in > > the process > > > of forming the surface temperature reconstruction. And I > > thought to > > > myself, hmm--this could lead to some minor problems, but I > > don't see how > > > they get this divergence from the Mann et al (1998) > > estimate that > > > increases so much back in time, and becomes huge before > > 1500 or so. That > > > can't be it, can it? > > > > > > Then I uncovered that they had used standard deviations of > > the raw > > > gridpoint temperature series to un-normalize the EOFs, > > while we had > > > normalized the data by the detrended standard deviations. > > Either > > > convention can be justified, but you can't mix and > > match--which is what > > > they effectively did by adopting our EOFs and PCs, and > > using their > > > standard deviations. And I thought, hmm--this could > > certainly lead to an > > > artificial inflation of the variance in the reconstruction > > in general, > > > and this could give an interesting spatial pattern of bias > > as well (which > > > might have an interesting influence on the areally-weighted > > hemispheric > > > mean). But I thought, hmm, this can't really lead to that > > tremendous > > > divergence before 1500 that the authors find. I was still > > scratching my > > > head a bit at this point. > > > > > > Then I read about the various transcription errors, values > > being shifted, > > > etc. that the authors describe as existing in the dataset. > > And I thought, > > > hmm, that sounds like an excel spread sheet problem, not a > > problem w/ the > > > MBH98 proxy data set. It started to occur to me at this > > point that there > > > might be some problems w/ the excel spreadsheet data that > > my colleague > > > Scott Rutherford had kindly provided the authors at their > > request. But > > > these problems sounded pretty minor from the authors' > > description, and > > > the authors described a procedure to try to fix any > > obvious > > > transcription errors, shifted cell values, etc. So I > > thought, hmm, they > > > might not have fixed things perfectly, and that could also > > lead to some > > > problems. But I still don't see how they get that huge > > divergence back in > > > time from this sort of error... > > > > > > Still scratching my head at this point...Then finally this > > afternoon, > > > some clues. After looking at their on-line description one > > more time, I > > > became disturbed at something I read. The data matrix > > they're using has > > > 112 columns! Well that can't be right! That's can't > > constitute the Mann > > > et al (1998) dataset. There are considerably more than that > > number of > > > independent proxy indicators necessary to reproduce the > > stepwise Mann et > > > al reconstruction. Something is amiss! > > > > > > Well, 112 is the number of proxy indicators used back to > > 1820. But some > > > of these indicators are principal components of regional > > sub-networks > > > (e.g. the Western U.S. ITRDB tree-ring data) to make the > > dataset more > > > managable in size, and those principal components (PCs) are > > unique to the > > > time interval analyzed. So there is some set of PC series > > for the > > > 1820-1980 period. Farther back in time, say, back to 1650 > > there are fewer > > > data series the regional sub-networks. So we recalculate a > > completely > > > different EOF/PC basis set for that period, and that > > constitutes an > > > additional, unique set of proxy indicators that are > > appropriate for a > > > reconstruction of the 1650-1980 period. PC #1 from one > > interval is not > > > equivalent to PC#1 from a different interval. This turns > > out to be the > > > essential detail. A reconstruction back to 1820 > > calibrated against the > > > 20th century needs to make use of the unique set of proxy > > PCs available > > > for the 1820-1980 period. A reconstruction back to 1650 > > calibrated > > > against the 20th century needs to make use of the > > independent (smaller) > > > set of PC series available for the 1650-1980 period, and so > > on, back to > > > 1400. > > > > > > So there have to be significantly more than 112 series > > available to > > > perform the iterative,stepwise reconstruction approach of > > Mann et al > > > (1998), because each sub interval actually has a unique set > > of PC series > > > representations of various proxy sub-networks. Then it > > started to hit > > > me. The PC#1 series calculated for networks of similar > > size (say, the > > > network available back to 1820 and that available back to > > 1750) should be > > > similar. But as the sub-network gets sparser back in time, > > the PC#1 > > > series will resemble less and less the PC#1 series of the > > denser networks > > > available at later times. PC#1 of the western ITRDB > > tree-ring calculated > > > for the 1400-1980 period will bear almost no resemblance > > to the PC#1 > > > series of the western N.Amer ITRDB data calculated for the > > 1820-1980 > > > period during their interval (1820-1980) of mutual overlap. > > > > > > Then it really hit me. What--just what--if the proxy data > > had been > > > pigeonholed into a 112 column matrix by the following > > (completely > > > inappropriate!) procedure: What if it had been decided that > > there would > > > only be 1 column for "PC #1 of the Western ITRDB tree ring > > data", even > > > though that PC reflects something completely different over > > each > > > sub-interval. Well, that can't be done in a reasonable way. > > But it can be > > > done in an *unreasonable* way: by successively overprinting > > the data in > > > that column as one stores the PCs from later and later > > intervals. So a > > > given column would reflect PC#1 of the 1400-1980 data from > > 1400-1450, > > > PC#1 of the 1450-1980 from 1450-1500, PC#1 of the 1500-1980 > > data for > > > 1500-1650, PC#1 of the 1650-1980 data for 1650-1750, etc. > > and so on. In > > > this process, the information necessary to calibrate the > > early PCs would > > > be obliterated with each successive overprint. The > > resulting 'series' > > > corresponding to that column of the data matrix, an amalgam > > of > > > increasingly unrelated information down the column, would > > be completely > > > useless for calibration of the earlier data. A > > reconstruction back to AD > > > 1400 would be reconstructing the PC#1 of the 1400-1450 > > interval based on > > > calibration against the almost entirely unrelated PC#1 of > > the 1820-1980 > > > interval. The reconstruction of the earliest centuries > > would be based on > > > a completely spurious calibration of an unrelated PC of a > > much later > > > proxy sub network. And I thought, gee, what if Scott (sorry > > Scott), had > > > *happened* to do this in preparing the excel file that the > > authors used. > > > Well it would mean that, progressively in earlier > > centuries, one would > > > be reconstructing an apple, based on calibration against > > an orange. It > > > would yield completely meaningless results more than a few > > centuries ago. > > > And then came the true epiphany--ahhh, this could lead to > > the kind of > > > result the authors produced. In fact, it seemed to me that > > this would > > > almost *insure* the result that the authors get--an > > increasing divergence > > > back in time, and total nonsense prior to 1500 or so. At > > this point, I > > > knew that's what Scott must have done. But I had to > > confirm. > > > > > > I simply had to contact Scott, and ask him: Scott, when you > > prepared that > > > excel file for these guys, you don't suppose by any chance > > that you might > > > have.... > > > > > > And, well, I think you know the answer. > > > > > > So the proxy data back to AD 1820 used by the authors may > > by-in-large be > > > correct (aside from the apparent transcription/cell shift > > errors which > > > they purport to have caught, and fixed, anyway). The data > > become > > > progressively corrupted in earlier centuries. By the time > > one goes back > > > to AD 1400, the 1400-1980 data series are, in many cases, > > entirely > > > meaningless combinations of early and late information, and > > have no > > > relation to the actual proxy series used by Mann et al > > (1998). > > > > > > And so, the authors results are wrong/meaningless/useless. > > The mistake > > > made insures, especially, that the estimates during the > > 15th and 16th > > > centuries are entirely spurious. > > > > > > So whose fault is this? Well, the full, raw ascii proxy > > data set has been > > > available on our anonymous ftp site > > > ftp://holocene.evsc.virginia.edu/pub/MBH98/ > > > and the authors were informed of this in email > > correspondence. But they > > > specifically requested that the data be provided to them in > > excel format. > > > And Scott prepared it for them in that format, in good > > faith--but > > > overlooked the fact that all of the required information > > couldn't > > > possibly be fit into a 112 column format. So the file Scott > > produced was > > > a complete corruption of the actual Mann et al proxy data > > set, and > > > essentially useless, transcription errors, etc. aside. The > > authors had > > > full access to the uncorrupted data set. We therefore take > > no > > > reasonability for their use of corrupted data. > > > > > > One would have thought that the authors might have tried to > > reconcile > > > their completely inconsistent result prior to publication. > > One might have > > > thought that it would at least occur to them as odd that > > the Mann et al > > > (1998) reconstruction is remarkably similar to entirely > > independent > > > estimates, for example, by Crowley and Lowery (2000). Could > > both have > > > made the same supposed mistake, even though the data and > > method are > > > entirely unrelated. Or might M&M have made a mistake? Just > > possibly, > > > perhaps??? > > > > > > Of course, a legitimate peer-review process would have > > caught this > > > problem. In fact, in about 48 hours if I (or probably, many > > of my > > > colleagues) had been given the opportunity to review the > > paper. But that > > > isn't quite the way things work at "E&E" I guess. I guess > > there may just > > > be some corruption of scientific objectivity when a journal > > editor seems > > > more interested in politics than science. > > > > > > The long and short of this. I think it is morally > > incumbent upon E&E to > > > publish a full retraction of the M&M article immediately. > > Its unlikely > > > that they'll do this, but its reasonable to assert that it > > would be > > > irresponsible for them not to if the issue arises. > > > > > > I think that's the end of the story. Please, again, keep > > this information > > > under wraps for next day or two. Then, by all means, feel > > free to > > > disseminate this information as widely as you like... > > > > > > Mike > > > > > > > > ______________________________________________________________ > > > Professor Michael E. Mann > > > Department of Environmental Sciences, Clark Hall > > > University of Virginia > > > Charlottesville, VA 22903 > > > > > > _______________________________________________________________________ > > > e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: > > (434) 982-2137 > > > > > http://www.evsc.virginia.edu/faculty/people/mann.shtml > > > > > > > ------ > > Stephen H. Schneider, Professor > > Dept. of Biological Sciences > > Stanford University > > Stanford, CA 94305-5020 U.S.A. > > > > Tel: (650)725-9978 > > Fax: (650)725-4387 > > shs@stanford.edu > > > > ______________________________________________________________ > > Professor Michael E. Mann > > Department of Environmental Sciences, Clark Hall > > University of Virginia > > Charlottesville, VA 22903 > > _______________________________________________________________________ > > e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137 > > http://www.evsc.virginia.edu/faculty/people/mann.shtml > > > >------ >Stephen H. Schneider, Professor >Dept. of Biological Sciences >Stanford University >Stanford, CA 94305-5020 U.S.A. > >Tel: (650)725-9978 >Fax: (650)725-4387 >shs@stanford.edu Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------