cc: "Tim Osborn" , "Ross McKitrick" date: Wed, 12 Nov 2003 00:36:23 -0500 from: "Michael E. Mann" subject: Re: MBH98 to: "Steve McIntyre" Dear Mr. McIntyre, There seems to be some confusion on your part regarding the public posting of the MBH98 data. All of the data used by MBH98 have been available in plain ASCII format on this public ftp site [1]ftp://holocene.evsc.virginia.edu/pub/MBH98/ They have been available in the various clearly indicated sub-directories, back through at least summer 2002 according to the dates on those directories. This includes all 159 predictors used by MBH98 back to AD 1400 *and* all of the proxy data that go into those indicators. When I sent you the below email message on 4/9/2003, it was my expectation that you would go to that ftp site to get the individual data series in question. I was not party to the various emails you and Scott Rutherford exchanged regarding alternative versions of the dataset that he prepared, though I am told he offered you all of the proxy data, and you instead preferred a dataset of 112 proxy indicators (that is the number of indicators available back to 1820). With regard to the latest changes made by Scott on the ftp site, I believe this was to replace the incorrect spreadsheet version of the data that had been posted previously with a corrected version, so that people do not continue to download an incorrect version of the data set. To reiterate once last time, the original data that you requested before and now request again are all on the indicated ftp site, in the indicated directories, and have been there since at least 2002. I therefore trust you should have no problem acquiring the data you now seek. Mike Mann >Dear Mr. McIntyre, > >These data are available on an anonymous ftp site we have set up. I've forgotten the exact >location, but I've asked my Colleague Dr. Scott Rutherford if he can provide you with that >information. > >best regards, > >Mike Mann At 01:47 PM 4/8/2003 -0400, Steve McIntyre wrote: Dear Dr. Mann, I have been studying MBH98 and 99. I located datasets for the 13 series used in 99 at [2]ftp://eclogite.geo.umass.edu/pub/mann/ONLINE-PREPRINTS/Millennium/DATA/PROXIES/ (the convenience of the ftp: location being excellent) and was intereseted in locating similar information on the 112 proxies referred to in MBH98, as well as listing (the listing at [3]http://www.ngdc.noaa.gov/paleo/ei/data_supp.html is for 390 datasets, and I gather/presume that many of these listed datasets have been condensed into PCs, as mentioned in the paper itself. Thank you for your attention. Yours truly, Stephen McIntyre, Toronto, Canada At 11:39 PM 11/11/2003 -0500, Steve McIntyre wrote: November 11, 2003 Professor Michael E. Mann School of Earth Sciences University of Virginia Dear Professor Mann, We apologize for not sending you a copy of our recent paper (MM) in Energy and Environment for comment, as we understood from your email of September 25, 2003 that time constraints prevented you from considering our material. We notice that you seem to have subsequently changed your mind and hope that you will both be able to clarify some points for us and to rectify the public record on other points. 1) You have claimed that we used the wrong data and the wrong computational methodology. We would like to reconcile our results to actual data and methodology used in MBH98. We would therefore appreciate copies of the computer programs you actually used to read in data (the 159 data series referred to in your recent comments) and construct the temperature index shown in Nature (1998) (MBH98), either through email or, preferably through public FTP or web posting. 2) In some recent comments, you are reported as stating that we requested an Excel file and that you instead directed us to an FTP site for the MBH98 data. You are also reported as saying that despite having pointed us to the FTP site, you and your colleague took trouble to prepare an Excel spreadsheet, but inadvertently introduced some collation errors at that time. In fact, as you no doubt recall, we did not request an Excel spreadsheet, but specifically asked for an FTP location, which you were unable or unwilling to provide. Nor was an Excel spreadsheet ever supplied to us; instead we were given a text file, pcproxy.txt. Nor was this file created in April 2003. After we learned on October 29, 2003 that the pertinent data was reported to be located on your FTP site [4]ftp://holocene.evsc.virginia.edu/pub (and that we were being faulted for not getting it from there), we examined this site and found it contains the exact same file (pcproxy.txt) as the one we received, bearing a date of creation of August 8, 2002. On October 29, 2003, your FTP site also contained the file pcproxy.mat, a Matlab file, the header to which read: MATLAB 5.0 MAT-file, Platform: SOL2, Created on: Thu Aug 8 10:18:19 2002. Both files contain identical data to the file pcproxy.txt emailed to one of us (McIntyre) in April 2003, including all collation errors, fills and other problems identified in MM. It is therefore clear that the file pcproxy.txt as sent to us was not prepared in April 2003 in response to our requests, nor was it prepared as an Excel spreadsheet, but in fact it was prepared many months earlier with Matlab. It is also clear that, had we gone to your FTP site earlier, we would simply have found the same data collation as we received from Scott Rutherford. Would you please forthwith issue a statement withdrawing and correcting your earlier comments. 3) In reported comments, you also claimed that we overlooked the collation errors in pcproxy.txt and slid the incorrect data into our calculations, a statement which is untrue and made without a reasonable basis. In MM, we described numerous errors including, but not limited to, the collation errors, indicating quite obviously that we noticed the data problems. We then describe how we firewalled our data from the errors contained in the data you provided us, by re-collating tree ring proxy data from original sources and carrying out fresh principal component calculations. We request that you forthwith withdraw the claim that we deliberately used data we knew to be in error. 4) On November 8, 2003, when we re-visited your FTP site, we noticed the following changes since October 29, 2003: (1) the file pcproxy.mat had been deleted from your FTP site; (2) the file pcproxy.txt no longer was displayed under the /sdr directory, where it had previously been located, although it could still be retrieved through an exact call if one previously knew the exact file name; (3) without any notice, a new file named mbhfilled.mat prepared on November 4, 2003 had been inserted into the directory. Obviously, the files pcproxy.mat and pcproxy.txt are pertinent to the comments referred to above and we view the deletion of pcproxy.mat from the archival record under the current circumstances as unjustifiable. Would you please restore these files to your FTP site, together with an annotated text file documenting the dates of their deletion and restoration. 5) We note that the new file mbhfilled.mat is an array of dimension 381x2016. Could you state whether this file has any connection to MBH98, and, if so, please explain the purpose of this file, why it has been posted now and why it was not previously available at the FTP site. 6) Can you advise us whether the directory MBH98 has been a subdirectory within the folder pub since July 30, 2002 or whether it was transferred from another (possibly private) directory at a date after July 30, 2002? If the latter, could you advise on the date of such transfer. We have prepared a 3-part response to your reply to MM. The first, which we have released publicly, goes over some of the matters raised in points #2-#5 above. The second is undergoing review. It deals with additional issues of data quality and disclosure, resulting from inspection of your FTP site since October 29, 2003. The third part will consider the points made in your response, both in terms of data and methodology, and will attempt a careful reconciliation of our calculation methods, hence the necessity of our request in point #1. Thank you for your attention. Yours truly, Stephen McIntyre Ross McKitrick cc: Timothy Osborn ______________________________________________________________ Professor Michael E. Mann Department of Environmental Sciences, Clark Hall University of Virginia Charlottesville, VA 22903 _______________________________________________________________________ e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137 [5]http://www.evsc.virginia.edu/faculty/people/mann.shtml