date: Tue Dec 4 16:10:42 2001 from: Tim Osborn subject: Re: Review of JCL3759 Zorita to: Brenda Morris At 02:59 30/11/01, you wrote: Sorry to bother you but I need to know how your review of the Zorita manuscript is coming along. When will you be able to send us your review. Dr. Mann is waiting for your review so he can make a decision on the paper. Dear Brenda, Sorry for the delay - one problem with an all-electronic system is that, with no paper version to put on a pile on my desk, the e-mail can get rapidly buried in my in-box and once it's scrolled off the screen it's easy to forget that I had even agreed to do a review! Please pass my apologies on to Mike. When I did come to review it, I found it to be one of those tricky cases on the borderline between recommending publication or not. My review is appended below. Best regards Tim ----------------------- Review of manuscript JCL3759 Zorita et al.: "Statistical temperature reconstruction in a 1000-year-long control climate simulation: an excercise with Mann's et al. (1998) method" The research presented in this manuscript uses a climate model simulation, where the complete coverage of global temperature is known, to test various aspects of the estimation of global temperatures from values at a limited sample of locations. Two aspects are considered: (i) the dependence of reconstruction skill on the size of the limited sample; and (ii) the skill of the reconstruction at time scales (apparently) longer than those resolved within the calibration period of the empirical estimation procedure. The latter aspect is not, in practise, tested properly, since the analysis (see Figure 6) groups together all time scales longer than about 40 years (frequency 0.025); given that a 201-year period was used for calibration, it remains unclear how skilfully the inter-century variability is being captured. Since the first aspect was not considered very exhaustively (only a limited set of possible samples was used, with no investigation of why values from certain locations might be more useful), I am not convinced that this manuscript tells us very much. There is nothing major at fault with the work, it simply fails to advance the field sufficiently to warrant publication at this stage. I strongly encourage the approach taken, however, and recommend that the work be extended and a more comprehensive set of results would then certainly be suitable for publication. To aid in this process I also give some comments that are pertinent to the present manuscript. (1) Title: not only has a spelling mistake ("exercise" not "excercise"), but doesn't really sound right. (Indeed there are some minor spelling or grammatical errors elsewhere in the manuscript). (2) Abstract, lines 16-17: I'm not sure that the sample locations match tree-ring chronology locations: the main text describes them as selected from the Mann et al. (1998) network, which certainly included more than just tree-ring series. (3) Page 1, lines 6-8: again there seems to a misconception that Mann et al. used only tree-ring proxies. I'd suggest modifying this sentence to become: "...such as tree-ring chronologies (Cook et al., 1998; Briffa et al., 2001), ice-core snow deposition rates (Appenzeller et al., 1998), documentary evidence (Pfister et al., 1999), or a combination of multiple proxy types (Mann et al., 1998)." (4) Page 1, bottom: it would be useful to refer to studies that have utilised model simulations for assessing the suitability of the incomplete instrumental data network at measuring global temperature changes, since that is a similar (though perhaps less extreme) application. For example: Madden et al. (1993). (5) Page 3, lines 10-11: "external forcing" is better than "external sources". (6) Pages 3, lines 16-18: should cite Jones et al. (1997) here, because they compared the degrees of freedom in climate model output with that in the observed record. (7) Page 4, line 11: use "flux adjustment" not "flux correction", since the fluxes are not necessarily correct after application of the procedure. (8) Page 5, line 1: cite Mann et al. (1999) here, since they found that fewer proxies could indeed only reconstruct the leading PC. (9) Page 5, lines 6-7: cite Jones et al. (1998; already in manuscript reference list) here, because they present local climate skill of various long climate proxies and support the point being made. (10) Page 5, lines 10-20: need to be a bit more precise about the statistical approach. Did the EOF analysis use the correlation or covariance matrix? Was any rotation of EOFs used? Did you use annual mean temperature at each location? State, for completeness, that Mann et al. (1998) used monthly temperature anomalies for the EOF analysis (with all months of the year pooled together), and then averaged the monthly PC time series into annual-mean time series. This appears different to what you are doing. No need to modify what you do, just state that this difference exists. How many temperature EOF/PCs were retained for the reconstruction and did this vary with number of proxy records (and if so, how did it vary)? Also state that the 201-year calibration period far exceeds the 79-year period used by Mann et al. (1998), who used 1902-1980 I think. Why did you use a longer period? (11) What is 'n' in equation 2? (12) Page 6, lines 5-7: clarify that it is not just equation 2 that is "solved", but the whole set of equations given by formula 2, for k = 1 to the number of "proxies" being used. Also it is not really being "solved": rather a best-fit is being found to the whole set of over-determined equations. (13) Should the denominator of equation 3 be Tsim not T? (14) Page 6, line 25, and elsewhere: ECHO and ECHO-G aren't defined anywhere. (15) Page 7, lines 3-6: comment further on this "Arctic Oscillation" mode, since the temperature pattern doesn't show the expected cold-ocean, warm-land pattern expected. Is this because it is annual rather than cold-season? (16) Page 7, lines 17-19: cite Barnett (1999), who computed EOFs of the temperature fields from many different climate models. (17) Page 8, last paragraph, and figure 3: how many EOFs were used, just the 4 shown in Figure 1? The explained variance never attains values greater than 1! If you mean zero, shown by the dashed line, then this is achieved for about 35 or more proxies, not just > 50. Figure 3 caption should say "Root mean squared error" not "Mean squared error". Does Figure 3 show the results from the calibration period, the verification period (i.e., everything except the calibration period), or the full period? It's not clear. I assume it is the verification period, because the explained variance shouldn't be negative during the calibration period. Is this the spatially-resolved temperature variability, or the spatial-mean (i.e., global-mean) temperature? It would be good to see both, plus maybe maps of the local variance explained. Do the results indicate that fewer than 30 proxies (even perfect proxies, as here) give no useful information (i.e. negative explained variance) at all? This would have implications for other studies, and possibly for the Mann et al. (1998) method, since Jones et al. (1998) find a small number of series to give useful information. Perhaps the number of EOFs wasn't modified with number of proxies, resulting in sub-optimal performance. (18) Page 10, bottom, and page 11, top: I do not find the explanation of the Antarctic proxies' lack of use very convincing. Did you retain extra EOFs when using the 3 extra proxies? Do any of the retained EOFs actually contain the pattern that is causing reconstruction difficulties? If not, then you cannot expect to reconstruct it, even with good proxy coverage. As for the weighting argument: well if there was an EOF capturing this Antarctic pattern then it may be unrelated to all the other proxies and hence the ak coefficients in equation 2 will be small - except for those involving the Antarctic proxies, which can then dominate even if small in number. (19) Page 12, top: Again cite Jones et al. (1997) who looked at degrees of freedom versus time scale issues. (20) Page 13, bottom: this final conclusion is not based on quantitative results, but on speculation - please qualify this. (21) Figure 4 caption: again, need "Root mean squared error" not just "Mean squared error". References Jones et al. (1997) J. Climate 10, 2548-2568. Briffa et al. (2001) J. Geophys. Res. 106, 2929-2941 Mann et al. (1999) Geophys. Res. Lett. 26, 759-762. Barnett (1999) J. Climate 12, 511-518. Madden et al. (1993) J. Climate 6, 1057-1066. ------------------------