date: Thu Feb 21 10:15:59 2002 from: Keith Briffa subject: Re: paper in review to: j.burgess@uea.ac.uk Sharon Thank you for looking at this for us. I am in a difficult situation as Neil has (after a long delay) simply come back to me saying he can not do the review. On the basis of your review I am not happy to even offer a rewrite , at least without another qualified opinion. Can you suggest (in confidence) such a person? thanks again Keith At 09:47 AM 2/21/02 +0000, you wrote: Dear Dr Nicholson, Thank you for your prompt reply. I am very sorry about the incorrectly addressed email. It was entirely my mix- up, which I feel very embarassed about. Please accept my apologies and assurance that the email was intended to go to you, despite having the wrong name! I do hope you are recovered from your health problems and feeling better now. I will pass your reply on to Prof. Briffa, who will be in touch regarding your offer to complete the review. Yours sincerely, Julie Burgess Dear Keith, Please find enclosed a review of the Brooks/Hulme paper. I realize now why I initially procrastinated on this manuscript. It is one of those in which you know there are serious problems and should not be published, but cannot describe them easily. Conceptually, what they did sounded OK, but the method of approach does not appear to be well conceived. My reasons for drawing this conclusion are indicated in the attached sheet. I will also send a copy of this review and the review form by fax to the number indicated at the bottom of your letter. I really don't want to be identified as a reviewer. It became apparent that Neil Ward was also reviewer of this paper, when your secretary addressed a note to him to my email address. If I have misjudged the paper, I would gladly defer to his review, as he is much more of a statistician than I. At the very least, the authors need to better clarify, explain and justify what they did. Best regards, Sharon Review of "1000 years of rainfall variability in the Sahel: an evaluation of a long-term climate model simulation against observational data", by N. Brooks and M. Hulme. This paper deals with a clearly defined topic, but has several shortcomings that make it unacceptable for publication. These are 1) inadequacy of the model for studying the Sahel, 2) poor validation of the model, and 3) arbitrary and unjustified statistical analyses. I am also uncomfortable with their interpretation of results. Finally, and this is a minor issue, I think better literature could be cited. When several papers were available for citation about a certain point, the choice was generally a minor paper, with the most important papers being omitted. There is a tendency to cite "soft science" literature in places where more technical literature is appropriate. 1. Model and Model Validation: A validation attempt was made via comparison of statistics such as the mean, season cycle, and time spectra. However, in doing so the authors compared statistics for this century with those for a 1000-year model run. There is no reason to assume that these periods are statistically comparable. Indeed, the results suggest they are not. However, some aspects of the climate can be considered fairly stable, such as the summer rainy season. The model produces less than 50% of its rainfall in July, July, August, compared to about 80% in the "real world". It also shows many years in which rainfall approaches zero in the rainy season. The proper validation approach would have been to compare a 20th century simulation with the observed statistics. Further, it is important to show that the model can capture the mean spatial pattern and the real temporal variability of the observed data. This was not done. 2. The model results are extensively "massaged", using what appear to be arbitrarily chosen filters of 9 years, 25 years, 45 years, 96 years and 101 years. This is compared with unfiltered observational data. What is the justification of these particular filters, how do they affect the results. Is it appropriate to do statistical analyses, such as spectra, on the filtered series? 3. As a result of all of this statistical manipulation, it is difficult to follow what the authors do. It is even more difficult to judge their results and its statistical significance. This is particular problematic when a major results is correlations for thousands of grid points (Fig. 8.). If this work were to be revised, much more attention would have to be paid to the statistical approach and to validating the results. At the moment I have no confidence in any of the conclusions draw from this simulation. -- Professor Keith Briffa, Climatic Research Unit University of East Anglia Norwich, NR4 7TJ, U.K. Phone: +44-1603-593909 Fax: +44-1603-507784 [1]http://www.cru.uea.ac.uk/cru/people/briffa[2]/