cc: mann@virginia.edu
date: Mon, 19 Feb 2001 10:58:36 -0500
from: "Michael E. Mann"
subject: Re: verification results
to: Tim Osborn , Scott Rutherford
HI Tim,
Thanks for your response.
Yes, I was a bit (though not too) surprised at the similarity between the
seasonal results, but clearly warm season and calender mean are superior to
cold-season, and at least this much is to be expected. Much of the skill
comes from getting the baseline 19th century temperature right, and this is
somewhat similar between seasons.
The combined (multiproxy+MXD) results, if you saw those, are particularly
promising w/ a verification resolved variance of 0.83. Scott is currently
extending all 3 reconstructions (Multiproxy, mxd and combined) back to AD
1400. This should be very telling!
I agree that the verification scores, as currently computed, may be
slightly inflated by the fact that the infilled data have been used rather
than the raw data. I've given this some thought, and I propose that we s
do the verification 3 different ways. Tim, perhaps you can help out in
particular w/ number "3".
1) (multiproxy, MXD, and combined): Use all reconstructed gridpoints, using
the infilled data, the way Scott has done it. [ALREADY DONE]
2) (multiproxy, MXD, and combined): Only perform the MSE and total variance
sums used to compute the resolved variance over those points (in both space
and time) where the original instrumental data is available (ie, just skip
in the sum any location where there is no original value in the raw Jones
data, or a point in time for a given gridpoint where the original data is
missing).
3) (only makes sense for MXD, warm season) Perform the sums using only the
actual available Jones gridpoint data (warm season mean) for only those
temperature gridpoints which actually are MXD gridpoints themselves.
Compare the verification results for both the current reconstruction of the
corresponding temperature gridpoints based on the ridge
regression/expectation maximization algorithm, with the reconstructions of
those gridpoints directly from the MXD data.
Scott, perhaps you can let Tim know where he might most easily be of help
here. Let me know if there are any questions/comments about the above. Thanks.
I should have more than enough to talk about at EGS once we are done w/ the
additional runs/exercises described above. Scott and I will meet in C'Ville
in early March to try to get the results together. After EGS, we should
think about starting to write some of this up!
more soon,
mike
At 01:20 PM 2/19/01 +0000, Tim Osborn wrote:
>Dear Scott and Mike,
>
>sorry for the lack of response to your e-mail in January. I've been away
>quite a bit and then had three proposal deadlines come up last week, so was
>rather busy. The results look *very* promising, with good verification
>statistics etc. The multivariate resolved variance is only slightly higher
>for the warm season than for the cold season which is a little surprising,
>though I guess warm & cold seasons are more strongly correlated for the
>leading covariance structures that are most highly weighted than for the
>smaller scale variations that are suppressed. A couple of specific things:
>
>(1) MXD errors. A really useful measure that one can use here is the EPS
>(Expressed Population Signal), based on the mean correlation between the
>tree cores that make up each chronology (rbar), and the number of tree
>cores (n). Time-dependence can be included by putting the time-varying
>number of tree cores into the EPS equation. This would give some measure
>of the 387 chronologies. For the gridded case, some improvement would have
>to be estimated for the many grid boxes that contain >1 chronology, but
>that's not difficult to do. What is *difficult* is that I don't have
>either rbar or n for each chronology! All I have is, for each year and
>each chronology, the number of tree cores with data expressed as a fraction
>of the maximum number of tree cores for that chronology, i.e.,
>fraction(x,t) = n(x,t) / nmax(x). Since I don't know nmax for each
>chronology x, I can't work out n(x,t) from fraction(x,t). Nor do I know
>rbar(x). I've been wanting to get rbar(x) and n(x,t) ever since I started
>using this tree-ring data set (1997), but it's apparently not easily
>available. How important do you think it is for the current piece of work?
> If it is important then I could try to get it again, but if it's a fairly
>minor side issue then I won't.
>
>(2) From what I remember about the plan of action, the idea was to do these
>first runs using the already infilled instrumental data (though of course
>witheld for the pre-1901 period). I assume, therefore, that the timeseries
>and statistics shown are from this infilled & complete data set (I see now
>from the annual maps that this *is* the case because the "raw data" maps
>are complete). It would be useful to compare these MXD-based
>reconstructions with the instrumental data by sampling only those grid
>boxes in both the raw and reconstructed fields that originally contained
>real data in the instrumental data set. [I assume that the infilled
>instrumental data will be the same as the original instrumental data for
>these grid boxes.] The comparisons/statistics shown are useful, but only
>in addition to the comparison done against the (subsampled) original data,
>because (i) the message is harder to get across when using already infilled
>data; and (ii) the statistics *might* be slightly (artificially) improved
>when using infilled data. If you'd prefer me to do this, then let me know
>where the data sets are (plus format) and I can compare against my copy of
>the Jones instrumental data.
>
>So I guess that you should have enough to talk about at the EGS in Nice
>then Mike? Please let me know if you need any further input from me at
>this stage.
>
>Best regards to you both,
>
>Tim
>
>
>
>
>
>Dr Timothy J Osborn | phone: +44 1603 592089
>Senior Research Associate | fax: +44 1603 507784
>Climatic Research Unit | e-mail: t.osborn@uea.ac.uk
>School of Environmental Sciences | web-site:
>University of East Anglia __________| http://www.cru.uea.ac.uk/~timo/
>Norwich NR4 7TJ | sunclock:
>UK | http://www.cru.uea.ac.uk/~timo/sunclock.htm
>
>
>
_______________________________________________________________________
Professor Michael E. Mann
Department of Environmental Sciences, Clark Hall
University of Virginia
Charlottesville, VA 22903
_______________________________________________________________________
e-mail: mann@virginia.edu Phone: (804) 924-7770 FAX: (804) 982-2137
http://www.evsc.virginia.edu/faculty/people/mann.html