date: Tue, 14 Oct 2003 17:27:24 -0400
from: "Michael E. Mann"
subject: Fwd: Re: smoothing
to: Tom Wigley , Kevin Trenberth , Keith Briffa , Phil Jones , ckfolland@meto.gov.uk, tkarl@ncdc.noaa.gov, jto@u.arizona.edu, mann@virginia.edu
Sorry--one more error. The MSE values for "minimum norm" and "minimum roughness" are
switched in the figure legend. Obviously the former is a better fit...
mike
Date: Tue, 14 Oct 2003 17:08:49 -0400
To: Tom Wigley , Kevin Trenberth , Keith Briffa
, Phil Jones , ckfolland@meto.gov.uk,
tkarl@ncdc.noaa.gov, jto@u.arizona.edu, mann@virginia.edu
From: "Michael E. Mann"
Subject: Re: smoothing
Bcc: Scott Rutherford
correction '1)' should read:
'1) minimum norm: sets padded values equal to mean of available data beyond the
available data (often the default constraint in smoothing routines)'
sorry for the confusion,
mike
At 05:05 PM 10/14/2003 -0400, Michael E. Mann wrote:
Dear All,
To those I thought might be interested, I've provided an example for discussion of
smoothing conventions. Its based on a simple matlab script which I've written (and
attached) that uses any one of 3 possible boundary constraints [minimum norm, minimum
slope, and minimum roughness] on the 'late' end of a time series (it uses the default
'minimum norm' constraint on the 'early' end of the series). Warming: you needs some
matlab toolboxes for this to run...
The routines uses a simple butterworth lowpass filter, and applies the 3 lowest order
constraints in the following way:
1) minimum norm: sets mean equal to zero beyond the available data (often the default
constraint in smoothing routines)
2) minimum slope: reflects the data in x (but not y) after the last available data
point. This tends to impose a local minimum or maximum at the edge of the data.
3) minimum roughness: reflects the data in both x and y (the latter w.r.t. to the y
value of the last available data point) after the last available data point. This tends
to impose a point of inflection at the edge of the data---this is most likely to
preserve a trend late in the series and is mathematically similar, though not identical,
to the more ad hoc approach of padding the series with a continuation of the trend over
the past 1/2 filter width.
The routine returns the mean square error of the smooth with respect to the raw data. It
is reasonable to argue that the minimum mse solution is the preferable one. In the
particular example I have chosen (attached), a 40 year lowpass filtering of the CRU NH
annual mean series 1856-2003, the preference is indicated for the "minimum roughness"
solution as indicated in the plot (though the minimum slope solution is a close 2nd)...
By the way, you may notice that the smooth is effected beyond a single filter width of
the boundary. That's because of spectral leakage, which is unavoidable (though minimized
by e.g. multiple-taper methods).
I'm hoping this provides some food for thought/discussion, esp. for purposes of IPCC...
mike
______________________________________________________________
Professor Michael E. Mann
Department of Environmental Sciences, Clark Hall
University of Virginia
Charlottesville, VA 22903
_______________________________________________________________________
e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137
[1]http://www.evsc.virginia.edu/faculty/people/mann.shtml
______________________________________________________________
Professor Michael E. Mann
Department of Environmental Sciences, Clark Hall
University of Virginia
Charlottesville, VA 22903
_______________________________________________________________________
e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137
[2]http://www.evsc.virginia.edu/faculty/people/mann.shtml
______________________________________________________________
Professor Michael E. Mann
Department of Environmental Sciences, Clark Hall
University of Virginia
Charlottesville, VA 22903
_______________________________________________________________________
e-mail: mann@virginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137
[3]http://www.evsc.virginia.edu/faculty/people/mann.shtml