cc: Kate Willett , Dick Dee date: Mon Dec 22 15:54:06 2008 from: Phil Jones subject: Re: CRUTEM3v and HadCRUH comparisons with ERA-40 and ERA-Interim to: Adrian.Simmons@ecmwf.int, "peter.thorne" Adrian, The grid box count plots you've sent clearly indicate to me that we will need to do something to CRUTEM3v. I knew we had a problem going from CRUTEM3 to CRUTEM3v, which affected the Antarctic. This influences SH extratropics as well. A couple of the other regions have seasonal cycles in the counts, which they shouldn't. John Kennedy at MOHC now has a corrected file (from CRU) that won't have this problem, but I don't know when it will be implemented. It is a long story as to why it happened. A simple way for you to get over it is to use CRUTEM3 - with no v. All this won't make any difference to your earlier plots except for the Antarctic, and then only possibly. I'll get on to John in the New Year to rerun CRUTEM3v. Cheers Phil At 12:25 22/12/2008, Adrian Simmons wrote: Thanks for the comments, Peter. A few quick responses: (i) I'll check different tolerances for including CRUTEM3v data, but doubt it will help coverage much. Actual numbers can be seen in the attached plots. They shift mostly only on the decadal timescale, consistent with the 10-year book explanation given by Phil earlier. (ii) For temperature, I could show the CRUTEM2v time series plus differences. To see anything much though I would have to use a different scale for differences, and that may detract from the main point, which is that the time series are very similar indeed. I would argue that if you can't see the differences, they can't be very significant. I've got a couple of tables to quantify differences. I'll try out the alternative nevertheless. This approach won't work for humidity, however, as none of the time series spans the full period. (iii) If you could extend HadCRUH over land to the end of 2007, albeit in provisional form, I would certainly be interested. (iv) I wondered myself about flipping the colour for the humidity plots. But q increases as T increases. So I thought to use red colours both for larger T (conventional) and larger q (unconventional, I know), so that the correlation stands out. I guess it's a matter of taste. If we stick with the current scheme, I would include a sentence explaining why blue means dry. (v) I had a longish chat with June Wang when I visited NCAR in October. She was then about to put in a proposal jointly with Aiguo Dai (who I've not met) and Leo for funding for an attempt at homogenising radiosonde q - something we're very interested in for future reanalysis. I've also just looked at a paper of hers on the dry RS80 bias, as this may play a part in the RH shift over Europe either side of ~1990 (generally, radiosondes dry our boundary-layer analysis whilst SYNOPs moisten it). A visit from them in the summer would be very welcome. Best regards Adrian peter.thorne wrote: > Dear Adrian, > > some comments below inter-leaved within the text. Hopefully they will be > useful. > > > On Fri, 2008-12-19 at 18:38 +0000, Adrian Simmons wrote: >> Dear Phil, Kate and Peter >> >> I finally managed to spend some time over the past few days assembling >> some provisional figures (attached) for a comparison of our reanalyses >> with your data sets. Era-Interim is now in January 2008, and when it >> completes 2008 (probably late next month or early February), we'll have >> 20 years of this reanalysis, and thirty years of combined >> ERA-40/ERA-Interim reanalysis from the FGGE year onwards. My idea is to >> start writing something soon, with a view to finalising the figures and >> text once we have results up to the end of 2008. > > Sounds great. > >> Here are some comments on the figures (all of which are land-only). >> >> (i) Figs. 1-3 These cover the temperature comparisons, and simply update >> the picture from Simmons, Jones et al. (2004). In these and other >> figures I have normalised everything to give zero anomaly for the >> ten-year mean 1989-1998, which is convenient as I can then plot maps of >> the anomalies for the decades 1979-1988 and 1999-2008. (Note that for >> now the maps labelled 1999-2008 are actually for 1998-2007, and those >> for 2004-2008 are actually for 2003-2007 - this will be fixed when we >> actually have reanalysis data for 2008). For the maps I show all CRU >> grid squares for which less than 5% of months are missing. >> > > Is there any reason that the criteria is this tight? Is there any > implication to loosening it to say 25%? You should find extra data in at > least some relatively data sparse areas which may be worth the cost of > the additional noise, or does it make no substantial difference to > coverage for either dataset? > >> Basically we see excellent continuation of the ERA-40 results when we >> sample ERA-Interim and ERA-40 in the same way as CRUTEM3v samples in >> space and time. A point that might be emphasized is that the coverage of >> CRUTEM3v is much poorer for the last ten years than the first one. Any >> comment on this? Are there many late-arriving CLIMATs that will change >> the picture when I download a new version of CRUTEM3v? It is noteworthy >> that CRUTEM3v samples poorly the Arctic region where ERA indicates the >> strongest warming (and ERA also shows strong warming over sea-ice - or >> where sea-ice used to be!). Fig 1 shows a stronger trend in ERA that in >> CRUTEM3v when we use the complete ERA record rather than sampling it as >> CRUTEM3v. In other words ERA-Interim shows less recent cooling than >> CRUTEM3v. Here we are treading on thin ice (sorry!) from a PR viewpoint, >> so I'm interested in your reaction to all this. > > For all the timeseries figures or at least those that you subsample to > coverage mimicking can we consider a slightly different presentational > style? I find it hard to ascertain what is going on because the lines > are sufficiently thick to overlap substantially for much of the period. > An alternative approach would be to show one dataset's (the > "observations"? Open to argument and counter-vailing views here) actual > timeseries and then offset from this the difference series to the other > two timeseries. We know the datasets agree on the high frequency, its > the low frequency behaviour that really interests us here. The > difference series illuminates this directly and could be assessed for > statistical trend significance probably without a d.o.f. correction to > the uncertainty. This would very considerably strengthen this part of > the analysis as it is actually really getting at a very thorough > investigation of the inter-dataset differences which is key. > >> (ii) Fig. 4 shows excellent agreement between HadCRUH and ERA-40 time >> series for q. After 2003, the "sampled as HadCRUH" means I sample >> ERA-Interim for the grid squares for which HadCRUH gives values for >> December 2003. Slightly less trend for the full ERA dataset. >> >> (iii) Fig. 5 and 6 are time series for RH. Generally good agreement >> also. Two points. One is shift in ERA-40 around 1990 for Europe. I think >> this is a reanalysis problem. Hard to be sure, but could be explained by >> increased dry bias of sondes. No time to write more now, but Figs 10 and >> 11 relate to this, and also to second point - the relative dryness of >> recent years. I cannot find a reanalysis problem to explain this, and am >> inclined to think for now it could be real - there is no fundamental >> physical reason why relative humidity should not show a trend in a >> climate that is shifting. But it is really frustrating that HadCRUH >> stops at the end of 2003. What are the prospects for extending it? Could >> something quick be done as a check on the reanalysis result - omitting >> homogenization and subtle QC checks for example. > > > We could, in theory, update the series as we now have, just, > unquarantined disk with the ISD source updated through the end of 2007 > (wouldn't help with '08). We could, as an interim measure, update > through 2007 by simply plonking it on the end for now (not quite that > simple as we'd have to convert to required res and format but the hard > yards are done). Adding 2008 would be technically feasible but a heck of > a lot of work and is probably outside the time remit. We were going to > produce a 2nd gen product for end of '09 through 2007 then worry about > instigating updates. That's quite a lot to do, but I guess in theory we > have the data through 2007 so could update the land component thru then. > Not sure about marine, but that's not your problem ...! > >> (iv) Fig 7 shows ten-year anomaly maps. Don't have HadCRUH for 1999-2008 >> of course. RH map for 1999-2008 show good temporal continuity - >> generally drying (relatively) in tropics and mid latitudes, and >> moistening (relatively) at high latitudes. Perhaps not implausible - for >> RH to stay uniform as T increases the water has to come from somewhere. >> That could be difficult over dry soils. And easier over melting soils?? >> Is anything like this seen in the Hadley Centre climate runs? >> > > In the humidity maps can you flip the colour scheme so that red is drier > and blue is wetter? That is more intuitive to me at least. > > We haven't had an in depth look at climate model output in this respect. > We should have some HadGEM1 fields that we could dig out, but Kate's > been doing some HadCRUH-climate model work and we want to ensure we > don't duplicate so suggest Kate updates you on status. > >> (v) Fig 8 shows consistency across boundary layer (level 49 is close to >> 850hPa, where analysis in influenced more strongly by sondes rather than >> SYNOPs). >> >> (vi) Fig 9 shows some 5-year means, comparing ERA-INT and HadCRUH for >> 1999-2003. >> >> (vii) Figs 10 and 11 are fits of background and analysis to assimilated >> observations. There are subtleties I don't have time to explain right >> now, but basically a flat line is a good thing from the viewpoint of >> trend analysis. It is the drop in the background RH curves and rise in >> the background Q curves for the northern hemisphere between 1988 and >> 1993 that makes me think the RH shift between the 1980s and 1990s in ERA >> relative to HadCRUH is a reanalysis problem. Equally though, there is >> nothing in these curves (or some others I've looked at) that points to >> the lowering of RH in the reanalyses for the last few years as being >> spurious. > > Which makes it more appealing to do a quick and dirty fix to update > HadCRUH through at least 2007 as that would be the clincher. Of course, > in this era we also have the GPS-PW network from June Wang at NCAR. > Perhaps this would be a useful addition? Aiguo Dai also has an > alternative observational dataset of surface q and RH which could be a > useful addition. I think his is updated through 2006. It has no > homogeneity adjustments but because its a compilation of synops probably > better coverage (swings and roundabouts). We have both of them visiting > here in the summer for 2 months and 3 months respectively so opportunity > to do some collaborative work on this. I know they were keen to visit > CRU and Reading institutes. If you want their email addresses so you can > send some enquiries just let me know. > > I hope you have a relaxing christmas > > Peter -- -------------------------------------------------- Adrian Simmons European Centre for Medium-Range Weather Forecasts Shinfield Park, Reading, RG2 9AX, UK Phone: +44 118 949 9700 Fax: +44 118 986 9450 -------------------------------------------------- Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------