Improved Quality Assurance for Historical Hourly Temperature and Humidity: Development and Application to Environmental Analysis

Historical hourly surface synoptic (airways) meteorological reports from around the United States have been digitized as part of the NOAA Climate Database Modernization Program. An important component is improvement of quality assurance procedures for hourly meteorological data. This paper presents the development and testing of two components of a new complex framework, as well as their application toward construction, for the first time, of a 75-yr time series of apparent temperature. A pilot study indicated that a majority of flags thrown from an existing algorithm represent single-hour blips, rather than steps, and that frontal passages were being flagged incorrectly. Therefore, a model focused on flagging blips is developed; two blip-magnitude measures are compared that define a blip as a departure from temporally neighboring observations. Switches of dewpoint with dewpoint depression have also been noted among observer/digitizer errors, and so an additional check was developed to screen for these cases. This check is based on a relationship between dewpoint depression and diurnal temperature range. Tests using artificial replication of common errors indicate that the new blip model outperforms traditional step models considerably, and the new model flags an order-of-magnitude fewer frontal passages. Operational use of this check suggests type-I and type-II error rates are similar in magnitude and are approximately 5%. More than two-fifths of known dewpoint depression switch errors are caught. However, poor performance with systematic errors suggests that using the depression-range check at a coarser temporal scale than hour to hour may be more fruitful.