Adaptive data assimilation including the effect of spatial variations in observation error

An optimal adaptive data assimilation algorithm is derived using the maximum likelihood method based on a conditional Gaussian probability density function for the first‐guess and direct observations of the state variables but including local estimates of the observation and first‐guess error statistics. An interpolation of the first‐guess field to the observation coordinates is not required under the assumption of locally homogeneous statistics for the random atmosphere. However, the definition of observation error requires a definition of model ‘truth’ which is defined as a spatial average of the continuous random atmospheric variables. Then the total observation error consists of two independent components: an instrument error and an observation sampling error defined by the spatial average of the observation and the statistics of the local turbulence. Estimates of the observation sampling error statistics are determined from an ensemble of background or first‐guess fields or from the analysis of the raw data from instrumented aircraft, Doppler lidars, or radar profilers. The spatial variations of the sampling error are referenced to the local turbulence conditions at each analysis coordinate and therefore each observation can have a different observation error for each nearby analysis coordinate. The extension of the adaptive assimilation concept to include the spatial variations in observation error for statistical interpolation, 3D‐Var, 4D‐Var, extended Kalman filtering, and ensemble Kalman filtering is also presented for the traditional meaning of observation error, i.e. each observation is assigned a single error. The conditional analysis error is derived for a single observation at the analysis coordinate and multiple observations around the analysis point. Example calculations of the conditional analysis error are presented for a few simple set of observation and measurement geometries to demonstrate the impact of the spatially variable observation errors. For rawinsonde observations, the observation sampling error is an important contribution to optimal adaptive data assimilation. Observations performed as a spatial average along a line, such as aircraft, radar profiler and Doppler lidar measurements, have smaller sampling errors than rawinsonde observations and therefore reduce the analysis error if the instrument error is sufficiently low. Copyright © 2006 Royal Meteorological Society.