On the reliability of errors-in-variables models

Reliability has been quantified in a simple Gauss–Markov model (GMM) by Baarda (1968) for the application to geodetic networks as the potential to detect outliers –with a specified significance and power – by testing the least-squares residuals for their zero expectation property after an adjustment assuming “no outliers”. It was shown that, under homoscedastic conditions, the so-called “redundancy numbers” could very well serve as indicators for the “local reliability” of an (individual) observation. In contrast, the maximum effect of any undetectible outlier on the estimated parameters would indicate “global reliability”. This concept had been extended successfully to the case of correlated observations by Schaffrin (1997) quite a while ago. However, no attempt has been made so far to extend Baarda’s results to the (homoscedastic) errors-in-variables (EIV) model for which Golub and van Loan (1980) had found their – now famous – algorithm to generate the total least-squares (TLS) solution, together with all the residuals. More recently, this algorithm has been generalized by Schaffrin and Wieser (2008) to the case where a truly – not just elementwise –weighted TLS solution can be computed when the covariance matrix has the structure of a Kronecker–Zehfuss product. Here, an attempt will be made to define reliability measures within such an EIV-model, in analogy to Baarda’s original approach. Introduction In geodetic science, since Baarda’s (1968) seminal report, reliability analysis in the frame work of a Gauss–Markov model is firmly based on • the potential to detect outliers in any given observation (“inner reliability”), and • the maximum effect of any undetected outlier on the estimated parameter vector (“outer reliability”). Received November 19, 2011. 2010 Mathematics Subject Classification. 15A09,15A52, 62F35.