Premium
Maximum likelihood fitting using ordinary least squares algorithms
Author(s) -
Bro Rasmus,
Sidiropoulos Nicholaos D.,
Smilde Age K.
Publication year - 2002
Publication title -
journal of chemometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.47
H-Index - 92
eISSN - 1099-128X
pISSN - 0886-9383
DOI - 10.1002/cem.734
Subject(s) - mathematics , algorithm , least squares function approximation , total least squares , residual , non linear least squares , heteroscedasticity , residual sum of squares , likelihood function , covariance , generalized least squares , iterative method , lack of fit sum of squares , gaussian , ordinary least squares , restricted maximum likelihood , statistics , estimation theory , singular value decomposition , physics , quantum mechanics , estimator
Abstract In this paper a general algorithm is provided for maximum likelihood fitting of deterministic models subject to Gaussian‐distributed residual variation (including any type of non‐singular covariance). By deterministic models is meant models in which no distributional assumptions are valid (or applied) on the parameters. The algorithm may also more generally be used for weighted least squares (WLS) fitting in situations where either distributional assumptions are not available or other than statistical assumptions guide the choice of loss function. The algorithm to solve the associated problem is called MILES (Maximum likelihood via Iterative Least squares EStimation). It is shown that the sought parameters can be estimated using simple least squares (LS) algorithms in an iterative fashion. The algorithm is based on iterative majorization and extends earlier work for WLS fitting of models with heteroscedastic uncorrelated residual variation. The algorithm is shown to include several current algorithms as special cases. For example, maximum likelihood principal component analysis models with and without offsets can be easily fitted with MILES. The MILES algorithm is simple and can be implemented as an outer loop in any least squares algorithm, e.g. for analysis of variance, regression, response surface modeling, etc. Several examples are provided on the use of MILES. Copyright © 2002 John Wiley & Sons, Ltd.