Premium
A closer look at the bias–variance trade‐off in multivariate calibration
Author(s) -
Faber Nicolaas KLAAS M.
Publication year - 1999
Publication title -
journal of chemometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.47
H-Index - 92
eISSN - 1099-128X
pISSN - 0886-9383
DOI - 10.1002/(sici)1099-128x(199903/04)13:2<185::aid-cem538>3.0.co;2-n
Subject(s) - variance (accounting) , econometrics , ordinary least squares , partial least squares regression , statistics , principal component regression , principal component analysis , multivariate statistics , contrast (vision) , calibration , set (abstract data type) , variance components , regression , mathematics , computer science , economics , artificial intelligence , accounting , programming language
Abstract Principal component regression (PCR) and partial least squares (PLS) avoid the high variance often associated with ordinary least squares (OLS) results by allowing a small bias in the model. This paper presents a closer look at this bias–variance trade‐off by discussing three practical aspects: (1) variance increases relatively slowly with increasing model complexity; (2) bias may be zero for the optimum model; (3) variance does not necessarily increase with increasing model complexity. While the first aspect is well known, the last two aspects are not. The second aspect implies that so‐called biased regression methods do not necessarily yield biased predictions, while the third aspect, which is only encountered with non‐linear estimation methods such as PLS, even contradicts the concept of bias–variance trade‐off. The possibility of having both variance and bias decreasing with increasing PLS model complexity is illustrated using a near‐infrared data set published by Fearn. Copyright © 1999 John Wiley & Sons, Ltd.