Premium
Characterizing multivariate calibration tradeoffs (bias, variance, selectivity, and sensitivity) to select model tuning parameters
Author(s) -
Kalivas John H.,
Palmer Jon
Publication year - 2014
Publication title -
journal of chemometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.47
H-Index - 92
eISSN - 1099-128X
pISSN - 0886-9383
DOI - 10.1002/cem.2555
Subject(s) - calibration , sensitivity (control systems) , partial least squares regression , principal component analysis , mathematics , regression , variance based sensitivity analysis , multivariate statistics , statistics , variance (accounting) , elastic net regularization , model selection , mean squared error , regression analysis , one way analysis of variance , analysis of variance , accounting , electronic engineering , engineering , business
Common methods used to form multivariate calibration models are partial least squares, principal component regression, and ridge regression. There are many aspects to forming a calibration model. The focus in this paper is determining “good” values of respective meta‐parameters (tuning parameters). In most situations, some form of cross‐validation and the resultant root mean square error of cross‐validation (RMSECV) values are evaluated. The RMSECV values can be thought of as a prediction accuracy (bias) measure and does not isolate variance information. The methods of partial least squares, principal component regression, and ridge regression (and others) are biased regression methods, and hence, there is a bias/variance tradeoff that should be considered in determining tuning parameter values. Presented in this paper are new combinations of model accuracy and fit that can be plotted with a model variance indicator for selecting appropriate tuning parameter values. For the spectroscopic and industrial data sets evaluated, these new combinations appear to remove ambiguities that can occur in RMSECV plots. Essentially, the user's preference for the degree of balance between bias and variance ultimately decides the tuning parameter selection and the underlying tradeoff between model selectivity and sensitivity. From the analysis of the selectivity/sensitivity tradeoff, a new definition of model selectivity is proposed that is able to discern when or when not a model vector deviates from the target orthogonal net analyte signal model vector as the well as the degree of deviation. A new merit is also proposed that simultaneously evaluates model selectivity and sensitivity. Copyright © 2013 John Wiley & Sons, Ltd.