Premium
Anomalies in the Foundations of Ridge Regression: Some Clarifications
Author(s) -
Kapat Prasenjit,
Goel Prem K.
Publication year - 2010
Publication title -
international statistical review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.051
H-Index - 54
eISSN - 1751-5823
pISSN - 0306-7734
DOI - 10.1111/j.1751-5823.2010.00113.x
Subject(s) - mathematics , monotone polygon , estimator , ridge , monotonic function , regression , norm (philosophy) , mathematical analysis , statistics , geometry , geology , philosophy , paleontology , epistemology
Summary Several anomalies in the foundations of ridge regression from the perspective of constrained least‐square (LS) problems were pointed out in Jensen & Ramirez. Some of these so‐called anomalies, attributed to the non‐monotonic behaviour of the norm of unconstrained ridge estimators and the consequent lack of sufficiency of Lagrange's principle, are shown to be incorrect. It is noted in this paper that, for a fixed Y , norms of unconstrained ridge estimators corresponding to the given basis are indeed strictly monotone. Furthermore, the conditions for sufficiency of Lagrange's principle are valid for a suitable range of the constraint parameter. The discrepancy arose in the context of one data set due to confusion between estimates of the parameter vector, β , corresponding to different parametrization (choice of bases) and/or constraint norms. In order to avoid such confusion, it is suggested that the parameter β corresponding to each basis be labelled appropriately.