Premium
A Geometrical Interpretation of Collinearity: A Natural Way to Justify Ridge Regression and Its Anomalies
Author(s) -
GarcíaPérez José,
Mar LópezMartín María,
GarcíaGarcía Catalina,
SalmerónGómez Román
Publication year - 2020
Publication title -
international statistical review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.051
H-Index - 54
eISSN - 1751-5823
pISSN - 0306-7734
DOI - 10.1111/insr.12381
Subject(s) - collinearity , ridge , regression , diagonal , mathematics , interpretation (philosophy) , regression analysis , matrix (chemical analysis) , raising (metalworking) , flexibility (engineering) , statistics , econometrics , geometry , computer science , geography , cartography , materials science , composite material , programming language
Summary Justifying ridge regression from a geometrical perspective is one of the main contributions of this paper. To the best of our knowledge, this question has not been treated previously. This paper shows that ridge regression is a particular case of raising procedures that provide greater flexibility by transforming the matrix X associated with the model. Thus, raising procedures, based on a geometrical idea of the vectorial space associated with the columns of matrix X , lead naturally to ridge regression and justify the presence of the well‐known constant k on the main diagonal of matrix X ′ X . This paper also analyses and compares different alternatives to raising with respect to collinearity mitigation. The results are illustrated with an empirical application.