Premium
A simple method to improve principal components regression
Author(s) -
Lang Wenjun,
Zou Hui
Publication year - 2020
Publication title -
stat
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.61
H-Index - 18
ISSN - 2049-1573
DOI - 10.1002/sta4.288
Subject(s) - principal component analysis , principal component regression , variance (accounting) , ordinary least squares , statistics , statistic , partial least squares regression , regression , regression analysis , mathematics , dimension (graph theory) , dimensionality reduction , goodness of fit , simple (philosophy) , explained sum of squares , computer science , artificial intelligence , philosophy , epistemology , accounting , pure mathematics , business
Principal components regression (PCR) is a well‐known method to achieve dimension reduction and often improved prediction over the ordinary least squares. The conventional PCR retains the principal components with large variance and discards those with smaller variance. This operation can easily lead to poor prediction when the response variable is related to principal components with small variance. In this work, we propose a simple remedy named response‐guided principal components regression (RgPCR) that selects principal components for regression based on both the variance of principal components and the goodness of fit to the response. RgPCR is easy to implement without using any optimization and works naturally for both low dimensional and high dimensional data. We derive a C p type statistic for selecting the tuning parameter in RgPCR. In our numerical experiments, RgPCR is shown to enjoy promising performance.