Premium
On the non‐negative garrotte estimator
Author(s) -
Yuan Ming,
Lin Yi
Publication year - 2007
Publication title -
journal of the royal statistical society: series b (statistical methodology)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.523
H-Index - 137
eISSN - 1467-9868
pISSN - 1369-7412
DOI - 10.1111/j.1467-9868.2007.00581.x
Subject(s) - estimator , elastic net regularization , mathematics , ordinary least squares , consistency (knowledge bases) , lasso (programming language) , least squares function approximation , variable (mathematics) , piecewise , statistics , mathematical optimization , regression , computer science , discrete mathematics , mathematical analysis , world wide web
Summary. We study the non‐negative garrotte estimator from three different aspects: consistency, computation and flexibility. We argue that the non‐negative garrotte is a general procedure that can be used in combination with estimators other than the original least squares estimator as in its original form. In particular, we consider using the lasso, the elastic net and ridge regression along with ordinary least squares as the initial estimate in the non‐negative garrotte. We prove that the non‐negative garrotte has the nice property that, with probability tending to 1, the solution path contains an estimate that correctly identifies the set of important variables and is consistent for the coefficients of the important variables, whereas such a property may not be valid for the initial estimators. In general, we show that the non‐negative garrotte can turn a consistent estimate into an estimate that is not only consistent in terms of estimation but also in terms of variable selection. We also show that the non‐negative garrotte has a piecewise linear solution path. Using this fact, we propose an efficient algorithm for computing the whole solution path for the non‐negative garrotte. Simulations and a real example demonstrate that the non‐negative garrotte is very effective in improving on the initial estimator in terms of variable selection and estimation accuracy.