Open AccessComputing Leapfrog Regularization Paths with Applications to Large-Scale K-mer Logistic Regression
Author(s) -
Philipp Benner
Publication year - 2021
Publication title -
journal of computational biology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.585
H-Index - 95
eISSN - 1557-8666
pISSN - 1066-5277
DOI - 10.1089/cmb.2020.0284
Subject(s) - regularization (linguistics) , regularization perspectives on support vector machines , inference , computer science , algorithm , backus–gilbert method , feature selection , mathematics , proximal gradient methods for learning , mathematical optimization , artificial intelligence , inverse problem , tikhonov regularization , mathematical analysis
High-dimensional statistics deals with statistical inference when the number of parameters or features p exceeds the number of observations n (i.e., p ≫ n ). In this case, the parameter space must be constrained either by regularization or by selecting a small subset of m ≤ n features. Feature selection throughl1-regularization combines the benefits of both approaches and has proven to yield good results in practice. However, the functional relation between the regularization strength λ and the number of selected features m is difficult to determine. Hence, parameters are typically estimated for all possible regularization strengths λ . These so-called regularization paths can be expensive to compute and most solutions may not even be of interest to the problem at hand. As an alternative, an algorithm is proposed that determines thel1-regularization strength λ iteratively for a fixed m . The algorithm can be used to compute leapfrog regularization paths by subsequently increasing m .