Premium
Robust estimation via robust gradient estimation
Author(s) -
Prasad Adarsh,
Suggala Arun Sai,
Balakrishnan Sivaraman,
Ravikumar Pradeep
Publication year - 2020
Publication title -
journal of the royal statistical society: series b (statistical methodology)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.523
H-Index - 137
eISSN - 1467-9868
pISSN - 1369-7412
DOI - 10.1111/rssb.12364
Subject(s) - estimator , robustness (evolution) , minification , robust regression , gradient descent , exponential family , computer science , robust statistics , mathematical optimization , regression , mathematics , statistics , artificial intelligence , machine learning , artificial neural network , biochemistry , gene , chemistry
Summary We provide a new computationally efficient class of estimators for risk minimization. We show that these estimators are robust for general statistical models, under varied robustness settings, including in the classical Huber ε ‐contamination model, and in heavy‐tailed settings. Our workhorse is a novel robust variant of gradient descent, and we provide conditions under which our gradient descent variant provides accurate estimators in a general convex risk minimization problem. We provide specific consequences of our theory for linear regression and logistic regression and for canonical parameter estimation in an exponential family. These results provide some of the first computationally tractable and provably robust estimators for these canonical statistical models. Finally, we study the empirical performance of our proposed methods on synthetic and real data sets, and we find that our methods convincingly outperform a variety of baselines.