English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Most high-dimensional estimation and prediction methods propose to minimize acost function (empirical risk) that is written as a sum of losses associated toeach data point. In this paper we focus on the case of non-convex losses, whichis practically important but still poorly understood. Classical empiricalprocess theory implies uniform convergence of the empirical risk to thepopulation risk. While uniform convergence implies consistency of the resultingM-estimator, it does not ensure that the latter can be computed efficiently.  In order to capture the complexity of computing M-estimators, we propose tostudy the landscape of the empirical risk, namely its stationary points andtheir properties. We establish uniform convergence of the gradient and Hessianof the empirical risk to their population counterparts, as soon as the numberof samples becomes larger than the number of unknown parameters (modulologarithmic factors). Consequently, good properties of the population risk canbe carried to the empirical risk, and we can establish one-to-onecorrespondence of their stationary points. We demonstrate that in severalproblems such as non-convex binary classification, robust regression, andGaussian mixture model, this result implies a complete characterization of thelandscape of the empirical risk, and of the convergence properties of descentalgorithms.  We extend our analysis to the very high-dimensional setting in which thenumber of parameters exceeds the number of samples, and provide acharacterization of the empirical risk landscape under a nearlyinformation-theoretically minimal condition. Namely, if the number of samplesexceeds the sparsity of the unknown parameters vector (modulo logarithmicfactors), then a suitable uniform convergence result takes place. We apply thisresult to non-convex binary classification and robust regression in veryhigh-dimension.

The Landscape of Empirical Risk for Non-convex Losses