English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern neural networks are typically trained in an over-parameterized regimewhere the parameters of the model far exceed the size of the training data.Such neural networks in principle have the capacity to (over)fit any set oflabels including pure noise. Despite this, somewhat paradoxically, neuralnetwork models trained via first-order methods continue to predict well on yetunseen test data. This paper takes a step towards demystifying this phenomena.Under a rich dataset model, we show that gradient descent is provably robust tonoise/corruption on a constant fraction of the labels despiteoverparameterization. In particular, we prove that: (i) In the first fewiterations where the updates are still in the vicinity of the initializationgradient descent only fits to the correct labels essentially ignoring the noisylabels. (ii) to start to overfit to the noisy labels network must stray ratherfar from from the initialization which can only occur after many moreiterations. Together, these results show that gradient descent with earlystopping is provably robust to label noise and shed light on the empiricalrobustness of deep networks as well as commonly adopted heuristics to preventoverfitting.

Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks