English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study benign overfitting in two-layer ReLU networks trained using gradientdescent and hinge loss on noisy data for binary classification. In particular,we consider linearly separable data for which a relatively small proportion oflabels are corrupted or flipped. We identify conditions on the margin of theclean data that give rise to three distinct training outcomes: benignoverfitting, in which zero loss is achieved and with high probability test datais classified correctly; overfitting, in which zero loss is achieved but testdata is misclassified with probability lower bounded by a constant; andnon-overfitting, in which clean points, but not corrupt points, achieve zeroloss and again with high probability test data is classified correctly. Ouranalysis provides a fine-grained description of the dynamics of neuronsthroughout training and reveals two distinct phases: in the first phase cleanpoints achieve close to zero loss, in the second phase clean points oscillateon the boundary of zero loss while corrupt points either converge towards zeroloss or are eventually zeroed by the network. We prove these results using acombinatorial approach that involves bounding the number of clean versuscorrupt updates across these phases of training.

Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?