English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Increasing the size of overparameterized neural networks has been a key inachieving state-of-the-art performance. This is captured by the double descentphenomenon, where the test loss follows a decreasing-increasing-decreasingpattern as model width increases. However, the effect of label noise on thetest loss curve has not been fully explored. In this work, we uncover anintriguing phenomenon where label noise leads to a \textit{final ascent} in theoriginally observed double descent curve. Specifically, under a sufficientlylarge noise-to-sample-size ratio, optimal generalization is achieved atintermediate widths. Through theoretical analysis, we attribute this phenomenonto the shape transition of test loss variance induced by label noise.Furthermore, we extend the final ascent phenomenon to model density and providethe first theoretical characterization showing that reducing density byrandomly dropping trainable parameters improves generalization under labelnoise. We also thoroughly examine the roles of regularization and sample size.Surprisingly, we find that larger $\ell_2$ regularization and robust learningmethods against label noise exacerbate the final ascent. We confirm thevalidity of our findings through extensive experiments on ReLu networks trainedon MNIST, ResNets trained on CIFAR-10/100, and InceptionResNet-v2 trained onStanford Cars with real-world noisy labels.

Investigating the Impact of Model Width and Density on Generalization in Presence of Label Noise