English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern deep learning models with great expressive power can be trained tooverfit the training data but still generalize well. This phenomenon isreferred to as \textit{benign overfitting}. Recently, a few studies haveattempted to theoretically understand benign overfitting in neural networks.However, these works are either limited to neural networks with smoothactivation functions or to the neural tangent kernel regime. How and whenbenign overfitting can occur in ReLU neural networks remains an open problem.In this work, we seek to answer this question by establishingalgorithm-dependent risk bounds for learning two-layer ReLU convolutionalneural networks with label-flipping noise. We show that, under mild conditions,the neural network trained by gradient descent can achieve near-zero trainingloss and Bayes optimal test risk. Our result also reveals a sharp transitionbetween benign and harmful overfitting under different conditions on datadistribution in terms of test risk. Experiments on synthetic data back up ourtheory.

Benign Overfitting for Two-layer ReLU Convolutional Neural Networks