English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite extensive studies, the underlying reason as to why overparameterizedneural networks can generalize remains elusive. Existing theory shows thatcommon stochastic optimizers prefer flatter minimizers of the training loss,and thus a natural potential explanation is that flatness impliesgeneralization. This work critically examines this explanation. Throughtheoretical and empirical investigation, we identify the following threescenarios for two-layer ReLU networks: (1) flatness provably impliesgeneralization; (2) there exist non-generalizing flattest models and sharpnessminimization algorithms fail to generalize, and (3) perhaps most surprisingly,there exist non-generalizing flattest models, but sharpness minimizationalgorithms still generalize. Our results suggest that the relationship betweensharpness and generalization subtly depends on the data distributions and themodel architectures and sharpness minimization algorithms do not only minimizesharpness to achieve better generalization. This calls for the search for otherexplanations for the generalization of over-parameterized neural networks.

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization