English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Why does training deep neural networks using stochastic gradient descent(SGD) result in a generalization error that does not worsen with the number ofparameters in the network? To answer this question, we advocate a notion ofeffective model capacity that is dependent on {\em a given randominitialization of the network} and not just the training algorithm and the datadistribution. We provide empirical evidences that demonstrate that the modelcapacity of SGD-trained deep networks is in fact restricted through implicitregularization of {\em the $\ell_2$ distance from the initialization}. We alsoprovide theoretical arguments that further highlight the need forinitialization-dependent notions of model capacity. We leave as open questionshow and why distance from initialization is regularized, and whether it issufficient to explain generalization.

Generalization in Deep Networks: The Role of Distance from Initialization