English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

One of the defining properties of deep learning is that models are chosen tohave many more parameters than available training data. In light of thiscapacity for overfitting, it is remarkable that simple algorithms like SGDreliably return solutions with low test error. One roadblock to explainingthese phenomena in terms of implicit regularization, structural properties ofthe solution, and/or easiness of the data is that many learning bounds arequantitatively vacuous when applied to networks learned by SGD in this "deeplearning" regime. Logically, in order to explain generalization, we neednonvacuous bounds. We return to an idea by Langford and Caruana (2001), whoused PAC-Bayes bounds to compute nonvacuous numerical bounds on generalizationerror for stochastic two-layer two-hidden-unit neural networks via asensitivity analysis. By optimizing the PAC-Bayes bound directly, we are ableto extend their approach and obtain nonvacuous generalization bounds for deepstochastic neural network classifiers with millions of parameters trained ononly tens of thousands of examples. We connect our findings to recent and oldwork on flat minima and MDL-based explanations of generalization.

Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data