English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern neural networks are highly overparameterized, with capacity tosubstantially overfit to training data. Nevertheless, these networks oftengeneralize well in practice. It has also been observed that trained networkscan often be "compressed" to much smaller representations. The purpose of thispaper is to connect these two empirical observations. Our main technical resultis a generalization bound for compressed networks based on the compressed size.Combined with off-the-shelf compression algorithms, the bound leads to state ofthe art generalization guarantees; in particular, we provide the firstnon-vacuous generalization guarantees for realistic architectures applied tothe ImageNet classification problem. As additional evidence connectingcompression and generalization, we show that compressibility of models thattend to overfit is limited: We establish an absolute limit on expectedcompressibility as a function of expected generalization error, where theexpectations are over the random choice of training examples. The bounds arecomplemented by empirical results that show an increase in overfitting impliesan increase in the number of bits required to describe a trained network.

Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach