English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We derive generalization and excess risk bounds for neural nets using afamily of complexity measures based on a multilevel relative entropy. Thebounds are obtained by introducing the notion of generated hierarchicalcoverings of neural nets and by using the technique of chaining mutualinformation introduced in Asadi et al. NeurIPS'18. The resulting bounds arealgorithm-dependent and exploit the multilevel structure of neural nets. This,in turn, leads to an empirical risk minimization problem with a multilevelentropic regularization. The minimization problem is resolved by introducing amulti-scale generalization of the celebrated Gibbs posterior distribution,proving that the derived distribution achieves the unique minimum. This leadsto a new training procedure for neural nets with performance guarantees, whichexploits the chain rule of relative entropy rather than the chain rule ofderivatives (as in backpropagation). To obtain an efficient implementation ofthe latter, we further develop a multilevel Metropolis algorithm simulating themulti-scale Gibbs distribution, with an experiment for a two-layer neural neton the MNIST data set.

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets