English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Existing Rademacher complexity bounds for neural networks rely only on normcontrol of the weight matrices and depend exponentially on depth via a productof the matrix norms. Lower bounds show that this exponential dependence ondepth is unavoidable when no additional properties of the training data areconsidered. We suspect that this conundrum comes from the fact that thesebounds depend on the training data only through the margin. In practice, manydata-dependent techniques such as Batchnorm improve the generalizationperformance. For feedforward neural nets as well as RNNs, we obtain tighterRademacher complexity bounds by considering additional data-dependentproperties of the network: the norms of the hidden layers of the network, andthe norms of the Jacobians of each layer with respect to all previous layers.Our bounds scale polynomially in depth when these empirical quantities aresmall, as is usually the case in practice. To obtain these bounds, we developgeneral tools for augmenting a sequence of functions to make their compositionLipschitz and then covering the augmented functions. Inspired by our theory, wedirectly regularize the network's Jacobians during training and empiricallydemonstrate that this improves test performance.

Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation