English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we investigate the Rademacher complexity of deep sparse neuralnetworks, where each neuron receives a small number of inputs. We provegeneralization bounds for multilayered sparse ReLU neural networks, includingconvolutional neural networks. These bounds differ from previous ones, as theyconsider the norms of the convolutional filters instead of the norms of theassociated Toeplitz matrices, independently of weight sharing between neurons.  As we show theoretically, these bounds may be orders of magnitude better thanstandard norm-based generalization bounds and empirically, they are almostnon-vacuous in estimating generalization in various simple classificationproblems. Taken together, these results suggest that compositional sparsity ofthe underlying target function is critical to the success of deep neuralnetworks.

Norm-based Generalization Bounds for Compositionally Sparse Neural Networks