English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We prove new upper and lower bounds on the VC-dimension of deep neuralnetworks with the ReLU activation function. These bounds are tight for almostthe entire range of parameters. Letting $W$ be the number of weights and $L$ bethe number of layers, we prove that the VC-dimension is $O(W L \log(W))$, andprovide examples with VC-dimension $\Omega( W L \log(W/L) )$. This improvesboth the previously known upper bounds and lower bounds. In terms of the number$U$ of non-linear units, we prove a tight bound $\Theta(W U)$ on theVC-dimension. All of these bounds generalize to arbitrary piecewise linearactivation functions, and also hold for the pseudodimensions of these functionclasses.  Combined with previous results, this gives an intriguing range ofdependencies of the VC-dimension on depth for networks with differentnon-linearities: there is no dependence for piecewise-constant, lineardependence for piecewise-linear, and no more than quadratic dependence forgeneral piecewise-polynomial.

Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks