English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This article concerns the expressive power of depth in deep feed-forwardneural nets with ReLU activations. Specifically, we answer the followingquestion: for a fixed $d_{in}\geq 1,$ what is the minimal width $w$ so thatneural nets with ReLU activations, input dimension $d_{in}$, hidden layerwidths at most $w,$ and arbitrary depth can approximate any continuous,real-valued function of $d_{in}$ variables arbitrarily well? It turns out thatthis minimal width is exactly equal to $d_{in}+1.$ That is, if all the hiddenlayer widths are bounded by $d_{in}$, then even in the infinite depth limit,ReLU nets can only express a very limited class of functions, and, on the otherhand, any continuous function on the $d_{in}$-dimensional unit cube can beapproximated to arbitrary precision by ReLU nets in which all hidden layershave width exactly $d_{in}+1.$ Our construction in fact shows that anycontinuous function $f:[0,1]^{d_{in}}\to\mathbb R^{d_{out}}$ can beapproximated by a net of width $d_{in}+d_{out}$. We obtain quantitative depthestimates for such an approximation in terms of the modulus of continuity of$f$.

Approximating Continuous Functions by ReLU Nets of Minimal Width