English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The classical Universal Approximation Theorem holds for neural networks ofarbitrary width and bounded depth. Here we consider the natural `dual' scenariofor networks of bounded width and arbitrary depth. Precisely, let $n$ be thenumber of inputs neurons, $m$ be the number of output neurons, and let $\rho$be any nonaffine continuous function, with a continuous nonzero derivative atsome point. Then we show that the class of neural networks of arbitrary depth,width $n + m + 2$, and activation function $\rho$, is dense in $C(K;\mathbb{R}^m)$ for $K \subseteq \mathbb{R}^n$ with $K$ compact. This coversevery activation function possible to use in practice, and also includespolynomial activation functions, which is unlike the classical version of thetheorem, and provides a qualitative difference between deep narrow networks andshallow wide networks. We then consider several extensions of this result. Inparticular we consider nowhere differentiable activation functions, density innoncompact domains with respect to the $L^p$-norm, and how the width may bereduced to just $n + m + 1$ for `most' activation functions.

Universal Approximation with Deep Narrow Networks