English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper explores the expressive power of deep neural networks through theframework of function compositions. We demonstrate that the repeatedcompositions of a single fixed-size ReLU network exhibit surprising expressivepower, despite the limited expressive capabilities of the individual networkitself. Specifically, we prove by construction that $\mathcal{L}_2\circ\boldsymbol{g}^{\circ r}\circ \boldsymbol{\mathcal{L}}_1$ can approximate$1$-Lipschitz continuous functions on $[0,1]^d$ with an error$\mathcal{O}(r^{-1/d})$, where $\boldsymbol{g}$ is realized by a fixed-sizeReLU network, $\boldsymbol{\mathcal{L}}_1$ and $\mathcal{L}_2$ are two affinelinear maps matching the dimensions, and $\boldsymbol{g}^{\circ r}$ denotes the$r$-times composition of $\boldsymbol{g}$. Furthermore, we extend such a resultto generic continuous functions on $[0,1]^d$ with the approximation errorcharacterized by the modulus of continuity. Our results reveal that acontinuous-depth network generated via a dynamical system has immenseapproximation power even if its dynamics function is time-independent andrealized by a fixed-size ReLU network.

On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network