English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We analyze the topological properties of the set of functions that can beimplemented by neural networks of a fixed size. Surprisingly, this set has manyundesirable properties. It is highly non-convex, except possibly for a fewexotic activation functions. Moreover, the set is not closed with respect to$L^p$-norms, $0 < p < \infty$, for all practically-used activation functions,and also not closed with respect to the $L^\infty$-norm for allpractically-used activation functions except for the ReLU and the parametricReLU. Finally, the function that maps a family of weights to the functioncomputed by the associated network is not inverse stable for every practicallyused activation function. In other words, if $f_1, f_2$ are two functionsrealized by neural networks and if $f_1, f_2$ are close in the sense that$\|f_1 - f_2\|_{L^\infty} \leq \varepsilon$ for $\varepsilon > 0$, it is,regardless of the size of $\varepsilon$, usually not possible to find weights$w_1, w_2$ close together such that each $f_i$ is realized by a neural networkwith weights $w_i$. Overall, our findings identify potential causes for issuesin the training procedure of deep learning such as no guaranteed convergence,explosion of parameters, and slow convergence.

Topological properties of the set of functions generated by neural networks of fixed size