English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the training of deep neural networks by gradient descent wherefloating-point arithmetic is used to compute the gradients. In this frameworkand under realistic assumptions, we demonstrate that it is highly unlikely tofind ReLU neural networks that maintain, in the course of training withgradient descent, superlinearly many affine pieces with respect to their numberof layers. In virtually all approximation theoretical arguments that yieldhigh-order polynomial rates of approximation, sequences of ReLU neural networkswith exponentially many affine pieces compared to their numbers of layers areused. As a consequence, we conclude that approximating sequences of ReLU neuralnetworks resulting from gradient descent in practice differ substantially fromtheoretically constructed sequences. The assumptions and the theoreticalresults are compared to a numerical study, which yields concurring results.

Limitations of neural network training due to numerical instability of backpropagation