English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this article we study fully-connected feedforward deep ReLU ANNs with anarbitrarily large number of hidden layers and we prove convergence of the riskof the GD optimization method with random initializations in the training ofsuch ANNs under the assumption that the unnormalized probability densityfunction of the probability distribution of the input data of the consideredsupervised learning problem is piecewise polynomial, under the assumption thatthe target function (describing the relationship between input data and theoutput data) is piecewise polynomial, and under the assumption that the riskfunction of the considered supervised learning problem admits at least oneregular global minimum. In addition, in the special situation of shallow ANNswith just one hidden layer and one-dimensional input we also verify thisassumption by proving in the training of such shallow ANNs that for everyLipschitz continuous target function there exists a global minimum in the risklandscape. Finally, in the training of deep ANNs with ReLU activation we alsostudy solutions of gradient flow (GF) differential equations and we prove thatevery non-divergent GF trajectory converges with a polynomial rate ofconvergence to a critical point (in the sense of limiting Fr\'echetsubdifferentiability). Our mathematical convergence analysis builds up on ideasfrom our previous article Eberle et al., on tools from real algebraic geometrysuch as the concept of semi-algebraic functions and generalizedKurdyka-Lojasiewicz inequalities, on tools from functional analysis such as theArzel\`a-Ascoli theorem, on tools from nonsmooth analysis such as the conceptof limiting Fr\'echet subgradients, as well as on the fact that the set ofrealization functions of shallow ReLU ANNs with fixed architecture forms aclosed subset of the set of continuous functions revealed by Petersen et al.

On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks