English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many mathematical convergence results for gradient descent (GD) basedalgorithms employ the assumption that the GD process is (almost surely) boundedand, also in concrete numerical simulations, divergence of the GD process mayslow down, or even completely rule out, convergence of the error function. Inpractical relevant learning problems, it thus seems to be advisable to designthe ANN architectures in a way so that GD optimization processes remainbounded. The property of the boundedness of GD processes for a given learningproblem seems, however, to be closely related to the existence of minimizers inthe optimization landscape and, in particular, GD trajectories may escape toinfinity if the infimum of the error function (objective function) is notattained in the optimization landscape. This naturally raises the question ofthe existence of minimizers in the optimization landscape and, in the situationof shallow residual ANNs with multi-dimensional input layers andmulti-dimensional hidden layers with the ReLU activation, the main result ofthis work answers this question affirmatively for a general class of lossfunctions and all continuous target functions. In our proof of this statement,we propose a kind of closure of the search space, where the limits are calledgeneralized responses, and, thereafter, we provide sufficient criteria for theloss function and the underlying probability distribution which ensure that alladditional artificial generalized responses are suboptimal which finally allowsus to conclude the existence of minimizers in the optimization landscape.

On the existence of minimizers in shallow residual ReLU neural network optimization landscapes