English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Gradient descent (GD) type optimization schemes are the standard instrumentsto train fully connected feedforward artificial neural networks (ANNs) withrectified linear unit (ReLU) activation and can be considered as temporaldiscretizations of solutions of gradient flow (GF) differential equations. Ithas recently been proved that the risk of every bounded GF trajectory convergesin the training of ANNs with one hidden layer and ReLU activation to the riskof a critical point. Taking this into account it is one of the key researchissues in the mathematical convergence analysis of GF trajectories and GD typeoptimization schemes, respectively, to study sufficient and necessaryconditions for critical points of the risk function and, thereby, to obtain anunderstanding about the appearance of critical points in dependence of theproblem parameters such as the target function. In the first main result ofthis work we prove in the training of ANNs with one hidden layer and ReLUactivation that for every $ a, b \in \mathbb{R} $ with $ a < b $ and everyarbitrarily large $ \delta > 0 $ we have that there exists a Lipschitzcontinuous target function $ f \colon [a,b] \to \mathbb{R} $ such that forevery number $ H > 1 $ of neurons on the hidden layer we have that the riskfunction has uncountably many different realization functions of non-globallocal minimum points whose risks are strictly larger than the sum of the riskof the global minimum points and the arbitrarily large $ \delta $. In thesecond main result of this work we show in the training of ANNs with one hiddenlayer and ReLU activation in the special situation where there is only oneneuron on the hidden layer and where the target function is continuous andpiecewise polynomial that there exist at most finitely many differentrealization functions of critical points.

On the existence of infinitely many realization functions of non-global  local minima in the training of artificial neural networks with ReLU  activation