English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the loss landscape of training problems for deep artificial neuralnetworks with a one-dimensional real output whose activation functions containan affine segment and whose hidden layers have width at least two. It is shownthat such problems possess a continuum of spurious (i.e., not globally optimal)local minima for all target functions that are not affine. In contrast toprevious works, our analysis covers all sampling and parameterization regimes,general differentiable loss functions, arbitrary continuous nonpolynomialactivation functions, and both the finite- and infinite-dimensional setting. Itis further shown that the appearance of the spurious local minima in theconsidered training problems is a direct consequence of the universalapproximation theorem and that the underlying mechanisms also cause, e.g.,$L^p$-best approximation problems to be ill-posed in the sense of Hadamard forall networks that do not have a dense image. The latter result also holdswithout the assumption of local affine linearity and without any conditions onthe hidden layers.

On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems