English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper we model the loss function of high-dimensional optimizationproblems by a Gaussian random field, or equivalently a Gaussian process. Ouraim is to study gradient descent in such loss functions or energy landscapesand compare it to results obtained from real high-dimensional optimizationproblems such as encountered in deep learning. In particular, we analyze thedistribution of the improved loss function after a step of gradient descent,provide analytic expressions for the moments as well as prove asymptoticnormality as the dimension of the parameter space becomes large. Moreover, wecompare this with the expectation of the global minimum of the landscapeobtained by means of the Euler characteristic of excursion sets. Besidescomplementing our analytical findings with numerical results from simulatedGaussian random fields, we also compare it to loss functions obtained fromoptimisation problems on synthetic and real data sets by proposing a "blackbox" random field toy-model for a deep neural network loss function.

Gradient descent in Gaussian random fields as a toy model for high-dimensional optimisation in deep learning