English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We consider a singularly perturbed system of stochastic differentialequations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximatethe Entropic Gradient Descent in the optimization of deep neural networks, viahomogenisation. We embed it in a much larger class of two-scale stochasticcontrol problems and rely on convergence results for Hamilton-Jacobi-Bellmanequations with unbounded data proved recently by ourselves (ESAIM ControlOptim. Calc. Var. 2023). We show that the limit of the value functions isitself the value function of an effective control problem with extendedcontrols, and that the trajectories of the perturbed system converge in asuitable sense to the trajectories of the limiting effective control system.These rigorous results improve the understanding of the convergence of thealgorithms used by Chaudhari et al., as well as of their possible extensionswhere some tuning parameters are modelled as dynamic controls.

Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations