English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present a unified framework to analyze the global convergence of Langevindynamics based algorithms for nonconvex finite-sum optimization with $n$component functions. At the core of our analysis is a direct analysis of theergodicity of the numerical approximations to Langevin dynamics, which leads tofaster convergence rates. Specifically, we show that gradient Langevin dynamics(GLD) and stochastic gradient Langevin dynamics (SGLD) converge to the almostminimizer within $\tilde O\big(nd/(\lambda\epsilon) \big)$ and $\tildeO\big(d^7/(\lambda^5\epsilon^5) \big)$ stochastic gradient evaluationsrespectively, where $d$ is the problem dimension, and $\lambda$ is the spectralgap of the Markov chain generated by GLD. Both results improve upon the bestknown gradient complexity results (Raginsky et al., 2017). Furthermore, for thefirst time we prove the global convergence guarantee for variance reducedstochastic gradient Langevin dynamics (SVRG-LD) to the almost minimizer within$\tilde O\big(\sqrt{n}d^5/(\lambda^4\epsilon^{5/2})\big)$ stochastic gradientevaluations, which outperforms the gradient complexities of GLD and SGLD in awide regime. Our theoretical analyses shed some light on using Langevindynamics based algorithms for nonconvex optimization with provable guarantees.

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization