English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The stochastic gradient Langevin Dynamics is one of the most fundamentalalgorithms to solve sampling problems and non-convex optimization appearing inseveral machine learning applications. Especially, its variance reducedversions have nowadays gained particular attention. In this paper, we study twovariants of this kind, namely, the Stochastic Variance Reduced GradientLangevin Dynamics and the Stochastic Recursive Gradient Langevin Dynamics. Weprove their convergence to the objective distribution in terms of KL-divergenceunder the sole assumptions of smoothness and Log-Sobolev inequality which areweaker conditions than those used in prior works for these algorithms. With thebatch size and the inner loop length set to $\sqrt{n}$, the gradient complexityto achieve an $\epsilon$-precision is$\tilde{O}((n+dn^{1/2}\epsilon^{-1})\gamma^2 L^2\alpha^{-2})$, which is animprovement from any previous analyses. We also show some essentialapplications of our result to non-convex optimization.

Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization