English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant ofstochastic gradient with momentum where a controlled and properly scaledGaussian noise is added to the stochastic gradients to steer the iteratestowards a global minimum. Many works reported its empirical success in practicefor solving stochastic non-convex optimization problems, in particular it hasbeen observed to outperform overdamped Langevin Monte Carlo-based methods suchas stochastic gradient Langevin dynamics (SGLD) in many applications. Althoughasymptotic global convergence properties of SGHMC are well known, itsfinite-time performance is not well-understood. In this work, we study twovariants of SGHMC based on two alternative discretizations of the underdampedLangevin diffusion. We provide finite-time performance bounds for the globalconvergence of both SGHMC variants for solving stochastic non-convexoptimization problems with explicit constants. Our results lead tonon-asymptotic guarantees for both population and empirical risk minimizationproblems. For a fixed target accuracy level, on a class of non-convex problems,we obtain complexity bounds for SGHMC that can be tighter than those for SGLD.These results show that acceleration with momentum is possible in the contextof global non-convex optimization.

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for  Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and  Momentum-Based Acceleration