English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Stochastic gradient descent with momentum (SGDM) is the dominant algorithm inmany optimization scenarios, including convex optimization instances andnon-convex neural network training. Yet, in the stochastic setting, momentuminterferes with gradient noise, often leading to specific step size andmomentum choices in order to guarantee convergence, set aside acceleration.Proximal point methods, on the other hand, have gained much attention due totheir numerical stability and elasticity against imperfect tuning. Theirstochastic accelerated variants though have received limited attention: howmomentum interacts with the stability of (stochastic) proximal point methodsremains largely unstudied. To address this, we focus on the convergence andstability of the stochastic proximal point algorithm with momentum (SPPAM), andshow that SPPAM allows a faster linear convergence to a neighborhood comparedto the stochastic proximal point algorithm (SPPA) with a better contractionfactor, under proper hyperparameter tuning. In terms of stability, we show thatSPPAM depends on problem constants more favorably than SGDM, allowing a widerrange of step size and momentum that lead to convergence.

Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum