English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Stochastic gradient methods (SGMs) are predominant approaches for solvingstochastic optimization. On smooth nonconvex problems, a few accelerationtechniques have been applied to improve the convergence rate of SGMs. However,little exploration has been made on applying a certain acceleration techniqueto a stochastic subgradient method (SsGM) for nonsmooth nonconvex problems. Inaddition, few efforts have been made to analyze an (accelerated) SsGM withdelayed derivatives. The information delay naturally happens in a distributedsystem, where computing workers do not coordinate with each other.  In this paper, we propose an inertial proximal SsGM for solving nonsmoothnonconvex stochastic optimization problems. The proposed method can haveguaranteed convergence even with delayed derivative information in adistributed environment. Convergence rate results are established to threeclasses of nonconvex problems: weakly-convex nonsmooth problems with a convexregularizer, composite nonconvex problems with a nonsmooth convex regularizer,and smooth nonconvex problems. For each problem class, the convergence rate is$O(1/K^{\frac{1}{2}})$ in the expected value of the gradient norm square, for$K$ iterations. In a distributed environment, the convergence rate of theproposed method will be slowed down by the information delay. Nevertheless, theslow-down effect will decay with the number of iterations for the latter twoproblem classes. We test the proposed method on three applications. Thenumerical results clearly demonstrate the advantages of using theinertial-based acceleration. Furthermore, we observe higher parallelizationspeed-up in asynchronous updates over the synchronous counterpart, though theformer uses delayed derivatives. Our source code is released athttps://github.com/RPI-OPT/Inertial-SsGM

Distributed stochastic inertial-accelerated methods with delayed derivatives for nonconvex problems