English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Stochastic gradient descent (SGD) is a simple and popular method to solvestochastic optimization problems which arise in machine learning. For stronglyconvex problems, its convergence rate was known to be O(\log(T)/T), by runningSGD for T iterations and returning the average point. However, recent resultsshowed that using a different algorithm, one can get an optimal O(1/T) rate.This might lead one to believe that standard SGD is suboptimal, and maybeshould even be replaced as a method of choice. In this paper, we investigatethe optimality of SGD in a stochastic setting. We show that for smoothproblems, the algorithm attains the optimal O(1/T) rate. However, fornon-smooth problems, the convergence rate with averaging might really be\Omega(\log(T)/T), and this is not just an artifact of the analysis. On theflip side, we show that a simple modification of the averaging step suffices torecover the O(1/T) rate, and no other change of the algorithm is necessary. Wealso present experimental results which support our findings, and point outopen problems.

Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization