English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we propose a unified view of gradient-based algorithms forstochastic convex composite optimization by extending the concept of estimatesequence introduced by Nesterov. More precisely, we interpret a large class ofstochastic optimization methods as procedures that iteratively minimize asurrogate of the objective, which covers the stochastic gradient descent methodand variants of the incremental approaches SAGA, SVRG, and MISO/Finito/SDCA.This point of view has several advantages: (i) we provide a simple genericproof of convergence for all of the aforementioned methods; (ii) we naturallyobtain new algorithms with the same guarantees; (iii) we derive genericstrategies to make these algorithms robust to stochastic noise, which is usefulwhen data is corrupted by small random perturbations. Finally, we propose a newaccelerated stochastic gradient descent algorithm and an accelerated SVRGalgorithm with optimal complexity that is robust to stochastic noise.

Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise