English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Techniques for reducing the variance of gradient estimates used in stochasticprogramming algorithms for convex finite-sum problems have received a greatdeal of attention in recent years. By leveraging dissipativity theory fromcontrol, we provide a new perspective on two important variance-reductionalgorithms: SVRG and its direct accelerated variant Katyusha. Our perspectiveprovides a physically intuitive understanding of the behavior of SVRG-likemethods via a principle of energy conservation. The tools discussed here allowus to automate the convergence analysis of SVRG-like methods by capturing theiressential properties in small semidefinite programs amenable to standardanalysis and computational techniques. Our approach recovers existingconvergence results for SVRG and Katyusha and generalizes the theory toalternative parameter choices. We also discuss how our approach complements thelinear coupling technique. Our combination of perspectives leads to a betterunderstanding of accelerated variance-reduced stochastic methods for finite-sumproblems.

Dissipativity Theory for Accelerating Stochastic Variance Reduction: A Unified Analysis of SVRG and Katyusha Using Semidefinite Programs