English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently it has been shown that the step sizes of a family of variancereduced gradient methods called the JacSketch methods depend on the expectedsmoothness constant. In particular, if this expected smoothness constant couldbe calculated a priori, then one could safely set much larger step sizes whichwould result in a much faster convergence rate. We fill in this gap, andprovide simple closed form expressions for the expected smoothness constant andcareful numerical experiments verifying these bounds. Using these bounds, andsince the SAGA algorithm is part of this JacSketch family, we suggest a newstandard practice for setting the step sizes and mini-batch size for SAGA thatare competitive with a numerical grid search. Furthermore, we can now show thatthe total complexity of the SAGA algorithm decreases linearly in the mini-batchsize up to a pre-defined value: the optimal mini-batch size. This is a rareresult in the stochastic variance reduced literature, only previously shown forthe Katyusha algorithm. Finally we conjecture that this is the case for manyother stochastic variance reduced methods and that our bounds and analysis ofthe expected smoothness constant is key to extending these results.

Optimal mini-batch and step sizes for SAGA