English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper we study a family of variance reduction methods with randomizedbatch size---at each step, the algorithm first randomly chooses the batch sizeand then selects a batch of samples to conduct a variance-reduced stochasticupdate. We give the linear convergence rate for this framework for compositefunctions, and show that the optimal strategy to achieve the optimalconvergence rate per data access is to always choose batch size of 1, which isequivalent to the SAGA algorithm. However, due to the presence of cache/disk IOeffect in computer architecture, the number of data access cannot reflect therunning time because of 1) random memory access is much slower than sequentialaccess, 2) when data is too big to fit into memory, disk seeking takes evenlonger time. After taking these into account, choosing batch size of $1$ is nolonger optimal, so we propose a new algorithm called SAGA++ and show how tocalculate the optimal average batch size theoretically. Our algorithmoutperforms SAGA and other existing batched and stochastic solvers on realdatasets. In addition, we also conduct a precise analysis to compare differentupdate rules for variance reduction methods, showing that SAGA++ convergesfaster than SVRG in theory.

Fast Variance Reduction Method with Stochastic Batch Size