English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Variance-reduced algorithms, although achieve great theoretical performance,can run slowly in practice due to the periodic gradient estimation with a largebatch of data. Batch-size adaptation thus arises as a promising approach toaccelerate such algorithms. However, existing schemes either apply prescribedbatch-size adaption rule or exploit the information along optimization path viaadditional backtracking and condition verification steps. In this paper, wepropose a novel scheme, which eliminates backtracking line search but stillexploits the information along optimization path by adapting the batch size viahistory stochastic gradients. We further theoretically show that such a schemesubstantially reduces the overall complexity for popular variance-reducedalgorithms SVRG and SARAH/SPIDER for both conventional nonconvex optimizationand reinforcement learning problems. To this end, we develop a new convergenceanalysis framework to handle the dependence of the batch size on historystochastic gradients. Extensive experiments validate the effectiveness of theproposed batch-size adaptation scheme.

History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms