English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The mini-batch versions of StochAstic Recursive grAdient algoritHm andSemi-Stochastic Gradient Descent method, employed the random Barzilai-Borweinstep sizes (shorted as MB-SARAH-RBB and mS2GD-RBB), have surged into prominencethrough timely step size sequence. Inspired by modern adaptors and variancereduction techniques, we propose two new variant rules in the paper, referredto as RHBB and RHBB+, thereby leading to four algorithms MB-SARAH-RHBB,MB-SARAH-RHBB+, mS2GD-RHBB and mS2GD-RHBB+ respectively. RHBB+ is an enhancedversion that additionally incorporates the importance sampling technique. Theyare aggressive in updates, robust in performance and self-adaptive alongiterative periods. We analyze the flexible convergence structures and thecorresponding complexity bounds in strongly convex cases. Comprehensive tuningguidance is theoretically provided for reference in practical implementations.Experiments show that the proposed methods consistently outperform the originaland various state-of-the-art methods on frequently tested data sets. Inparticular, tests on the RHBB+ verify the efficacy of applying the importancesampling technique to the step size level. Numerous explorations display thepromising scalability of our iterative adaptors.

Accelerating Stochastic Recursive and Semi-stochastic Gradient Methods with Adaptive Barzilai-Borwein Step Sizes