English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this work, we consider minimizing the average of a very large number ofsmooth and possibly non-convex functions, and we focus on two widely usedminibatch frameworks to tackle this optimization problem: Incremental Gradient(IG) and Random Reshuffling (RR). We define ease-controlled modifications ofthe IG/RR schemes, which require a light additional computational effort {but}can be proved to converge under {weak} and standard assumptions. In particular,we define two algorithmic schemes in which the IG/RR iteration is controlled byusing a watchdog rule and a derivative-free linesearch that activates onlysporadically to guarantee convergence. The two schemes differ in the watchdogand the linesearch, which are performed using either a monotonic or anon-monotonic rule. The two schemes also allow controlling the updating of thestepsize used in the main IG/RR iteration, avoiding the use of pre-set rulesthat may drive the stepsize to zero too fast, reducing the effort in designingeffective updating rules of the stepsize. We prove convergence under the mildassumption of Lipschitz continuity of the gradients of the component functionsand perform extensive computational analysis using different deep neuralarchitectures and a benchmark of varying-size datasets. We compare ourimplementation with both a full batch gradient method (i.e. L-BFGS) and animplementation of IG/RR methods, proving that our algorithms require a similarcomputational effort compared to the other online algorithms and that thecontrol on the learning rate may allow a faster decrease of the objectivefunction.

Convergence of ease-controlled Random Reshuffling gradient Algorithms under Lipschitz smoothness