English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Stochastic Gradient Descent (SGD) algorithms are widely used in optimizingneural networks, with Random Reshuffling (RR) and Single Shuffle (SS) beingpopular choices for cycling through random or single permutations of thetraining data. However, the convergence properties of these algorithms in thenon-convex case are not fully understood. Existing results suggest that, inrealistic training scenarios where the number of epochs is smaller than thetraining set size, RR may perform worse than SGD.  In this paper, we analyze a general SGD algorithm that allows for arbitrarydata orderings and show improved convergence rates for non-convex functions.Specifically, our analysis reveals that SGD with random and single shuffling isalways faster or at least as good as classical SGD with replacement, regardlessof the number of iterations. Overall, our study highlights the benefits ofusing SGD with random/single shuffling and provides new insights into itsconvergence properties for non-convex optimization.

Shuffle SGD is Always Better than SGD: Improved Analysis of SGD with Arbitrary Data Orders