English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While stochastic gradient descent (SGD) is one of the major workhorses inmachine learning, the learning properties of many practically used variants arepoorly understood. In this paper, we consider least squares learning in anonparametric setting and contribute to filling this gap by focusing on theeffect and interplay of multiple passes, mini-batching and averaging, and inparticular tail averaging. Our results show how these different variants of SGDcan be combined to achieve optimal learning errors, hence providing practicalinsights. In particular, we show for the first time in the literature that tailaveraging allows faster convergence rates than uniform averaging in thenonparametric setting. Finally, we show that a combination of tail-averagingand minibatching allows more aggressive step-size choices than using any one ofsaid components.

Beating SGD Saturation with Tail-Averaging and Minibatching