English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many iterative procedures in stochastic optimization exhibit a transientphase followed by a stationary phase. During the transient phase the procedureconverges towards a region of interest, and during the stationary phase theprocedure oscillates in that region, commonly around a single point. In thispaper, we develop a statistical diagnostic test to detect such phase transitionin the context of stochastic gradient descent with constant learning rate. Wepresent theory and experiments suggesting that the region where the proposeddiagnostic is activated coincides with the convergence region. For a class ofloss functions, we derive a closed-form solution describing such region.Finally, we suggest an application to speed up convergence of stochasticgradient descent by halving the learning rate each time stationarity isdetected. This leads to a new variant of stochastic gradient descent, which inmany settings is comparable to state-of-art.

Convergence diagnostics for stochastic gradient descent with constant step size