English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Structured non-convex learning problems, for which critical points havefavorable statistical properties, arise frequently in statistical machinelearning. Algorithmic convergence and statistical estimation rates arewell-understood for such problems. However, quantifying the uncertaintyassociated with the underlying training algorithm is not well-studied in thenon-convex setting. In order to address this shortcoming, in this work, weestablish an asymptotic normality result for the constant step size stochasticgradient descent (SGD) algorithm--a widely used algorithm in practice.Specifically, based on the relationship between SGD and Markov Chains [DDB19],we show that the average of SGD iterates is asymptotically normally distributedaround the expected value of their unique invariant distribution, as long asthe non-convex and non-smooth objective function satisfies a dissipativityproperty. We also characterize the bias between this expected value and thecritical points of the objective function under various local regularityconditions. Together, the above two results could be leveraged to constructconfidence intervals for non-convex problems that are trained using the SGDalgorithm.

An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias