English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the implicit regularization of mini-batch stochastic gradientdescent, when applied to the fundamental problem of least squares regression.We leverage a continuous-time stochastic differential equation having the samemoments as stochastic gradient descent, which we call stochastic gradient flow.We give a bound on the excess risk of stochastic gradient flow at time $t$,over ridge regression with tuning parameter $\lambda = 1/t$. The bound may becomputed from explicit constants (e.g., the mini-batch size, step size, numberof iterations), revealing precisely how these quantities drive the excess risk.Numerical examples show the bound can be small, indicating a tight relationshipbetween the two estimators. We give a similar result relating the coefficientsof stochastic gradient flow and ridge. These results hold under no conditionson the data matrix $X$, and across the entire optimization path (not just atconvergence).

The Implicit Regularization of Stochastic Gradient Flow for Least Squares