English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We provide sharp path-dependent generalization and excess risk guarantees forthe full-batch Gradient Descent (GD) algorithm on smooth losses (possiblynon-Lipschitz, possibly nonconvex). At the heart of our analysis is an upperbound on the generalization error, which implies that average output stabilityand a bounded expected optimization error at termination lead togeneralization. This result shows that a small generalization error occursalong the optimization path, and allows us to bypass Lipschitz or sub-Gaussianassumptions on the loss prevalent in previous works. For nonconvex, convex, andstrongly convex losses, we show the explicit dependence of the generalizationerror in terms of the accumulated path-dependent optimization error, terminaloptimization error, number of samples, and number of iterations. For nonconvexsmooth losses, we prove that full-batch GD efficiently generalizes close to anystationary point at termination, and recovers the generalization errorguarantees of stochastic algorithms with fewer assumptions. For smooth convexlosses, we show that the generalization error is tighter than existing boundsfor SGD (up to one order of error magnitude). Consequently the excess riskmatches that of SGD for quadratically less iterations. Lastly, for stronglyconvex smooth losses, we show that full-batch GD achieves essentially the sameexcess risk rate as compared with the state of the art on SGD, but with anexponentially smaller number of iterations (logarithmic in the dataset size).

Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD