English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study generalization properties of random features (RF) regression in highdimensions optimized by stochastic gradient descent (SGD) inunder-/over-parameterized regime. In this work, we derive precisenon-asymptotic error bounds of RF regression under both constant andpolynomial-decay step-size SGD setting, and observe the double descentphenomenon both theoretically and empirically. Our analysis shows how to copewith multiple randomness sources of initialization, label noise, and datasampling (as well as stochastic gradients) with no closed-form solution, andalso goes beyond the commonly-used Gaussian/spherical data assumption. Ourtheoretical results demonstrate that, with SGD training, RF regression stillgeneralizes well for interpolation learning, and is able to characterize thedouble descent behavior by the unimodality of variance and monotonic decreaseof bias. Besides, we also prove that the constant step-size SGD setting incursno loss in convergence rate when compared to the exact minimum-norminterpolator, as a theoretical justification of using SGD in practice.

On the Double Descent of Random Features Models Trained with SGD