English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We prove a non-asymptotic distribution-independent lower bound for theexpected mean squared generalization error caused by label noise in ridgelesslinear regression. Our lower bound generalizes a similar known result to theoverparameterized (interpolating) regime. In contrast to most previous works,our analysis applies to a broad class of input distributions with almost surelyfull-rank feature matrices, which allows us to cover various types ofdeterministic or random feature maps. Our lower bound is asymptotically sharpand implies that in the presence of label noise, ridgeless linear regressiondoes not perform well around the interpolation threshold for any of thesefeature maps. We analyze the imposed assumptions in detail and provide a theoryfor analytic (random) feature maps. Using this theory, we can show that ourassumptions are satisfied for input distributions with a (Lebesgue) density andfeature maps given by random deep neural networks with analytic activationfunctions like sigmoid, tanh, softplus or GELU. As further examples, we showthat feature maps from random Fourier features and polynomial kernels alsosatisfy our assumptions. We complement our theory with further experimental andanalytic results.

On the Universality of the Double Descent Peak in Ridgeless Regression