English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We prove a precise geometric description of all one layer ReLU networks$z(x;\theta)$ with a single linear unit and input/output dimensions equal toone that interpolate a given dataset $\mathcal D=\{(x_i,f(x_i))\}$ and, amongall such interpolants, minimize the $\ell_2$-norm of the neuron weights. Suchnetworks can intuitively be thought of as those that minimize the mean-squarederror over $\mathcal D$ plus an infinitesimal weight decay penalty. Wetherefore refer to them as ridgeless ReLU interpolants. Our description provesthat, to extrapolate values $z(x;\theta)$ for inputs $x\in (x_i,x_{i+1})$ lyingbetween two consecutive datapoints, a ridgeless ReLU interpolant simplycompares the signs of the discrete estimates for the curvature of $f$ at $x_i$and $x_{i+1}$ derived from the dataset $\mathcal D$. If the curvature estimatesat $x_i$ and $x_{i+1}$ have different signs, then $z(x;\theta)$ must be linearon $(x_i,x_{i+1})$. If in contrast the curvature estimates at $x_i$ and$x_{i+1}$ are both positive (resp. negative), then $z(x;\theta)$ is convex(resp. concave) on $(x_i,x_{i+1})$. Our results show that ridgeless ReLUinterpolants achieve the best possible generalization for learning $1d$Lipschitz functions, up to universal constants.

Ridgeless Interpolation with Shallow ReLU Networks in $1D$ is Nearest  Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz  Functions