English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the dynamics and implicit bias of gradient flow (GF) on univariateReLU neural networks with a single hidden layer in a binary classificationsetting. We show that when the labels are determined by the sign of a targetnetwork with $r$ neurons, with high probability over the initialization of thenetwork and the sampling of the dataset, GF converges in direction (suitablydefined) to a network achieving perfect training accuracy and having at most$\mathcal{O}(r)$ linear regions, implying a generalization bound. Unlike manyother results in the literature, under an additional assumption on thedistribution of the data, our result holds even for mild over-parameterization,where the width is $\tilde{\mathcal{O}}(r)$ and independent of the sample size.

On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias