English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This study explores the sample complexity for two-layer neural networks tolearn a single-index target function under Stochastic Gradient Descent (SGD),focusing on the challenging regime where many flat directions are present atinitialization. It is well-established that in this scenario $n=O(d\log{d})$samples are typically needed. However, we provide precise results concerningthe pre-factors in high-dimensional contexts and for varying widths. Notably,our findings suggest that overparameterization can only enhance convergence bya constant factor within this problem class. These insights are grounded in thereduction of SGD dynamics to a stochastic process in lower dimensions, whereescaping mediocrity equates to calculating an exit time. Yet, we demonstratethat a deterministic approximation of this process adequately represents theescape time, implying that the role of stochasticity may be minimal in thisscenario.

Escaping mediocrity: how two-layer networks learn hard single-index models with SGD