English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent theoretical work has guaranteed that overparameterized networkstrained by gradient descent achieve arbitrarily low training error, andsometimes even low test error. The required width, however, is alwayspolynomial in at least one of the sample size $n$, the (inverse) target error$1/\epsilon$, and the (inverse) failure probability $1/\delta$. This work showsthat $\widetilde{\Theta}(1/\epsilon)$ iterations of gradient descent with$\widetilde{\Omega}(1/\epsilon^2)$ training examples on two-layer ReLU networksof any width exceeding $\mathrm{polylog}(n,1/\epsilon,1/\delta)$ suffice toachieve a test misclassification error of $\epsilon$. We also prove thatstochastic gradient descent can achieve $\epsilon$ test error withpolylogarithmic width and $\widetilde{\Theta}(1/\epsilon)$ samples. Theanalysis relies upon the separation margin of the limiting kernel, which isguaranteed positive, can distinguish between true labels and random labels, andcan give a tight sample-complexity analysis in the infinite-width setting

Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks