English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Despite recent theoretical progress on the non-convex optimization oftwo-layer neural networks, it is still an open question whether gradientdescent on neural networks without unnatural modifications can achieve bettersample complexity than kernel methods. This paper provides a clean mean-fieldanalysis of projected gradient flow on polynomial-width two-layer neuralnetworks. Different from prior works, our analysis does not require unnaturalmodifications of the optimization algorithm. We prove that with sample size $n= O(d^{3.1})$ where $d$ is the dimension of the inputs, the network trainedwith projected gradient flow converges in $\text{poly}(d)$ time to anon-trivial error that is not achievable by kernel methods using $n \ll d^4$samples, hence demonstrating a clear separation between unmodified gradientdescent and NTK. As a corollary, we show that projected gradient descent with apositive learning rate and a polynomial number of iterations converges to lowerror with the same sample complexity.

Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time