English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We analyze the convergence of the averaged stochastic gradient descent foroverparameterized two-layer neural networks for regression problems. It wasrecently found that a neural tangent kernel (NTK) plays an important role inshowing the global convergence of gradient-based methods under the NTK regime,where the learning dynamics for overparameterized neural networks can be almostcharacterized by that for the associated reproducing kernel Hilbert space(RKHS). However, there is still room for a convergence rate analysis in the NTKregime. In this study, we show that the averaged stochastic gradient descentcan achieve the minimax optimal convergence rate, with the global convergenceguarantee, by exploiting the complexities of the target function and the RKHSassociated with the NTK. Moreover, we show that the target function specifiedby the NTK of a ReLU network can be learned at the optimal convergence ratethrough a smooth approximation of a ReLU network under certain conditions.

Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime