English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning empirically achieves high performance in many applications, butits training dynamics has not been fully understood theoretically. In thispaper, we explore theoretical analysis on training two-layer ReLU neuralnetworks in a teacher-student regression model, in which a student networklearns an unknown teacher network through its outputs. We show that with aspecific regularization and sufficient over-parameterization, the studentnetwork can identify the parameters of the teacher network with highprobability via gradient descent with a norm dependent stepsize even though theobjective function is highly non-convex. The key theoretical tool is themeasure representation of the neural networks and a novel application of a dualcertificate argument for sparse estimation on a measure space. We analyze theglobal minima and global convergence property in the measure space.

On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Setting