English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the relationship between the frequency of a function and the speedat which a neural network learns it. We build on recent results that show thatthe dynamics of overparameterized neural networks trained with gradient descentcan be well approximated by a linear system. When normalized training data isuniformly distributed on a hypersphere, the eigenfunctions of this linearsystem are spherical harmonic functions. We derive the correspondingeigenvalues for each frequency after introducing a bias term in the model. Thisbias term had been omitted from the linear network model without significantlyaffecting previous theoretical results. However, we show theoretically andexperimentally that a shallow neural network without bias cannot represent orlearn simple, low frequency functions with odd frequencies. Our results lead tospecific predictions of the time it will take a network to learn functions ofvarying frequency. These predictions match the empirical behavior of bothshallow and deep networks.

The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies