English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the asymptotic behavior of gradient-descent training of deepneural networks is essential for revealing inductive biases and improvingnetwork performance. We derive the infinite-time training limit of amathematically tractable class of deep nonlinear neural networks, gated linearnetworks (GLNs), and generalize these results to gated networks described bygeneral homogeneous polynomials. We study the implications of our results,focusing first on two-layer GLNs. We then apply our theoretical predictions toGLNs trained on MNIST and show how architectural constraints and the implicitbias of gradient descent affect performance. Finally, we show that our theorycaptures a substantial portion of the inductive bias of ReLU networks. Bymaking the inductive bias explicit, our framework is poised to inform thedevelopment of more efficient, biologically plausible, and robust learningalgorithms.

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks