English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent works have examined theoretical and empirical properties of wideneural networks trained in the Neural Tangent Kernel (NTK) regime. Given thatbiological neural networks are much wider than their artificial counterparts,we consider NTK regime wide neural networks as a possible model of biologicalneural networks. Leveraging NTK theory, we show theoretically that gradientdescent drives layerwise weight updates that are aligned with their inputactivity correlations weighted by error, and demonstrate empirically that theresult also holds in finite-width wide networks. The alignment result allows usto formulate a family of biologically-motivated, backpropagation-free learningrules that are theoretically equivalent to backpropagation in infinite-widthnetworks. We test these learning rules on benchmark problems in feedforward andrecurrent neural networks and demonstrate, in wide networks, comparableperformance to backpropagation. The proposed rules are particularly effectivein low data regimes, which are common in biological learning settings.

How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective