English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent work showed that overparameterized autoencoders can be trained toimplement associative memory via iterative maps, when the trained input-outputJacobian of the network has all of its eigenvalue norms strictly below one.Here, we theoretically analyze this phenomenon for sigmoid networks byleveraging recent developments in deep learning theory, especially thecorrespondence between training neural networks in the infinite-width limit andperforming kernel regression with the Neural Tangent Kernel (NTK). We find thatoverparameterized sigmoid autoencoders can have attractors in the NTK limit forboth training with a single example and multiple examples under certainconditions. In particular, for multiple training examples, we find that thenorm of the largest Jacobian eigenvalue drops below one with increasing inputnorm, leading to associative memory.

Associative Memory in Iterated Overparameterized Sigmoid Autoencoders