English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the eigenvalue distributions of the Conjugate Kernel and NeuralTangent Kernel associated to multi-layer feedforward neural networks. In anasymptotic regime where network width is increasing linearly in sample size,under random initialization of the weights, and for input samples satisfying anotion of approximate pairwise orthogonality, we show that the eigenvaluedistributions of the CK and NTK converge to deterministic limits. The limit forthe CK is described by iterating the Marcenko-Pastur map across the hiddenlayers. The limit for the NTK is equivalent to that of a linear combination ofthe CK matrices across layers, and may be described by recursive fixed-pointequations that extend this Marcenko-Pastur map. We demonstrate the agreement ofthese asymptotic predictions with the observed spectra for both synthetic andCIFAR-10 training data, and we perform a small simulation to investigate theevolutions of these spectra over training.

Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks