English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The paper deals with the distribution of singular values of the input-outputJacobian of deep untrained neural networks in the limit of their infinitewidth. The Jacobian is the product of random matrices where the independentrectangular weight matrices alternate with diagonal matrices whose entriesdepend on the corresponding column of the nearest neighbor weight matrix. Theproblem was considered in \cite{Pe-Co:18} for the Gaussian weights and biasesand also for the weights that are Haar distributed orthogonal matrices andGaussian biases. Basing on a free probability argument, it was claimed that inthese cases the singular value distribution of the Jacobian in the limit ofinfinite width (matrix size) coincides with that of the analog of the Jacobianwith special random but weight independent diagonal matrices, the case wellknown in random matrix theory. The claim was rigorously proved in\cite{Pa-Sl:21} for a quite general class of weights and biases with i.i.d.(including Gaussian) entries by using a version of the techniques of randommatrix theory. In this paper we use another version of the techniques tojustify the claim for random Haar distributed weight matrices and Gaussianbiases.

Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case