English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We argue the Fisher information matrix (FIM) of one hidden layer networkswith the ReLU activation function. For a network, let $W$ denote the $d \timesp$ weight matrix from the $d$-dimensional input to the hidden layer consistingof $p$ neurons, and $v$ the $p$-dimensional weight vector from the hidden layerto the scalar output. We focus on the FIM of $v$, which we denote as $I$. Undercertain conditions, we characterize the first three clusters of eigenvalues andeigenvectors of the FIM. Specifically, we show that 1) Since $I$ isnon-negative owing to the ReLU, the first eigenvalue is the Perron-Frobeniuseigenvalue. 2) For the cluster of the next maximum values, the eigenspace isspanned by the row vectors of $W$. 3) The direct sum of the eigenspace of thefirst eigenvalue and that of the third cluster is spanned by the set of all thevectors obtained as the Hadamard product of any pair of the row vectors of $W$.We confirmed by numerical calculation that the above is approximately correctwhen the number of hidden nodes is about 10000.

Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks