English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The infinitely wide neural network has been proven a useful and manageablemathematical model that enables the understanding of many phenomena appearingin deep learning. One example is the convergence of random deep networks toGaussian processes that allows a rigorous analysis of the way the choice ofactivation function and network weights impacts the training dynamics. In thispaper, we extend the seminal proof of Matthews et al. (2018) to a larger classof initial weight distributions (which we call PSEUDO-IID), including theestablished cases of IID and orthogonal weights, as well as the emerginglow-rank and structured sparse settings celebrated for their computationalspeed-up benefits. We show that fully-connected and convolutional networksinitialized with PSEUDO-IID distributions are all effectively equivalent up totheir variance. Using our results, one can identify the Edge-of-Chaos for abroader class of neural networks and tune them at criticality in order toenhance their training.

Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes