English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the practical success of deep neural networks, a comprehensivetheoretical framework that can predict practically relevant scores, such as thetest accuracy, from knowledge of the training data is currently lacking. Hugesimplifications arise in the infinite-width limit, where the number of units$N_\ell$ in each hidden layer ($\ell=1,\dots, L$, being $L$ the depth of thenetwork) far exceeds the number $P$ of training examples. This idealisation,however, blatantly departs from the reality of deep learning practice. Here, weuse the toolset of statistical mechanics to overcome these limitations andderive an approximate partition function for fully-connected deep neuralarchitectures, which encodes information about the trained models. Thecomputation holds in the ''thermodynamic limit'' where both $N_\ell$ and $P$are large and their ratio $\alpha_\ell = P/N_\ell$ is finite. This advanceallows us to obtain (i) a closed formula for the generalisation errorassociated to a regression task in a one-hidden layer network with finite$\alpha_1$; (ii) an approximate expression of the partition function for deeparchitectures (via an ''effective action'' that depends on a finite number of''order parameters''); (iii) a link between deep neural networks in theproportional asymptotic limit and Student's $t$ processes.

A statistical mechanics framework for Bayesian deep neural networks beyond the infinite-width limit