English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep neural networks (DNNs) in the infinite width/channel limit have receivedmuch attention recently, as they provide a clear analytical window to deeplearning via mappings to Gaussian Processes (GPs). Despite its theoreticalappeal, this viewpoint lacks a crucial ingredient of deep learning in finiteDNNs, laying at the heart of their success -- feature learning. Here weconsider DNNs trained with noisy gradient descent on a large training set andderive a self consistent Gaussian Process theory accounting for strongfinite-DNN and feature learning effects. Applying this to a toy model of atwo-layer linear convolutional neural network (CNN) shows good agreement withexperiments. We further identify, both analytical and numerically, a sharptransition between a feature learning regime and a lazy learning regime in thismodel. Strong finite-DNN effects are also derived for a non-linear two-layerfully connected network. Our self consistent theory provides a rich andversatile analytical framework for studying feature learning and other non-lazyeffects in finite DNNs.

A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs