English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce a GP generalization of ResNets (including ResNets as aparticular case). We show that ResNets (and their GP generalization) converge,in the infinite depth limit, to a generalization of image registrationvariational algorithms. Whereas computational anatomy aligns images via warpingof the material space, this generalization aligns ideas (or abstract shapes asin Plato's theory of forms) via the warping of the RKHS of functions mappingthe input space to the output space. While the Hamiltonian interpretation ofResNets is not new, it was based on an Ansatz. We do not rely on this Ansatzand present the first rigorous proof of convergence of ResNets with trainedweights and biases towards a Hamiltonian dynamics driven flow. Our constructiveproof reveals several remarkable properties of ResNets and their GPgeneralization. ResNets regressors are kernel regressors with data-dependentwarping kernels. Minimizers of $L_2$ regularized ResNets satisfy a discreteleast action principle implying the near preservation of the norm of weightsand biases across layers. The trained weights of ResNets with $L^2$regularization can be identified by solving an autonomous Hamiltonian system.The trained ResNet parameters are unique up to the initial momentum whoserepresentation is generally sparse. The kernel regularization strategy providesa provably robust alternative to Dropout for ANNs. We introduce a functionalgeneralization of GPs leading to error estimates for ResNets. We identify the(EPDiff) mean fields limit of trained ResNet parameters. We show that thecomposition of warping regression blocks with reduced equivariant multichannelkernels (introduced here) recovers and generalizes CNNs to arbitrary spaces andgroups of transformations.

Do ideas have shape? Idea registration as the continuous limit of artificial neural networks