English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We develop a variational framework to understand the properties of functionslearned by fitting deep neural networks with rectified linear unit activationsto data. We propose a new function space, which is reminiscent of classicalbounded variation-type spaces, that captures the compositional structureassociated with deep neural networks. We derive a representer theorem showingthat deep ReLU networks are solutions to regularized data fitting problems overfunctions from this space. The function space consists of compositions offunctions from the Banach spaces of second-order bounded variation in the Radondomain. These are Banach spaces with sparsity-promoting norms, giving insightinto the role of sparsity in deep neural networks. The neural network solutionshave skip connections and rank bounded weight matrices, providing newtheoretical support for these common architectural choices. The variationalproblem we study can be recast as a finite-dimensional neural network trainingproblem with regularization schemes related to the notions of weight decay andpath-norm regularization. Finally, our analysis builds on techniques fromvariational spline theory, providing new connections between deep neuralnetworks and splines.

What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory