English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper we approach the problem of unique and stable identifiability ofgeneric deep artificial neural networks with pyramidal shape and smoothactivation functions from a finite number of input-output samples. Morespecifically we introduce the so-called entangled weights, which composeweights of successive layers intertwined with suitable diagonal and invertiblematrices depending on the activation functions and their shifts. We prove thatentangled weights are completely and stably approximated by an efficient androbust algorithm as soon as $\mathcal O(D^2 \times m)$ nonadaptive input-outputsamples of the network are collected, where $D$ is the input dimension and $m$is the number of neurons of the network. Moreover, we empirically observe thatthe approach applies to networks with up to $\mathcal O(D \times m_L)$ neurons,where $m_L$ is the number of output neurons at layer $L$. Provided knowledge oflayer assignments of entangled weights and of remaining scaling and shiftparameters, which may be further heuristically obtained by least squares, theentangled weights identify the network completely and uniquely. To highlightthe relevance of the theoretical result of stable recovery of entangledweights, we present numerical experiments, which demonstrate that multilayerednetworks with generic weights can be robustly identified and thereforeuniformly approximated by the presented algorithmic pipeline. In contrastbackpropagation cannot generalize stably very well in this setting, beingalways limited by relatively large uniform error. In terms of practical impact,our study shows that we can relate input-output information uniquely and stablyto network parameters, providing a form of explainability. Moreover, our methodpaves the way for compression of overparametrized networks and for the trainingof minimal complexity networks.

Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples