English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the recent past, a property of neural training trajectories inweight-space had been isolated, that of "local elasticity" (denoted as $S_{\rmrel}$). Local elasticity attempts to quantify the propagation of the influenceof a sampled data point on the prediction at another data. In this work, weembark on a comprehensive study of the existing notion of $S_{\rm rel}$ andalso propose a new definition that addresses the limitations that we point outfor the original definition in the classification setting. On variousstate-of-the-art neural network training on SVHN, CIFAR-10 and CIFAR-100 wedemonstrate how our new proposal of $S_{\rm rel}$, as opposed to the originaldefinition, much more sharply detects the property of the weight updatespreferring to make prediction changes within the same class as the sampleddata.  In neural regression experiments we demonstrate that the original $S_{\rmrel}$ reveals a $2-$phase behavior -- that the training proceeds via an initialelastic phase when $S_{\rm rel}$ changes rapidly and an eventual inelasticphase when $S_{\rm rel}$ remains large. We show that some of these propertiescan be analytically reproduced in various instances of doing regression viagradient flows on model predictor classes.

Dynamics of Local Elasticity During Training of Neural Nets