English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the stability of accuracy during the training of deep neuralnetworks (DNNs). In this context, the training of a DNN is performed via theminimization of a cross-entropy loss function, and the performance metric isaccuracy (the proportion of objects that are classified correctly). Whiletraining results in a decrease of loss, the accuracy does not necessarilyincrease during the process and may sometimes even decrease. The goal ofachieving stability of accuracy is to ensure that if accuracy is high at someinitial time, it remains high throughout training.  A recent result by Berlyand, Jabin, and Safsten introduces a doublingcondition on the training data, which ensures the stability of accuracy duringtraining for DNNs using the absolute value activation function. For trainingdata in $\mathbb{R}^n$, this doubling condition is formulated using slabs in$\mathbb{R}^n$ and depends on the choice of the slabs. The goal of this paperis twofold. First, to make the doubling condition uniform, that is, independentof the choice of slabs. This leads to sufficient conditions for stability interms of training data only. In other words, for a training set $T$ thatsatisfies the uniform doubling condition, there exists a family of DNNs suchthat a DNN from this family with high accuracy on the training set at sometraining time $t_0$ will have high accuracy for all time $t>t_0$. Moreover,establishing uniformity is necessary for the numerical implementation of thedoubling condition.  The second goal is to extend the original stability results from the absolutevalue activation function to a broader class of piecewise linear activationfunctions with finitely many critical points, such as the popular Leaky ReLU.

Stability of Accuracy for the Training of DNNs Via the Uniform Doubling Condition