English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The stochastic gradient descent (SGD) method is most widely used for deepneural network (DNN) training. However, the method does not always converge toa flat minimum of the loss surface that can demonstrate high generalizationcapability. Weight noise injection has been extensively studied for findingflat minima using the SGD method. We devise a new weight-noise injection-basedSGD method that adds symmetrical noises to the DNN weights. The training withsymmetrical noise evaluates the loss surface at two adjacent points, by whichconvergence to sharp minima can be avoided. Fixed-magnitude symmetric noisesare added to minimize training instability. The proposed method is comparedwith the conventional SGD method and previous weight-noise injection algorithmsusing convolutional neural networks for image classification. Particularly,performance improvements in large batch training are demonstrated. This methodshows superior performance compared with conventional SGD and weight-noiseinjection methods regardless of the batch-size and learning rate schedulingalgorithms.

S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima