English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The paper discusses the use of the Absolute activation function inclassification neural networks. An examples are shown of using this activationfunction in simple and more complex problems. Using as a baseline LeNet-5network for solving the MNIST problem, the efficiency of Absolute activationfunction is shown in comparison with the use of Tanh, ReLU and SeLUactivations. It is shown that in deep networks Absolute activation does notcause vanishing and exploding gradients, and therefore Absolute activation canbe used in both simple and deep neural networks. Due to high volatility oftraining networks with Absolute activation, a special modification of ADAMtraining algorithm is used, that estimates lower bound of accuracy at any testdataset using validation dataset analysis at each training epoch, and uses thisvalue to stop/decrease learning rate, and re-initializes ADAM algorithm betweenthese steps. It is shown that solving the MNIST problem with the LeNet-likearchitectures based on Absolute activation allows to significantly reduce thenumber of trained parameters in the neural network with improving theprediction accuracy.

Improving Classification Neural Networks by using Absolute activation function (MNIST/LeNET-5 example)