English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The vanishing gradient problem was a major obstacle for the success of deeplearning. In recent years it was gradually alleviated through multipledifferent techniques. However the problem was not really overcome in afundamental way, since it is inherent to neural networks with activationfunctions based on dot products. In a series of papers, we are going to analyzealternative neural network structures which are not based on dot products. Inthis first paper, we revisit neural networks built up of layers based ondistance measures and Gaussian activation functions. These kinds of networkswere only sparsely used in the past since they are hard to train when usingplain stochastic gradient descent methods. We show that by using Root MeanSquare Propagation (RMSProp) it is possible to efficiently learn multi-layerneural networks. Furthermore we show that when appropriately initialized thesekinds of neural networks suffer much less from the vanishing and explodinggradient problem than traditional neural networks even for deep networks.

Training of Deep Neural Networks based on Distance Measures using RMSProp