English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The brain cortex, which processes visual, auditory and sensory data in thebrain, is known to have many recurrent connections within its layers and fromhigher to lower layers. But, in the case of machine learning with neuralnetworks, it is generally assumed that strict feed-forward architectures aresuitable for static input data, such as images, whereas recurrent networks arerequired mainly for the processing of sequential input, such as language.However, it is not clear whether also processing of static input data benefitsfrom recurrent connectivity. In this work, we introduce and test a novelimplementation of recurrent neural networks with lateral and feed-backconnections into deep learning. This departure from the strict feed-forwardstructure prevents the use of the standard error backpropagation algorithm fortraining the networks. Therefore we provide an algorithm which implements thebackpropagation algorithm on a implicit implementation of recurrent networks,which is different from state-of-the-art implementations of recurrent neuralnetworks. Our method, in contrast to current recurrent neural networks,eliminates the use of long chains of derivatives due to many iterative updatesteps, which makes learning computationally less costly. It turns out that thepresence of recurrent intra-layer connections within a one-layer implicitrecurrent network enhances the performance of neural networks considerably: Asingle-layer implicit recurrent network is able to solve the XOR problem, whilea feed-forward network with monotonically increasing activation function failsat this task. Finally, we demonstrate that a two-layer implicit recurrentarchitecture leads to a better performance in a regression task of physicalparameters from the measured trajectory of a damped pendulum.

Implicit recurrent networks: A novel approach to stationary input processing with recurrent neural networks in deep learning