English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Supralinear and sublinear pre-synaptic and dendritic integration isconsidered to be responsible for nonlinear computation power of biologicalneurons, emphasizing the role of nonlinear integration as opposed to nonlinearoutput thresholding. How, why, and to what degree the transfer functionnonlinearity helps biologically inspired neural network models is not fullyunderstood. Here, we study these questions in the context of echo statenetworks (ESN). ESN is a simple neural network architecture in which a fixedrecurrent network is driven with an input signal, and the output is generatedby a readout layer from the measurements of the network states. ESNarchitecture enjoys efficient training and good performance on certainsignal-processing tasks, such as system identification and time seriesprediction. ESN performance has been analyzed with respect to the connectivitypattern in the network structure and the input bias. However, the effects ofthe transfer function in the network have not been studied systematically.Here, we use an approach tanh on the Taylor expansion of a frequently usedtransfer function, the hyperbolic tangent function, to systematically study theeffect of increasing nonlinearity of the transfer function on the memory,nonlinear capacity, and signal processing performance of ESN. Interestingly, wefind that a quadratic approximation is enough to capture the computationalpower of ESN with tanh function. The results of this study apply to bothsoftware and hardware implementation of ESN.

Exploring Transfer Function Nonlinearity in Echo State Networks