English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Echo state networks (ESNs) are a powerful form of reservoir computing thatonly require training of linear output weights whilst the internal reservoir isformed of fixed randomly connected neurons. With a correctly scaledconnectivity matrix, the neurons' activity exhibits the echo-state property andresponds to the input dynamics with certain timescales. Tuning the timescalesof the network can be necessary for treating certain tasks, and someenvironments require multiple timescales for an efficient representation. Herewe explore the timescales in hierarchical ESNs, where the reservoir ispartitioned into two smaller linked reservoirs with distinct properties. Overthree different tasks (NARMA10, a reconstruction task in a volatileenvironment, and psMNIST), we show that by selecting the hyper-parameters ofeach partition such that they focus on different timescales, we achieve asignificant performance improvement over a single ESN. Through a linearanalysis, and under the assumption that the timescales of the first partitionare much shorter than the second's (typically corresponding to optimaloperating conditions), we interpret the feedforward coupling of the partitionsin terms of an effective representation of the input signal, provided by thefirst partition to the second, whereby the instantaneous input signal isexpanded into a weighted combination of its time derivatives. Furthermore, wepropose a data-driven approach to optimise the hyper-parameters through agradient descent optimisation method that is an online approximation ofbackpropagation through time. We demonstrate the application of the onlinelearning rule across all the tasks considered.

Exploiting Multiple Timescales in Hierarchical Echo State Networks