English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Spiking neural networks (SNNs) with leaky integrate and fire (LIF) neurons,can be operated in an event-driven manner and have internal states to retaininformation over time, providing opportunities for energy-efficientneuromorphic computing, especially on edge devices. Note, however, manyrepresentative works on SNNs do not fully demonstrate the usefulness of theirinherent recurrence (membrane potentials retaining information about the past)for sequential learning. Most of the works train SNNs to recognize staticimages by artificially expanded input representation in time through ratecoding. We show that SNNs can be trained for sequential tasks and proposemodifications to a network of LIF neurons that enable internal states to learnlong sequences and make their inherent recurrence resilient to the vanishinggradient problem. We then develop a training scheme to train the proposed SNNswith improved inherent recurrence dynamics. Our training scheme allows spikingneurons to produce multi-bit outputs (as opposed to binary spikes) which helpmitigate the mismatch between a derivative of spiking neurons' activationfunction and a surrogate derivative used to overcome spiking neurons'non-differentiability. Our experimental results indicate that the proposed SNNarchitecture on TIMIT and LibriSpeech 100h dataset yields accuracy comparableto that of LSTMs (within 1.10% and 0.36%, respectively), but with 2x fewerparameters than LSTMs. The sparse SNN outputs also lead to 10.13x and 11.14xsavings in multiplication operations compared to GRUs, which is generallycon-sidered as a lightweight alternative to LSTMs, on TIMIT and LibriSpeech100h datasets, respectively.

Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning