English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The event-driven and sparse nature of communication between spiking neuronsin the brain holds great promise for flexible and energy-efficient AI. Recentadvances in learning algorithms have demonstrated that recurrent networks ofspiking neurons can be effectively trained to achieve competitive performancecompared to standard recurrent neural networks. Still, as these learningalgorithms use error-backpropagation through time (BPTT), they suffer from highmemory requirements, are slow to train, and are incompatible with onlinelearning. This limits the application of these learning algorithms torelatively small networks and to limited temporal sequence lengths. Onlineapproximations to BPTT with lower computational and memory complexity have beenproposed (e-prop, OSTL), but in practice also suffer from memory limitationsand, as approximations, do not outperform standard BPTT training. Here, we showhow a recently developed alternative to BPTT, Forward Propagation Through Time(FPTT) can be applied in spiking neural networks. Different from BPTT, FPTTattempts to minimize an ongoing dynamically regularized risk on the loss. As aresult, FPTT can be computed in an online fashion and has fixed complexity withrespect to the sequence length. When combined with a novel dynamic spikingneuron model, the Liquid-Time-Constant neuron, we show that SNNs trained withFPTT outperform online BPTT approximations, and approach or exceed offline BPTTaccuracy on temporal classification tasks. This approach thus makes it feasibleto train SNNs in a memory-friendly online fashion on long sequences and scaleup SNNs to novel and complex neural architectures.

Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time