English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Spiking Neural Networks (SNNs) have emerged as a popular spatio-temporalcomputing paradigm for complex vision tasks. Recently proposed SNN trainingalgorithms have significantly reduced the number of time steps (down to 1) forimproved latency and energy efficiency, however, they target only convolutionalneural networks (CNN). These algorithms, when applied on the recentlyspotlighted vision transformers (ViT), either require a large number of timesteps or fail to converge. Based on analysis of the histograms of the ANN andSNN activation maps, we hypothesize that each ViT block has a differentsensitivity to the number of time steps. We propose a novel training frameworkthat dynamically allocates the number of time steps to each ViT moduledepending on a trainable score assigned to each timestep. In particular, wegenerate a scalar binary time step mask that filters spikes emitted by eachneuron in a leaky-integrate-and-fire (LIF) layer. The resulting SNNs have highactivation sparsity and require only accumulate operations (AC), except for theinput embedding layer, in contrast to expensive multiply-and-accumulates (MAC)needed in traditional ViTs. This yields significant improvements in energyefficiency. We evaluate our training framework and resulting SNNs on imagerecognition tasks including CIFAR10, CIFAR100, and ImageNet with different ViTarchitectures. We obtain a test accuracy of 95.97% with 4.97 time steps withdirect encoding on CIFAR10.

Spiking Neural Networks with Dynamic Time Steps for Vision Transformers