English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As the third-generation neural network, the Spiking Neural Network (SNN) hasthe advantages of low power consumption and high energy efficiency, making itsuitable for implementation on edge devices. More recently, the most advancedSNN, Spikformer, combines the self-attention module from Transformer with SNNto achieve remarkable performance. However, it adopts larger channel dimensionsin MLP layers, leading to an increased number of redundant model parameters. Toeffectively decrease the computational complexity and weight parameters of themodel, we explore the Lottery Ticket Hypothesis (LTH) and discover a verysparse ($\ge$90%) subnetwork that achieves comparable performance to theoriginal network. Furthermore, we also design a lightweight token selectormodule, which can remove unimportant background information from images basedon the average spike firing rate of neurons, selecting only essentialforeground image tokens to participate in attention calculation. Based on that,we present SparseSpikformer, a co-design framework aimed at achieving sparsityin Spikformer through token and weight pruning techniques. Experimental resultsdemonstrate that our framework can significantly reduce 90% model parametersand cut down Giga Floating-Point Operations (GFLOPs) by 20% while maintainingthe accuracy of the original model.

SparseSpikformer: A Co-Design Framework for Token and Weight Pruning in Spiking Transformer