English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Activation sparsity improves compute efficiency and resource utilization insparsity-aware neural network accelerators. As the predominant operation inDNNs is multiply-accumulate (MAC) of activations with weights to compute innerproducts, skipping operations where (at least) one of the two operands is zerocan make inference more efficient in terms of latency and power. Spatialsparsification of activations is a popular topic in DNN literature and severalmethods have already been established to bias a DNN for it. On the other hand,temporal sparsity is an inherent feature of bio-inspired spiking neuralnetworks (SNNs), which neuromorphic processing exploits for hardwareefficiency. Introducing and exploiting spatio-temporal sparsity, is a topicmuch less explored in DNN literature, but in perfect resonance with the trendin DNN, to shift from static signal processing to more streaming signalprocessing. Towards this goal, in this paper we introduce a new DNN layer(called Delta Activation Layer), whose sole purpose is to promote temporalsparsity of activations during training. A Delta Activation Layer caststemporal sparsity into spatial activation sparsity to be exploited whenperforming sparse tensor multiplications in hardware. By employing deltainference and ``the usual'' spatial sparsification heuristics during training,the resulting model learns to exploit not only spatial but also temporalactivation sparsity (for a given input data distribution). One may use theDelta Activation Layer either during vanilla training or during a refinementphase. We have implemented Delta Activation Layer as an extension of thestandard Tensoflow-Keras library, and applied it to train deep neural networkson the Human Action Recognition (UCF101) dataset. We report an almost 3ximprovement of activation sparsity, with recoverable loss of model accuracyafter longer training.

Training for temporal sparsity in deep neural networks, application in video processing