English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the rapid scaling up of deep neural networks (DNNs), extensive researchstudies on network model compression such as weight pruning have been performedfor improving deployment efficiency. This work aims to advance the compressionbeyond the weights to neuron activations. We propose the joint regularizationtechnique which simultaneously regulates the distribution of weights andactivations. By distinguishing and leveraging the significance difference amongneuron responses and connections during learning, the jointly pruned network,namely \textit{JPnet}, optimizes the sparsity of activations and weights forimproving execution efficiency. The derived deep sparsification of JPnetreveals more optimization space for the existing DNN accelerators dedicated forsparse matrix operations. We thoroughly evaluate the effectiveness of jointregularization through various network models with different activationfunctions and on different datasets. With $0.4\%$ degradation constraint oninference accuracy, a JPnet can save $72.3\% \sim 98.8\%$ of computation costcompared to the original dense models, with up to $5.2\times$ and $12.3\times$reductions in activation and weight numbers, respectively.

Joint Regularization on Activations and Weights for Efficient Neural Network Pruning