English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The study of sparsity in Convolutional Neural Networks (CNNs) has becomewidespread to compress and accelerate models in environments with limitedresources. By constraining N consecutive weights along the output channel to begroup-wise non-zero, the recent network with 1$\times$N sparsity has receivedtremendous popularity for its three outstanding advantages: 1) A large amountof storage space saving by a \emph{Block Sparse Row} matrix. 2) Excellentperformance at a high sparsity. 3) Significant speedups on CPUs with AdvancedVector Extensions. Recent work requires selecting and fine-tuning 1$\times$Nsparse weights based on dense pre-trained weights, leading to the problems suchas expensive training cost and memory access, sub-optimal model quality, aswell as unbalanced workload across threads (different sparsity across outputchannels). To overcome them, this paper proposes a novel \emph{\textbf{S}oft\textbf{U}niform \textbf{B}lock \textbf{P}runing} (SUBP) approach to train auniform 1$\times$N sparse structured network from scratch. Specifically, ourapproach tends to repeatedly allow pruned blocks to regrow to the network basedon block angular redundancy and importance sampling in a uniform mannerthroughout the training process. It not only makes the model less dependent onpre-training, reduces the model redundancy and the risk of pruning theimportant blocks permanently but also achieves balanced workload. Empirically,on ImageNet, comprehensive experiments across various CNN architectures showthat our SUBP consistently outperforms existing 1$\times$N and structuredsparsity methods based on pre-trained models or training from scratch. Sourcecodes and models are available at \url{https://github.com/JingyangXiang/SUBP}.

SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration