English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pruning is a popular technique for reducing the model size and computationalcost of convolutional neural networks (CNNs). However, a slow retraining orfine-tuning procedure is often required to recover the accuracy loss caused bypruning. Recently, a new research direction on weight pruning,pruning-at-initialization (PAI), is proposed to directly prune CNNs beforetraining so that fine-tuning or retraining can be avoided. While PAI has shownpromising results in reducing the model size, existing approaches rely onfine-grained weight pruning which requires unstructured sparse matrixcomputation, making it difficult to achieve real speedup in practice unless thesparsity is very high. This work is the first to show that fine-grained weightpruning is in fact not necessary for PAI. Instead, the layerwise compressionratio is the main critical factor to determine the accuracy of a CNN modelpruned at initialization. Based on this key observation, we proposePreCropping, a structured hardware-efficient model compression scheme.PreCropping directly compresses the model at the channel level following thelayerwise compression ratio. Compared to weight pruning, the proposed scheme isregular and dense in both storage and computation without sacrificing accuracy.In addition, since PreCropping compresses CNNs at initialization, thecomputational and memory costs of CNNs are reduced for both training andinference on commodity hardware. We empirically demonstrate our approaches onseveral modern CNN architectures, including ResNet, ShuffleNet, and MobileNetfor both CIFAR-10 and ImageNet.

Structured Pruning is All You Need for Pruning CNNs at Initialization