English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Weight pruning and weight quantization are two important categories of DNNmodel compression. Prior work on these techniques are mainly based onheuristics. A recent work developed a systematic frame-work of DNN weightpruning using the advanced optimization technique ADMM (Alternating DirectionMethods of Multipliers), achieving one of state-of-art in weight pruningresults. In this work, we first extend such one-shot ADMM-based framework toguarantee solution feasibility and provide fast convergence rate, andgeneralize to weight quantization as well. We have further developed amulti-step, progressive DNN weight pruning and quantization framework, withdual benefits of (i) achieving further weight pruning/quantization thanks tothe special property of ADMM regularization, and (ii) reducing the search spacewithin each step. Extensive experimental results demonstrate the superiorperformance compared with prior work. Some highlights: (i) we achieve 246x,36x,and 8x weight pruning on LeNet-5, AlexNet, and ResNet-50 models, respectively,with (almost) zero accuracy loss; (ii) even a significant 61x weight pruning inAlexNet (ImageNet) results in only minor degradation in actual accuracycompared with prior work; (iii) we are among the first to derive notable weightpruning results for ResNet and MobileNet models; (iv) we derive the firstlossless, fully binarized (for all layers) LeNet-5 for MNIST and VGG-16 forCIFAR-10; and (v) we derive the first fully binarized (for all layers) ResNetfor ImageNet with reasonable accuracy loss.

Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM