English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Model compression techniques are recently gaining explosive attention forobtaining efficient AI models for various real-time applications. Channelpruning is one important compression strategy and is widely used in slimmingvarious DNNs. Previous gate-based or importance-based pruning methods aim toremove channels whose importance is smallest. However, it remains unclear whatcriteria the channel importance should be measured on, leading to variouschannel selection heuristics. Some other sampling-based pruning methods deploysampling strategies to train sub-nets, which often causes the traininginstability and the compressed model's degraded performance. In view of theresearch gaps, we present a new module named Gates with DifferentiablePolarization (GDP), inspired by principled optimization ideas. GDP can beplugged before convolutional layers without bells and whistles, to control theon-and-off of each channel or whole layer block. During the training process,the polarization effect will drive a subset of gates to smoothly decrease toexact zero, while other gates gradually stay away from zero by a large margin.When training terminates, those zero-gated channels can be painlessly removed,while other non-zero gates can be absorbed into the succeeding convolutionkernel, causing completely no interruption to training nor damage to thetrained model. Experiments conducted over CIFAR-10 and ImageNet datasets showthat the proposed GDP algorithm achieves the state-of-the-art performance onvarious benchmark DNNs at a broad range of pruning ratios. We also apply GDP toDeepLabV3Plus-ResNet50 on the challenging Pascal VOC segmentation task, whosetest performance sees no drop (even slightly improved) with over 60% FLOPssaving.

GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization