English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The rapidly growing parameter volume of deep neural networks (DNNs) hindersthe artificial intelligence applications on resource constrained devices, suchas mobile and wearable devices. Neural network pruning, as one of themainstream model compression techniques, is under extensive study to reduce thenumber of parameters and computations. In contrast to irregular pruning thatincurs high index storage and decoding overhead, structured pruning techniqueshave been proposed as the promising solutions. However, prior studies onstructured pruning tackle the problem mainly from the perspective offacilitating hardware implementation, without analyzing the characteristics ofsparse neural networks. The neglect on the study of sparse neural networkscauses inefficient trade-off between regularity and pruning ratio.Consequently, the potential of structurally pruning neural networks is notsufficiently mined.  In this work, we examine the structural characteristics of the irregularlypruned weight matrices, such as the diverse redundancy of different rows, thesensitivity of different rows to pruning, and the positional characteristics ofretained weights. By leveraging the gained insights as a guidance, we firstpropose the novel block-max weight masking (BMWM) method, which can effectivelyretain the salient weights while imposing high regularity to the weight matrix.As a further optimization, we propose a density-adaptive regular-block (DARB)pruning that outperforms prior structured pruning work with high pruning ratioand decoding efficiency. Our experimental results show that DARB can achieve13$\times$ to 25$\times$ pruning ratio, which are 2.8$\times$ to 4.3$\times$improvements than the state-of-the-art counterparts on multiple neural networkmodels and tasks. Moreover, DARB can achieve 14.3$\times$ decoding efficiencythan block pruning with higher pruning ratio.

DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks