English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As deep neural networks are growing in size and being increasingly deployedto more resource-limited devices, there has been a recent surge of interest innetwork pruning methods, which aim to remove less important weights oractivations of a given network. A common limitation of most existing pruningtechniques, is that they require pre-training of the network at least oncebefore pruning, and thus we can benefit from reduction in memory andcomputation only at the inference time. However, reducing the training cost ofneural networks with rapid structural pruning may be beneficial either tominimize monetary cost with cloud computing or to enable on-device learning ona resource-limited device. Recently introduced random-weight pruning approachescan eliminate the needs of pretraining, but they often obtain suboptimalperformance over conventional pruning techniques and also does not allow forfaster training since they perform unstructured pruning. To overcome theirlimitations, we propose Set-based Task-Adaptive Meta Pruning (STAMP), whichtask-adaptively prunes a network pretrained on a large reference dataset bygenerating a pruning mask on it as a function of the target dataset. To ensuremaximum performance improvements on the target task, we meta-learn the maskgenerator over different subsets of the reference dataset, such that it cangeneralize well to any unseen datasets within a few gradient steps of training.We validate STAMP against recent advanced pruning methods on benchmarkdatasets, on which it not only obtains significantly improved compression ratesover the baselines at similar accuracy, but also orders of magnitude fastertraining speed.

Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning