English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Network pruning is a promising avenue for compressing deep neural networks. Atypical approach to pruning starts by training a model and then removingredundant parameters while minimizing the impact on what is learned.Alternatively, a recent approach shows that pruning can be done atinitialization prior to training, based on a saliency criterion calledconnection sensitivity. However, it remains unclear exactly why pruning anuntrained, randomly initialized neural network is effective. In this work, bynoting connection sensitivity as a form of gradient, we formally characterizeinitialization conditions to ensure reliable connection sensitivitymeasurements, which in turn yields effective pruning results. Moreover, weanalyze the signal propagation properties of the resulting pruned networks andintroduce a simple, data-free method to improve their trainability. Ourmodifications to the existing pruning at initialization method lead to improvedresults on all tested network models for image classification tasks.Furthermore, we empirically study the effect of supervision for pruning anddemonstrate that our signal propagation perspective, combined with unsupervisedpruning, can be useful in various scenarios where pruning is applied tonon-standard arbitrarily-designed architectures.

A Signal Propagation Perspective for Pruning Neural Networks at Initialization