English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent discoveries on neural network pruning reveal that, with a carefullychosen layerwise sparsity, a simple magnitude-based pruning achievesstate-of-the-art tradeoff between sparsity and performance. However, without aclear consensus on "how to choose," the layerwise sparsities are mostlyselected algorithm-by-algorithm, often resorting to handcrafted heuristics oran extensive hyperparameter search. To fill this gap, we propose a novelimportance score for global pruning, coined layer-adaptive magnitude-basedpruning (LAMP) score; the score is a rescaled version of weight magnitude thatincorporates the model-level $\ell_2$ distortion incurred by pruning, and doesnot require any hyperparameter tuning or heavy computation. Under various imageclassification setups, LAMP consistently outperforms popular existing schemesfor layerwise sparsity selection. Furthermore, we observe that LAMP continuesto outperform baselines even in weight-rewinding setups, while theconnectivity-oriented layerwise sparsity (the strongest baseline overall)performs worse than a simple global magnitude-based pruning in this case. Code:https://github.com/jaeho-lee/layer-adaptive-sparsity

Layer-adaptive sparsity for the Magnitude-based Pruning