English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The state of neural network pruning has been noticed to be unclear and evenconfusing for a while, largely due to "a lack of standardized benchmarks andmetrics" [3]. To standardize benchmarks, first, we need to answer: what kind ofcomparison setup is considered fair? This basic yet crucial question has barelybeen clarified in the community, unfortunately. Meanwhile, we observe severalpapers have used (severely) sub-optimal hyper-parameters in pruningexperiments, while the reason behind them is also elusive. These sub-optimalhyper-parameters further exacerbate the distorted benchmarks, rendering thestate of neural network pruning even more obscure.  Two mysteries in pruning represent such a confusing status: theperformance-boosting effect of a larger finetuning learning rate, and theno-value argument of inheriting pretrained weights in filter pruning.  In this work, we attempt to explain the confusing state of network pruning bydemystifying the two mysteries. Specifically, (1) we first clarify the fairnessprinciple in pruning experiments and summarize the widely-used comparisonsetups; (2) then we unveil the two pruning mysteries and point out the centralrole of network trainability, which has not been well recognized so far; (3)finally, we conclude the paper and give some concrete suggestions regarding howto calibrate the pruning benchmarks in the future. Code:https://github.com/mingsun-tse/why-the-state-of-pruning-so-confusing.

Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning