English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Network pruning is a method for reducing test-time computational resourcerequirements with minimal performance degradation. Conventional wisdom ofpruning algorithms suggests that: (1) Pruning methods exploit information fromtraining data to find good subnetworks; (2) The architecture of the prunednetwork is crucial for good performance. In this paper, we conduct sanitychecks for the above beliefs on several recent unstructured pruning methods andsurprisingly find that: (1) A set of methods which aims to find goodsubnetworks of the randomly-initialized network (which we call "initialtickets"), hardly exploits any information from the training data; (2) For thepruned networks obtained by these methods, randomly changing the preservedweights in each layer, while keeping the total number of preserved weightsunchanged per layer, does not affect the final performance. These findingsinspire us to choose a series of simple \emph{data-independent} prune ratiosfor each layer, and randomly prune each layer accordingly to get a subnetwork(which we call "random tickets"). Experimental results show that our zero-shotrandom tickets outperform or attain a similar performance compared to existing"initial tickets". In addition, we identify one existing pruning method thatpasses our sanity checks. We hybridize the ratios in our random ticket withthis method and propose a new method called "hybrid tickets", which achievesfurther improvement. (Our code is publicly available athttps://github.com/JingtongSu/sanity-checking-pruning)

Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot