English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A challenge of channel pruning is designing efficient and effective criteriato select channels to prune. A widely used criterion is minimal performancedegeneration. To accurately evaluate the truth performance degenerationrequires retraining the survived weights to convergence, which is prohibitivelyslow. Hence existing pruning methods use previous weights (without retraining)to evaluate the performance degeneration. However, we observe the loss changesdiffer significantly with and without retraining. It motivates us to develop atechnique to evaluate true loss changes without retraining, with which channelsto prune can be selected more reliably and confidently. We first derive aclosed-form estimator of the true loss change per pruning mask change, usinginfluence functions without retraining. Influence function which is from robuststatistics reveals the impacts of a training sample on the model's predictionand is repurposed by us to assess impacts on true loss changes. We then showhow to assess the importance of all channels simultaneously and develop a novelglobal channel pruning algorithm accordingly. We conduct extensive experimentsto verify the effectiveness of the proposed algorithm. To the best of ourknowledge, we are the first that shows evaluating true loss changes for pruningwithout retraining is possible. This finding will open up opportunities for aseries of new paradigms to emerge that differ from existing pruning methods.The code is available at https://github.com/hrcheng1066/IFSO.

Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining