English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Second-order information, in the form of Hessian- or Inverse-Hessian-vectorproducts, is a fundamental tool for solving optimization problems. Recently,there has been significant interest in utilizing this information in thecontext of deep neural networks; however, relatively little is known about thequality of existing approximations in this context. Our work examines thisquestion, identifies issues with existing approaches, and proposes a methodcalled WoodFisher to compute a faithful and efficient estimate of the inverseHessian.  Our main application is to neural network compression, where we build on theclassic Optimal Brain Damage/Surgeon framework. We demonstrate that WoodFishersignificantly outperforms popular state-of-the-art methods for one-shotpruning. Further, even when iterative, gradual pruning is considered, ourmethod results in a gain in test accuracy over the state-of-the-art approaches,for pruning popular neural networks (like ResNet-50, MobileNetV1) trained onstandard image classification datasets such as ImageNet ILSVRC. We examine howour method can be extended to take into account first-order information, aswell as illustrate its ability to automatically set layer-wise pruningthresholds and perform compression in the limited-data regime. The code isavailable at the following link, https://github.com/IST-DASLab/WoodFisher.

WoodFisher: Efficient Second-Order Approximation for Neural Network Compression