English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite their prevalence, deep networks are poorly understood. This is due,at least in part, to their highly parameterized nature. As such, while certainstructures have been found to work better than others, the significance of amodel's unique structure, or the importance of a given layer, and how thesetranslate to overall accuracy, remains unclear. In this paper, we analyze theseproperties of deep neural networks via a process we term deep net triage. Likemedical triage---the assessment of the importance of various wounds---we assessthe importance of layers in a neural network, or as we call it, theircriticality. We do this by applying structural compression, whereby we reduce ablock of layers to a single layer. After compressing a set of layers, we applya combination of initialization and training schemes, and look at networkaccuracy, convergence, and the layer's learned filters to assess thecriticality of the layer. We apply this analysis across four data sets ofvarying complexity. We find that the accuracy of the model does not depend onwhich layer was compressed; that accuracy can be recovered or exceeded aftercompression by fine-tuning across the entire model; and, lastly, that KnowledgeDistillation can be used to hasten convergence of a compressed network, butconstrains the accuracy attainable to that of the base model.

Deep Net Triage: Analyzing the Importance of Network Layers via Structural Compression