English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper proposes a novel knowledge distillation-based learning method toimprove the classification performance of convolutional neural networks (CNNs)without a pre-trained teacher network, called exit-ensemble distillation. Ourmethod exploits the multi-exit architecture that adds auxiliary classifiers(called exits) in the middle of a conventional CNN, through which earlyinference results can be obtained. The idea of our method is to train thenetwork using the ensemble of the exits as the distillation target, whichgreatly improves the classification performance of the overall network. Ourmethod suggests a new paradigm of knowledge distillation; unlike theconventional notion of distillation where teachers only teach students, we showthat students can also help other students and even the teacher to learnbetter. Experimental results demonstrate that our method achieves significantimprovement of classification performance on various popular CNN architectures(VGG, ResNet, ResNeXt, WideResNet, etc.). Furthermore, the proposed method canexpedite the convergence of learning with improved stability. Our code will beavailable on Github.

Students are the Best Teacher: Exit-Ensemble Distillation with Multi-Exits