English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Convolutional neural networks have been widely deployed in variousapplication scenarios. In order to extend the applications' boundaries to someaccuracy-crucial domains, researchers have been investigating approaches toboost accuracy through either deeper or wider network structures, which bringswith them the exponential increment of the computational and storage cost,delaying the responding time. In this paper, we propose a general trainingframework named self distillation, which notably enhances the performance(accuracy) of convolutional neural networks through shrinking the size of thenetwork rather than aggrandizing it. Different from traditional knowledgedistillation - a knowledge transformation methodology among networks, whichforces student neural networks to approximate the softmax layer outputs ofpre-trained teacher neural networks, the proposed self distillation frameworkdistills knowledge within network itself. The networks are firstly divided intoseveral sections. Then the knowledge in the deeper portion of the networks issqueezed into the shallow ones. Experiments further prove the generalization ofthe proposed self distillation framework: enhancement of accuracy at averagelevel is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 asmaximum. In addition, it can also provide flexibility of depth-wise scalableinference on resource-limited edge devices.Our codes will be released on githubsoon.

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation