English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce a simple yet effective distillation framework that is able toboost the vanilla ResNet-50 to 80%+ Top-1 accuracy on ImageNet without tricks.We construct such a framework through analyzing the problems in the existingclassification system and simplify the base method ensemble knowledgedistillation via discriminators by: (1) adopting the similarity loss anddiscriminator only on the final outputs and (2) using the average of softmaxprobabilities from all teacher ensembles as the stronger supervision.Intriguingly, three novel perspectives are presented for distillation: (1)weight decay can be weakened or even completely removed since the soft labelalso has a regularization effect; (2) using a good initialization for studentsis critical; and (3) one-hot/hard label is not necessary in the distillationprocess if the weights are well initialized. We show that such astraight-forward framework can achieve state-of-the-art results withoutinvolving any commonly-used techniques, such as architecture modification;outside training data beyond ImageNet; autoaug/randaug; cosine learning rate;mixup/cutmix training; label smoothing; etc. Our method obtains 80.67% top-1accuracy on ImageNet using a single crop-size of 224x224 with vanillaResNet-50, outperforming the previous state-of-the-arts by a significant marginunder the same network structure. Our result can be regarded as a strongbaseline using knowledge distillation, and to our best knowledge, this is alsothe first method that is able to boost vanilla ResNet-50 to surpass 80% onImageNet without architecture modification or additional training data. Onsmaller ResNet-18, our distillation framework consistently improves from 69.76%to 73.19%, which shows tremendous practical values in real-world applications.Our code and models are available at: https://github.com/szq0214/MEAL-V2.

MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks