English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Knowledge distillation has been widely used to produce portable and efficientneural networks which can be well applied on edge devices for computer visiontasks. However, almost all top-performing knowledge distillation methods needto access the original training data, which usually has a huge size and isoften unavailable. To tackle this problem, we propose a novel data-freeapproach in this paper, named Dual Discriminator Adversarial Distillation(DDAD) to distill a neural network without any training data or meta-data. Tobe specific, we use a generator to create samples through dual discriminatoradversarial distillation, which mimics the original training data. Thegenerator not only uses the pre-trained teacher's intrinsic statistics inexisting batch normalization layers but also obtains the maximum discrepancyfrom the student model. Then the generated samples are used to train thecompact student network under the supervision of the teacher. The proposedmethod obtains an efficient student network which closely approximates itsteacher network, despite using no original training data. Extensive experimentsare conducted to to demonstrate the effectiveness of the proposed approach onCIFAR-10, CIFAR-100 and Caltech101 datasets for classification tasks. Moreover,we extend our method to semantic segmentation tasks on several public datasetssuch as CamVid and NYUv2. All experiments show that our method outperforms allbaselines for data-free knowledge distillation.

Dual Discriminator Adversarial Distillation for Data-free Model Compression