English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Knowledge distillation refers to a technique of transferring the knowledgefrom a large learned model or an ensemble of learned models to a small model.This method relies on access to the original training set, which might notalways be available. A possible solution is a data-free adversarialdistillation framework, which deploys a generative network to transfer theteacher model's knowledge to the student model. However, the data generationefficiency is low in the data-free adversarial distillation. We add anactivation regularizer and a virtual interpolation method to improve the datageneration efficiency. The activation regularizer enables the students to matchthe teacher's predictions close to activation boundaries and decisionboundaries. The virtual interpolation method can generate virtual samples andlabels in-between decision boundaries. Our experiments show that our approachsurpasses state-of-the-art data-free distillation methods. The student modelcan achieve 95.42% accuracy on CIFAR-10 and 77.05% accuracy on CIFAR-100without any original training data. Our model's accuracy is 13.8% higher thanthe state-of-the-art data-free method on CIFAR-100.

Enhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolation