English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Data-Free Knowledge Distillation (DFKD) has recently made remarkableadvancements with its core principle of transferring knowledge from a teacherneural network to a student neural network without requiring access to theoriginal data. Nonetheless, existing approaches encounter a significantchallenge when attempting to generate samples from random noise inputs, whichinherently lack meaningful information. Consequently, these models struggle toeffectively map this noise to the ground-truth sample distribution, resultingin the production of low-quality data and imposing substantial timerequirements for training the generator. In this paper, we propose a novelNoisy Layer Generation method (NAYER) which relocates the randomness sourcefrom the input to a noisy layer and utilizes the meaningful label-textembedding (LTE) as the input. The significance of LTE lies in its ability tocontain substantial meaningful inter-class information, enabling the generationof high-quality samples with only a few training steps. Simultaneously, thenoisy layer plays a key role in addressing the issue of diversity in samplegeneration by preventing the model from overemphasizing the constrained labelinformation. By reinitializing the noisy layer in each iteration, we aim tofacilitate the generation of diverse samples while still retaining the method'sefficiency, thanks to the ease of learning provided by LTE. Experiments carriedout on multiple datasets demonstrate that our NAYER not only outperforms thestate-of-the-art methods but also achieves speeds 5 to 15 times faster thanprevious approaches.

Unleash Data Generation for Efficient and Effective Data-free Knowledge Distillation