English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent mainstream masked distillation methods function by reconstructingselectively masked areas of a student network from the feature map of itsteacher counterpart. In these methods, the masked regions need to be properlyselected, such that reconstructed features encode sufficient discrimination andrepresentation capability like the teacher feature. However, previous maskeddistillation methods only focus on spatial masking, making the resulting maskedareas biased towards spatial importance without encoding informative channelclues. In this study, we devise a Dual Masked Knowledge Distillation (DMKD)framework which can capture both spatially important and channel-wiseinformative clues for comprehensive masked feature reconstruction. Morespecifically, we employ dual attention mechanism for guiding the respectivemasking branches, leading to reconstructed feature encoding dual significance.Furthermore, fusing the reconstructed features is achieved by self-adjustableweighting strategy for effective feature distillation. Our experiments onobject detection task demonstrate that the student networks achieve performancegains of 4.1% and 4.3% with the help of our method when RetinaNet and CascadeMask R-CNN are respectively used as the teacher networks, while outperformingthe other state-of-the-art distillation methods.

DMKD: Improving Feature-based Knowledge Distillation for Object Detection Via Dual Masking Augmentation