English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Knowledge distillation (KD) is an effective model compression technique wherea compact student network is taught to mimic the behavior of a complex andhighly trained teacher network. In contrast, Mutual Learning (ML) provides analternative strategy where multiple simple student networks benefit fromsharing knowledge, even in the absence of a powerful but static teachernetwork. Motivated by these findings, we propose a single-teacher,multi-student framework that leverages both KD and ML to achieve betterperformance. Furthermore, an online distillation strategy is utilized to trainthe teacher and students simultaneously. To evaluate the performance of theproposed approach, extensive experiments were conducted using three differentversions of teacher-student networks on benchmark biomedical classification(MSI vs. MSS) and object detection (Polyp Detection) tasks. Ensemble of studentnetworks trained in the proposed manner achieved better results than theensemble of students trained using KD or ML individually, establishing thebenefit of augmenting knowledge transfer from teacher to students withpeer-to-peer learning between students.

Augmenting Knowledge Distillation With Peer-To-Peer Mutual Learning For Model Compression