English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Knowledge distillation is a potential solution for model compression. Theidea is to make a small student network imitate the target of a large teachernetwork, then the student network can be competitive to the teacher one. Mostprevious studies focus on model distillation in the classification task, wherethey propose different architects and initializations for the student network.However, only the classification task is not enough, and other related taskssuch as regression and retrieval are barely considered. To solve the problem,in this paper, we take face recognition as a breaking point and propose modeldistillation with knowledge transfer from face classification to alignment andverification. By selecting appropriate initializations and targets in theknowledge transfer, the distillation can be easier in non-classification tasks.Experiments on the CelebA and CASIA-WebFace datasets demonstrate that thestudent network can be competitive to the teacher one in alignment andverification, and even surpasses the teacher network under specific compressionrates. In addition, to achieve stronger knowledge transfer, we also use acommon initialization trick to improve the distillation performance ofclassification. Evaluations on the CASIA-Webface and large-scale MS-Celeb-1Mdatasets show the effectiveness of this simple trick.

Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification