English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

While knowledge distillation (transfer) has been attracting attentions fromthe research community, the recent development in the fields has heightened theneed for reproducible studies and highly generalized frameworks to lowerbarriers to such high-quality, reproducible deep learning research. Severalresearchers voluntarily published frameworks used in their knowledgedistillation studies to help other interested researchers reproduce theiroriginal work. Such frameworks, however, are usually neither well generalizednor maintained, thus researchers are still required to write a lot of code torefactor/build on the frameworks for introducing new methods, models, datasetsand designing experiments. In this paper, we present our developed open-sourceframework built on PyTorch and dedicated for knowledge distillation studies.The framework is designed to enable users to design experiments by declarativePyYAML configuration files, and helps researchers complete the recentlyproposed ML Code Completeness Checklist. Using the developed framework, wedemonstrate its various efficient training strategies, and implement a varietyof knowledge distillation methods. We also reproduce some of their originalexperimental results on the ImageNet and COCO datasets presented at majormachine learning conferences such as ICLR, NeurIPS, CVPR and ECCV, includingrecent state-of-the-art methods. All the source code, configurations, log filesand trained model weights are publicly available athttps://github.com/yoshitomo-matsubara/torchdistill .

torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation