English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In recent years, online distillation has emerged as a powerful technique foradapting real-time deep neural networks on the fly using a slow, but accurateteacher model. However, a major challenge in online distillation iscatastrophic forgetting when the domain shifts, which occurs when the studentmodel is updated with data from the new domain and forgets previously learnedknowledge. In this paper, we propose a solution to this issue by leveraging thepower of continual learning methods to reduce the impact of domain shifts.Specifically, we integrate several state-of-the-art continual learning methodsin the context of online distillation and demonstrate their effectiveness inreducing catastrophic forgetting. Furthermore, we provide a detailed analysisof our proposed solution in the case of cyclic domain shifts. Our experimentalresults demonstrate the efficacy of our approach in improving the robustnessand accuracy of online distillation, with potential applications in domainssuch as video surveillance or autonomous driving. Overall, our work representsan important step forward in the field of online distillation and continuallearning, with the potential to significantly impact real-world applications.

Online Distillation with Continual Learning for Cyclic Domain Shifts