English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural networks are very effective when trained on large datasets for a largenumber of iterations. However, when they are trained on non-stationary streamsof data and in an online fashion, their performance is reduced (1) by theonline setup, which limits the availability of data, (2) due to catastrophicforgetting because of the non-stationary nature of the data. Furthermore,several recent works (Caccia et al., 2022; Lange et al., 2023) arXiv:2205.13452showed that replay methods used in continual learning suffer from the stabilitygap, encountered when evaluating the model continually (rather than only ontask boundaries). In this article, we study the effect of model ensembling as away to improve performance and stability in online continual learning. Wenotice that naively ensembling models coming from a variety of training tasksincreases the performance in online continual learning considerably. Startingfrom this observation, and drawing inspirations from semi-supervised learningensembling methods, we use a lightweight temporal ensemble that computes theexponential moving average of the weights (EMA) at test time, and show that itcan drastically increase the performance and stability when used in combinationwith several methods from the literature.

Improving Online Continual Learning Performance and Stability with Temporal Ensembles