English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Continuous unsupervised representation learning (CURL) research has greatlybenefited from improvements in self-supervised learning (SSL) techniques. As aresult, existing CURL methods using SSL can learn high-quality representationswithout any labels, but with a notable performance drop when learning on amany-tasks data stream. We hypothesize that this is caused by theregularization losses that are imposed to prevent forgetting, leading to asuboptimal plasticity-stability trade-off: they either do not adapt fully tothe incoming data (low plasticity), or incur significant forgetting whenallowed to fully adapt to a new SSL pretext-task (low stability). In this work,we propose to train an expert network that is relieved of the duty of keepingthe previous knowledge and can focus on performing optimally on the new tasks(optimizing plasticity). In the second phase, we combine this new knowledgewith the previous network in an adaptation-retrospection phase to avoidforgetting and initialize a new expert with the knowledge of the old network.We perform several experiments showing that our proposed approach outperformsother CURL exemplar-free methods in few- and many-task split settings.Furthermore, we show how to adapt our approach to semi-supervised continuallearning (Semi-SCL) and show that we surpass the accuracy of otherexemplar-free Semi-SCL methods and reach the results of some others that useexemplars.

Plasticity-Optimized Complementary Networks for Unsupervised Continual Learning