English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Work on continual learning (CL) has largely focused on the problems arisingfrom the dynamically-changing data distribution. However, CL can be decomposedinto two sub-problems: (a) shifts in the data distribution, and (b) dealingwith the fact that the data is split into chunks and so only a part of the datais available to be trained on at any point in time. In this work, we look atthe latter sub-problem -- the chunking of data -- and note that previousanalysis of chunking in the CL literature is sparse. We show that chunking isan important part of CL, accounting for around half of the performance dropfrom offline learning in our experiments. Furthermore, our results reveal thatcurrent CL algorithms do not address the chunking sub-problem, only performingas well as plain SGD training when there is no shift in the data distribution.We analyse why performance drops when learning occurs on chunks of data, andfind that forgetting, which is often seen to be a problem due to distributionshift, still arises and is a significant problem. Motivated by an analysis ofthe linear case, we show that per-chunk weight averaging improves performancein the chunking setting and that this performance transfers to the full CLsetting. Hence, we argue that work on chunking can help advance CL in general.

Chunking: Forgetting Matters in Continual Learning even without Changing Tasks