English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The design of machines and algorithms capable of learning in a dynamicallychanging environment has become an increasingly topical problem with theincrease of the size and heterogeneity of data available to learning systems.As a consequence, the key issue of Continual Learning has become that ofaddressing the stability-plasticity dilemma of connectionist systems, as theyneed to adapt their model without forgetting previously acquired knowledge.Within this context, rehearsal-based methods i.e., solutions in where thelearner exploits memory to revisit past data, has proven to be very effective,leading to performance at the state-of-the-art. In our study, we propose ananalysis of the memory quantity/quality trade-off adopting various datareduction approaches to increase the number of instances storable in memory. Inparticular, we investigate complex instance compression techniques such as deepencoders, but also trivial approaches such as image resizing and lineardimensionality reduction. Our findings suggest that the optimal trade-off isseverely skewed toward instance quantity, where rehearsal approaches withseveral heavily compressed instances easily outperform state-of-the-artapproaches with the same amount of memory at their disposal. Further, in highmemory configurations, deep approaches extracting spatial structure combinedwith extreme resizing (of the order of $8\times8$ images) yield the bestresults, while in memory-constrained configurations where deep approachescannot be used due to their memory requirement in training, Extreme LearningMachines (ELM) offer a clear advantage.

Smaller Is Better: An Analysis of Instance Quantity/Quality Trade-off in Rehearsal-based Continual Learning