English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The backpropagation networks are notably susceptible to catastrophicforgetting, where networks tend to forget previously learned skills uponlearning new ones. To address such the 'sensitivity-stability' dilemma, mostprevious efforts have been contributed to minimizing the empirical risk withdifferent parameter regularization terms and episodic memory, but rarelyexploring the usages of the weight loss landscape. In this paper, weinvestigate the relationship between the weight loss landscape andsensitivity-stability in the continual learning scenario, based on which, wepropose a novel method, Flattening Sharpness for Dynamic Gradient ProjectionMemory (FS-DGPM). In particular, we introduce a soft weight to represent theimportance of each basis representing past tasks in GPM, which can beadaptively learned during the learning process, so that less important basescan be dynamically released to improve the sensitivity of new skill learning.We further introduce Flattening Sharpness (FS) to reduce the generalization gapby explicitly regulating the flatness of the weight loss landscape of all seentasks. As demonstrated empirically, our proposed method consistentlyoutperforms baselines with the superior ability to learn new skills whilealleviating forgetting effectively.

Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning