English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Catastrophic forgetting in neural networks is a significant problem forcontinual learning. A majority of the current methods replay previous dataduring training, which violates the constraints of an ideal continual learningsystem. Additionally, current approaches that deal with forgetting ignore theproblem of catastrophic remembering, i.e. the worsening ability to discriminatebetween data from different tasks. In our work, we introduce Relevance MappingNetworks (RMNs) which are inspired by the Optimal Overlap Hypothesis. Themappings reflects the relevance of the weights for the task at hand byassigning large weights to essential parameters. We show that RMNs learn anoptimized representational overlap that overcomes the twin problem ofcatastrophic forgetting and remembering. Our approach achieves state-of-the-artperformance across all common continual learning datasets, even significantlyoutperforming data replay methods while not violating the constraints for anideal continual learning system. Moreover, RMNs retain the ability to detectdata from new tasks in an unsupervised manner, thus proving their resilienceagainst catastrophic remembering.

Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping