English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Catastrophic forgetting is a challenge issue in continual learning when adeep neural network forgets the knowledge acquired from the former task afterlearning on subsequent tasks. However, existing methods try to find the jointdistribution of parameters shared with all tasks. This idea can be questionablebecause this joint distribution may not present when the number of tasksincrease. On the other hand, It also leads to "long-term" memory issue when thenetwork capacity is limited since adding tasks will "eat" the network capacity.In this paper, we proposed a Soft Parameters Pruning (SPP) strategy to reachthe trade-off between short-term and long-term profit of a learning model byfreeing those parameters less contributing to remember former task domainknowledge to learn future tasks, and preserving memories about previous tasksvia those parameters effectively encoding knowledge about tasks at the sametime. The SPP also measures the importance of parameters by information entropyin a label free manner. The experiments on several tasks shows SPP modelachieved the best performance compared with others state-of-the-art methods.Experiment results also indicate that our method is less sensitive tohyper-parameter and better generalization. Our research suggests that a softerstrategy, i.e. approximate optimize or sub-optimal solution, will benefitalleviating the dilemma of memory. The source codes are available athttps://github.com/lehaifeng/Learning_by_memory.

Overcoming Catastrophic Forgetting by Soft Parameter Pruning