English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The ability of neural networks (NNs) to learn and remember multiple taskssequentially is facing tough challenges in achieving general artificialintelligence due to their catastrophic forgetting (CF) issues. Fortunately, thelatest OWM Orthogonal Weights Modification) and other several continuallearning (CL) methods suggest some promising ways to overcome the CF issue.However, none of existing CL methods explores the following three crucialquestions for effectively overcoming the CF issue: that is, what knowledge doesit contribute to the effective weights modification of the NN during itssequential tasks learning? When the data distribution of a new learning taskchanges corresponding to the previous learned tasks, should a uniform/specificweight modification strategy be adopted or not? what is the upper bound of thelearningable tasks sequentially for a given CL method? ect. To achieve this, inthis paper, we first reveals the fact that of the weight gradient of a newlearning task is determined by both the input space of the new task and theweight space of the previous learned tasks sequentially. On this observationand the recursive least square optimal method, we propose a new efficient andeffective continual learning method EOWM via enhanced OWM. And we havetheoretically and definitively given the upper bound of the learningable taskssequentially of our EOWM. Extensive experiments conducted on the benchmarksdemonstrate that our EOWM is effectiveness and outperform all of thestate-of-the-art CL baselines.

Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification