English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Artificial neural networks face the well-known problem of catastrophicforgetting. What's worse, the degradation of previously learned skills becomesmore severe as the task sequence increases, known as the long-term catastrophicforgetting. It is due to two facts: first, as the model learns more tasks, theintersection of the low-error parameter subspace satisfying for these tasksbecomes smaller or even does not exist; second, when the model learns a newtask, the cumulative error keeps increasing as the model tries to protect theparameter configuration of previous tasks from interference. Inspired by thememory consolidation mechanism in mammalian brains with synaptic plasticity, wepropose a confrontation mechanism in which Adversarial Neural Pruning andsynaptic Consolidation (ANPyC) is used to overcome the long-term catastrophicforgetting issue. The neural pruning acts as long-term depression to prunetask-irrelevant parameters, while the novel synaptic consolidation acts aslong-term potentiation to strengthen task-relevant parameters. During thetraining, this confrontation achieves a balance in that only crucial parametersremain, and non-significant parameters are freed to learn subsequent tasks.ANPyC avoids forgetting important information and makes the model efficient tolearn a large number of tasks. Specifically, the neural pruning iterativelyrelaxes the current task's parameter conditions to expand the common parametersubspace of the task; the synaptic consolidation strategy, which consists of astructure-aware parameter-importance measurement and an element-wise parameterupdating strategy, decreases the cumulative error when learning new tasks. Thefull source code is available at https://github.com/GeoX-Lab/ANPyC.

Overcoming Long-Term Catastrophic Forgetting Through Adversarial Neural Pruning and Synaptic Consolidation