English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Sequential learning of multiple tasks in artificial neural networks usinggradient descent leads to catastrophic forgetting, whereby previously learnedknowledge is erased during learning of new, disjoint knowledge. Here, wepropose a new approach to sequential learning which leverages the recentdiscovery of adversarial examples. We use adversarial subspaces from previoustasks to enable learning of new tasks with less interference. We apply ourmethod to sequentially learning to classify digits 0, 1, 2 (task 1), 4, 5, 6,(task 2), and 7, 8, 9 (task 3) in MNIST (disjoint MNIST task). We compare andcombine our Adversarial Direction (AD) method with the recently proposedElastic Weight Consolidation (EWC) method for sequential learning. We traineach task for 20 epochs, which yields good initial performance (99.24% correcttask 1 performance). After training task 2, and then task 3, both plaingradient descent (PGD) and EWC largely forget task 1 (task 1 accuracy 32.95%for PGD and 41.02% for EWC), while our combined approach (AD+EWC) stillachieves 94.53% correct on task 1. We obtain similar results with a much moredifficult disjoint CIFAR10 task, which to our knowledge had not been attemptedbefore (70.10% initial task 1 performance, 67.73% after learning tasks 2 and 3for AD+EWC, while PGD and EWC both fall to chance level). Our results suggestthat AD+EWC can provide better sequential learning performance than either PGDor EWC.

Overcoming catastrophic forgetting problem by weight consolidation and long-term memory