English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While recent continual learning methods largely alleviate the catastrophicproblem on toy-sized datasets, some issues remain to be tackled to apply themto real-world problem domains. First, a continual learning model shouldeffectively handle catastrophic forgetting and be efficient to train even witha large number of tasks. Secondly, it needs to tackle the problem oforder-sensitivity, where the performance of the tasks largely varies based onthe order of the task arrival sequence, as it may cause serious problems wherefairness plays a critical role (e.g. medical diagnosis). To tackle thesepractical challenges, we propose a novel continual learning method that isscalable as well as order-robust, which instead of learning a completely sharedset of weights, represents the parameters for each task as a sum of task-sharedand sparse task-adaptive parameters. With our Additive Parameter Decomposition(APD), the task-adaptive parameters for earlier tasks remain mostly unaffected,where we update them only to reflect the changes made to the task-sharedparameters. This decomposition of parameters effectively prevents catastrophicforgetting and order-sensitivity, while being computation- andmemory-efficient. Further, we can achieve even better scalability with APDusing hierarchical knowledge consolidation, which clusters the task-adaptiveparameters to obtain hierarchically shared parameters. We validate our networkwith APD, APD-Net, on multiple benchmark datasets against state-of-the-artcontinual learning methods, which it largely outperforms in accuracy,scalability, and order-robustness.

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition