English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This work proposes a new method to sequentially train deep neural networks onmultiple tasks without suffering catastrophic forgetting, while endowing itwith the capability to quickly adapt to unseen tasks. Starting from existingwork on network masking (Wortsman et al., 2020), we show that simply learning alinear combination of a small number of task-specific supermasks (impressions)on a randomly initialized backbone network is sufficient to both retainaccuracy on previously learned tasks, as well as achieve high accuracy onunseen tasks. In contrast to previous methods, we do not require to generatededicated masks or contexts for each new task, instead leveraging transferlearning to keep per-task parameter overhead small. Our work illustrates thepower of linearly combining individual impressions, each of which fares poorlyin isolation, to achieve performance comparable to a dedicated mask. Moreover,even repeated impressions from the same task (homogeneous masks), whencombined, can approach the performance of heterogeneous combinations ifsufficiently many impressions are used. Our approach scales more efficientlythan existing methods, often requiring orders of magnitude fewer parameters andcan function without modification even when task identity is missing. Inaddition, in the setting where task labels are not given at inference, ouralgorithm gives an often favorable alternative to the one-shot procedure usedby Wortsman et al., 2020. We evaluate our method on a number of well-knownimage classification datasets and network architectures.

ImpressLearn: Continual Learning via Combined Task Impressions