English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Catastrophic forgetting (CF) happens whenever a neural network overwritespast knowledge while being trained on new tasks. Common techniques to handle CFinclude regularization of the weights (using, e.g., their importance on pasttasks), and rehearsal strategies, where the network is constantly re-trained onpast data. Generative models have also been applied for the latter, in order tohave endless sources of data. In this paper, we propose a novel method thatcombines the strengths of regularization and generative-based rehearsalapproaches. Our generative model consists of a normalizing flow (NF), aprobabilistic and invertible neural network, trained on the internal embeddingsof the network. By keeping a single NF conditioned on the task, we show thatour memory overhead remains constant. In addition, exploiting the invertibilityof the NF, we propose a simple approach to regularize the network's embeddingswith respect to past tasks. We show that our method performs favorably withrespect to state-of-the-art approaches in the literature, with boundedcomputational power and memory overheads.

Pseudo-Rehearsal for Continual Learning with Normalizing Flows