English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Pre-trained representation is one of the key elements in the success ofmodern deep learning. However, existing works on continual learning methodshave mostly focused on learning models incrementally from scratch. In thispaper, we explore an alternative framework to incremental learning where wecontinually fine-tune the model from a pre-trained representation. Our methodtakes advantage of linearization technique of a pre-trained neural network forsimple and effective continual learning. We show that this allows us to designa linear model where quadratic parameter regularization method is placed as theoptimal continual learning policy, and at the same time enjoying the highperformance of neural networks. We also show that the proposed algorithmenables parameter regularization methods to be applied to class-incrementalproblems. Additionally, we provide a theoretical reason why the existingparameter-space regularization algorithms such as EWC underperform on neuralnetworks trained with cross-entropy loss. We show that the proposed method canprevent forgetting while achieving high continual fine-tuning performance onimage classification tasks. To show that our method can be applied to generalcontinual learning settings, we evaluate our method in data-incremental,task-incremental, and class-incremental learning problems.

DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning