English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Plasticity-stability dilemma is a main problem for incremental learning,where plasticity is referring to the ability to learn new knowledge, andstability retains the knowledge of previous tasks. Many methods tackle thisproblem by storing previous samples, while in some applications, training datafrom previous tasks cannot be legally stored. In this work, we propose toemploy mode connectivity in loss landscapes to achieve betterplasticity-stability trade-off without any previous samples. We give ananalysis of why and how to connect two independently optimized optima ofnetworks, null-space projection for previous tasks and simple SGD for thecurrent task, can attain a meaningful balance between preserving alreadylearned knowledge and granting sufficient flexibility for learning a new task.This analysis of mode connectivity also provides us a new perspective andtechnology to control the trade-off between plasticity and stability. Weevaluate the proposed method on several benchmark datasets. The resultsindicate our simple method can achieve notable improvement, and perform well onboth the past and current tasks. On 10-split-CIFAR-100 task, our methodachieves 79.79% accuracy, which is 6.02% higher. Our method also achieves 6.33%higher accuracy on TinyImageNet. Code is available athttps://github.com/lingl1024/Connector.

Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector