English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

State-of-the-art deep neural networks are still struggling to address thecatastrophic forgetting problem in continual learning. In this paper, wepropose one simple paradigm (named as S-Prompting) and two concrete approachesto highly reduce the forgetting degree in one of the most typical continuallearning scenarios, i.e., domain increment learning (DIL). The key idea of theparadigm is to learn prompts independently across domains with pre-trainedtransformers, avoiding the use of exemplars that commonly appear inconventional methods. This results in a win-win game where the prompting canachieve the best for each domain. The independent prompting across domains onlyrequests one single cross-entropy loss for training and one simple K-NNoperation as a domain identifier for inference. The learning paradigm derivesan image prompt learning approach and a novel language-image prompt learningapproach. Owning an excellent scalability (0.03% parameter increase perdomain), the best of our approaches achieves a remarkable relative improvement(an average of about 30%) over the best of the state-of-the-art exemplar-freemethods for three standard DIL tasks, and even surpasses the best of themrelatively by about 6% in average when they use exemplars. Source code isavailable at \url{https://github.com/iamwangyabin/S-Prompts}.

S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning