English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Class-incremental learning (CIL) aims to adapt to emerging new classeswithout forgetting old ones. Traditional CIL models are trained from scratch tocontinually acquire knowledge as data evolves. Recently, pre-training hasachieved substantial progress, making vast pre-trained models (PTMs) accessiblefor CIL. Contrary to traditional methods, PTMs possess generalizableembeddings, which can be easily transferred. In this work, we revisit CIL withPTMs and argue that the core factors in CIL are adaptivity for model updatingand generalizability for knowledge transferring. 1) We first reveal that frozenPTM can already provide generalizable embeddings for CIL. Surprisingly, asimple baseline (SimpleCIL) which continually sets the classifiers of PTM toprototype features can beat state-of-the-art even without training on thedownstream task. 2) Due to the distribution gap between pre-trained anddownstream datasets, PTM can be further cultivated with adaptivity via modeladapting. We propose ADapt And Merge (ADAM), which aggregates the embeddings ofPTM and adapted models for classifier construction. ADAM is a general frameworkthat can be orthogonally combined with any parameter-efficient tuning method,which holds the advantages of PTM's generalizability and adapted model'sadaptivity. 3) Additionally, we find previous benchmarks are unsuitable in theera of PTM due to data overlapping and propose four new benchmarks forassessment, namely ImageNet-A, ObjectNet, OmniBenchmark, and VTAB. Extensiveexperiments validate the effectiveness of ADAM with a unified and conciseframework.

Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need