English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Online continual learning (OCL) aims to continuously learn new data from asingle pass over the online data stream. It generally suffers from thecatastrophic forgetting issue. Existing replay-based methods effectivelyalleviate this issue by replaying part of old data in a proxy-based orcontrastive-based replay manner. In this paper, we conduct a comprehensiveanalysis of these two replay manners and find they can be complementary.Inspired by this finding, we propose a novel replay-based method calledproxy-based contrastive replay (PCR), which replaces anchor-to-sample pairswith anchor-to-proxy pairs in the contrastive-based loss to alleviate thephenomenon of forgetting. Based on PCR, we further develop a more advancedmethod named holistic proxy-based contrastive replay (HPCR), which consists ofthree components. The contrastive component conditionally incorporatesanchor-to-sample pairs to PCR, learning more fine-grained semantic informationwith a large training batch. The second is a temperature component thatdecouples the temperature coefficient into two parts based on their impacts onthe gradient and sets different values for them to learn more novel knowledge.The third is a distillation component that constrains the learning process tokeep more historical knowledge. Experiments on four datasets consistentlydemonstrate the superiority of HPCR over various state-of-the-art methods.

HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning