English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastive self-supervised learning (CSL) with a prototypical regularizationhas been introduced in learning meaningful representations for downstream tasksthat require strong semantic information. However, to optimize CSL with a lossthat performs the prototypical regularization aggressively, e.g., the ProtoNCEloss, might cause the "coagulation" of examples in the embedding space. Thatis, the intra-prototype diversity of samples collapses to trivial solutions fortheir prototype being well-separated from others. Motivated by previous works,we propose to mitigate this phenomenon by learning Prototypical representationthrough Alignment, Uniformity and Correlation (PAUC). Specifically, theordinary ProtoNCE loss is revised with: (1) an alignment loss that pullsembeddings from positive prototypes together; (2) a uniformity loss thatdistributes the prototypical level features uniformly; (3) a correlation lossthat increases the diversity and discriminability between prototypical levelfeatures. We conduct extensive experiments on various benchmarks where theresults demonstrate the effectiveness of our method in improving the quality ofprototypical contrastive representations. Particularly, in the classificationdown-stream tasks with linear probes, our proposed method outperforms thestate-of-the-art instance-wise and prototypical contrastive learning methods onthe ImageNet-100 dataset by 2.96% and the ImageNet-1K dataset by 2.46% underthe same settings of batch size and epochs.

Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation