English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Contrastive self-supervised representation learning methods maximize thesimilarity between the positive pairs, and at the same time tend to minimizethe similarity between the negative pairs. However, in general the interplaybetween the negative pairs is ignored as they do not put in place specialmechanisms to treat negative pairs differently according to their specificdifferences and similarities. In this paper, we present Extended MomentumContrast (XMoCo), a self-supervised representation learning method founded uponthe legacy of the momentum-encoder unit proposed in the MoCo familyconfigurations. To this end, we introduce a cross consistency regularizationloss, with which we extend the transformation consistency to dissimilar images(negative pairs). Under the cross consistency regularization rule, we arguethat semantic representations associated with any pair of images (positive ornegative) should preserve their cross-similarity under pretext transformations.Moreover, we further regularize the training loss by enforcing a uniformdistribution of similarity over the negative pairs across a batch. The proposedregularization can easily be added to existing self-supervised learningalgorithms in a plug-and-play fashion. Empirically, we report a competitiveperformance on the standard Imagenet-1K linear head classification benchmark.In addition, by transferring the learned representations to common downstreamtasks, we show that using XMoCo with the prevalently utilized augmentations canlead to improvements in the performance of such tasks. We hope the findings ofthis paper serve as a motivation for researchers to take into consideration theimportant interplay among the negative examples in self-supervised learning.

Extending Momentum Contrast with Cross Similarity Consistency Regularization