English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, contrastive learning has risen to be a promising approach forlarge-scale self-supervised learning. However, theoretical understanding of howit works is still unclear. In this paper, we propose a new guarantee on thedownstream performance without resorting to the conditional independenceassumption that is widely adopted in previous work but hardly holds inpractice. Our new theory hinges on the insight that the support of differentintra-class samples will become more overlapped under aggressive dataaugmentations, thus simply aligning the positive samples (augmented views ofthe same sample) could make contrastive learning cluster intra-class samplestogether. Based on this augmentation overlap perspective, theoretically, weobtain asymptotically closed bounds for downstream performance under weakerassumptions, and empirically, we propose an unsupervised model selection metricARC that aligns well with downstream accuracy. Our theory suggests analternative understanding of contrastive learning: the role of aligningpositive samples is more like a surrogate task than an ultimate goal, and theoverlapped augmented views (i.e., the chaos) create a ladder for contrastivelearning to gradually learn class-separated representations. The code forcomputing ARC is available at https://github.com/zhangq327/ARC.

Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap