English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastive learning (CL) has emerged as a powerful technique forrepresentation learning, with or without label supervision. However, supervisedCL is prone to collapsing representations of subclasses within a class by notcapturing all their features, and unsupervised CL may suppress harderclass-relevant features by focusing on learning easy class-irrelevant features;both significantly compromise representation quality. Yet, there is notheoretical understanding of \textit{class collapse} or \textit{featuresuppression} at \textit{test} time. We provide the first unified theoreticallyrigorous framework to determine \textit{which} features are learnt by CL. Ouranalysis indicate that, perhaps surprisingly, bias of (stochastic) gradientdescent towards finding simpler solutions is a key factor in collapsingsubclass representations and suppressing harder class-relevant features.Moreover, we present increasing embedding dimensionality and improving thequality of data augmentations as two theoretically motivated solutions to{feature suppression}. We also provide the first theoretical explanation forwhy employing supervised and unsupervised CL together yields higher-qualityrepresentations, even when using commonly-used stochastic gradient methods.

Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression