English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastive learning, along with its variations, has been a highly effectiveself-supervised learning method across diverse domains. Contrastive learningmeasures the distance between representations using cosine similarity and usescross-entropy for representation learning. Within the same framework ofcosine-similarity-based representation learning, margins have played asignificant role in enhancing face and speaker recognition tasks.Interestingly, despite the shared reliance on the same similarity metrics andobjective functions, contrastive learning has not actively adopted margins.Furthermore, decision-boundary-based explanations are the only ones that havebeen used to explain the effect of margins in contrastive learning. In thiswork, we propose a new perspective to understand the role of margins based ongradient analysis. Based on the new perspective, we analyze how margins affectgradients of contrastive learning and separate the effect into more elementallevels. We separately analyze each and provide possible directions forimproving contrastive learning. Our experimental results demonstrate thatemphasizing positive samples and scaling gradients depending on positive sampleangles and logits are the keys to improving the generalization performance ofcontrastive learning in both seen and unseen datasets, and other factors canonly marginally improve performance.

Understanding Contrastive Learning Through the Lens of Margins