English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Cross-modal contrastive learning in vision language pretraining (VLP) facesthe challenge of (partial) false negatives. In this paper, we study thisproblem from the perspective of Mutual Information (MI) optimization. It iscommon sense that InfoNCE loss used in contrastive learning will maximize thelower bound of MI between anchors and their positives, while we theoreticallyprove that MI involving negatives also matters when noises commonly exist.Guided by a more general lower bound form for optimization, we propose acontrastive learning strategy regulated by progressively refined cross-modalsimilarity, to more accurately optimize MI between an image/text anchor and itsnegative texts/images instead of improperly minimizing it. Our method performscompetitively on four downstream cross-modal tasks and systematically balancesthe beneficial and harmful effects of (partial) false negative samples undertheoretical guidance.

Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation