English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Image-Text Retrieval (ITR) is challenging in bridging visual and lingualmodalities. Contrastive learning has been adopted by most prior arts. Exceptfor limited amount of negative image-text pairs, the capability of constrastivelearning is restricted by manually weighting negative pairs as well asunawareness of external knowledge. In this paper, we propose our novel CoupledDiversity-Sensitive Momentum Constrastive Learning (CODER) for improvingcross-modal representation. Firstly, a novel diversity-sensitive contrastivelearning (DCL) architecture is invented. We introduce dynamic dictionaries forboth modalities to enlarge the scale of image-text pairs, anddiversity-sensitiveness is achieved by adaptive negative pair weighting.Furthermore, two branches are designed in CODER. One learns instance-levelembeddings from image/text, and it also generates pseudo online clusteringlabels for its input image/text based on their embeddings. Meanwhile, the otherbranch learns to query from commonsense knowledge graph to form concept-leveldescriptors for both modalities. Afterwards, both branches leverage DCL toalign the cross-modal embedding spaces while an extra pseudo clustering labelprediction loss is utilized to promote concept-level representation learningfor the second branch. Extensive experiments conducted on two popularbenchmarks, i.e. MSCOCO and Flicker30K, validate CODER remarkably outperformsthe state-of-the-art approaches.

CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval