English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Text-image cross-modal retrieval is a challenging task in the field oflanguage and vision. Most previous approaches independently embed images andsentences into a joint embedding space and compare their similarities. However,previous approaches rarely explore the interactions between images andsentences before calculating similarities in the joint space. Intuitively, whenmatching between images and sentences, human beings would alternatively attendto regions in images and words in sentences, and select the most salientinformation considering the interaction between both modalities. In this paper,we propose Cross-modal Adaptive Message Passing (CAMP), which adaptivelycontrols the information flow for message passing across modalities. Ourapproach not only takes comprehensive and fine-grained cross-modal interactionsinto account, but also properly handles negative pairs and irrelevantinformation with an adaptive gating scheme. Moreover, instead of conventionaljoint embedding approaches for text-image matching, we infer the matching scorebased on the fused features, and propose a hardest negative binarycross-entropy loss for training. Results on COCO and Flickr30k significantlysurpass state-of-the-art methods, demonstrating the effectiveness of ourapproach.

CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval