English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Cross-Modal Retrieval (CMR) is an important research topic across multimodalcomputing and information retrieval, which takes one type of data as the queryto retrieve relevant data of another type. It has been widely used in manyreal-world applications. Recently, the vision-language pre-trained modelsrepresented by CLIP demonstrate its superiority in learning the visual andtextual representations and gain impressive performance on various vision andlanguage related tasks. Although CLIP as well as the previous pre-trainedmodels have shown great performance improvement in the unsupervised CMR, theperformance and impact of these pre-trained models on the supervised CMR wererarely explored due to the lack of common representation for the multimodalclass-level associations. In this paper, we take CLIP as the currentrepresentative vision-language pre-trained model to conduct a comprehensiveempirical study. We evaluate its performance and impact on the supervised CMR,and attempt to answer several key research questions. To this end, we firstpropose a novel model CLIP4CMR (CLIP enhanced network for Cross-ModalRetrieval) that employs the pre-trained CLIP as backbone network to perform thesupervised CMR. Then by means of the CLIP4CMR framework, we revisit the designof different learning objectives in current CMR methods to provide new insightson model design. Moreover, we investigate the most concerned aspects inapplying CMR, including the robustness to modality imbalance and sensitivity tohyper-parameters, to provide new perspectives for practical applications.Through extensive experiments, we show that CLIP4CMR achieves the SOTA resultswith prominent improvements on the benchmark datasets, and can be used as afundamental framework to empirically study the key research issues of thesupervised CMR, with significant implications for model design and practicalconsiderations.

A Comprehensive Empirical Study of Vision-Language Pre-trained Model for Supervised Cross-Modal Retrieval