English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

To train image-caption retrieval (ICR) methods, contrastive loss functionsare a common choice for optimization functions. Unfortunately, contrastive ICRmethods are vulnerable to predictive feature suppression. Predictive featuresare features that correctly indicate the similarity between a query and acandidate item. However, in the presence of multiple predictive features duringtraining, encoder models tend to suppress redundant predictive features, sincethese features are not needed to learn to discriminate between positive andnegative pairs. While some predictive features are redundant during training,these features might be relevant during evaluation. We introduce an approach toreduce predictive feature suppression for resource-constrained ICR methods:latent target decoding (LTD). We add an additional decoder to the contrastiveICR framework, to reconstruct the input caption in a latent space of ageneral-purpose sentence encoder, which prevents the image and caption encoderfrom suppressing predictive features. We implement the LTD objective as anoptimization constraint, to ensure that the reconstruction loss is below abound value while primarily optimizing for the contrastive loss. Importantly,LTD does not depend on additional training data or expensive (hard) negativemining strategies. Our experiments show that, unlike reconstructing the inputcaption in the input space, LTD reduces predictive feature suppression,measured by obtaining higher recall@k, r-precision, and nDCG scores than acontrastive ICR baseline. Moreover, we show that LTD should be implemented asan optimization constraint instead of a dual optimization objective. Finally,we show that LTD can be used with different contrastive learning losses and awide variety of resource-constrained ICR methods.

Reducing Predictive Feature Suppression in Resource-Constrained Contrastive Image-Caption Retrieval