English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Most existing image-text matching methods adopt triplet loss as theoptimization objective, and choosing a proper negative sample for the tripletof <anchor, positive, negative> is important for effectively training themodel, e.g., hard negatives make the model learn efficiently and effectively.However, we observe that existing methods mainly employ the most similarsamples as hard negatives, which may not be true negatives. In other words, thesamples with high similarity but not paired with the anchor may reservepositive semantic associations, and we call them false negatives. Repellingthese false negatives in triplet loss would mislead the semantic representationlearning and result in inferior retrieval performance. In this paper, wepropose a novel False Negative Elimination (FNE) strategy to select negativesvia sampling, which could alleviate the problem introduced by false negatives.Specifically, we first construct the distributions of positive and negativesamples separately via their similarities with the anchor, based on thefeatures extracted from image and text encoders. Then we calculate the falsenegative probability of a given sample based on its similarity with the anchorand the above distributions via the Bayes' rule, which is employed as thesampling weight during negative sampling process. Since there may not exist anyfalse negative in a small batch size, we design a memory module with momentumto retain a large negative buffer and implement our negative sampling strategyspanning over the buffer. In addition, to make the model focus on hardnegatives, we reassign the sampling weights for the simple negatives with acut-down strategy. The extensive experiments are conducted on Flickr30K andMS-COCO, and the results demonstrate the superiority of our proposed falsenegative elimination strategy. The code is available athttps://github.com/LuminosityX/FNE.

Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination