English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural ranking models (NRMs) have become one of the most important techniquesin information retrieval (IR). Due to the limitation of relevance labels, thetraining of NRMs heavily relies on negative sampling over unlabeled data. Ingeneral machine learning scenarios, it has shown that training with hardnegatives (i.e., samples that are close to positives) could lead to betterperformance. Surprisingly, we find opposite results from our empirical studiesin IR. When sampling top-ranked results (excluding the labeled positives) asnegatives from a stronger retriever, the performance of the learned NRM becomeseven worse. Based on our investigation, the superficial reason is that thereare more false negatives (i.e., unlabeled positives) in the top-ranked resultswith a stronger retriever, which may hurt the training process; The root is theexistence of pooling bias in the dataset constructing process, where annotatorsonly judge and label very few samples selected by some basic retrievers.Therefore, in principle, we can formulate the false negative issue in trainingNRMs as learning from labeled datasets with pooling bias. To solve thisproblem, we propose a novel Coupled Estimation Technique (CET) that learns botha relevance model and a selection model simultaneously to correct the poolingbias for training NRMs. Empirical results on three retrieval benchmarks showthat NRMs trained with our technique can achieve significant gains on rankingeffectiveness against other baseline strategies.

Hard Negatives or False Negatives: Correcting Pooling Bias in Training Neural Ranking Models