English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Neural retrieval models are generally regarded as fundamentally differentfrom the retrieval techniques used in the late 1990's when the TREC ad hoc testcollections were constructed. They thus provide the opportunity to empiricallytest the claim that pooling-built test collections can reliably evaluateretrieval systems that did not contribute to the construction of the collection(in other words, that such collections can be reusable). To test thereusability claim, we asked TREC assessors to judge new pools created from newsearch results for the TREC-8 ad hoc collection. These new search resultsconsisted of five new runs (one each from three transformer-based models andtwo baseline runs that use BM25) plus the set of TREC-8 submissions that didnot previously contribute to pools. The new runs did retrieve previously unseendocuments, but the vast majority of those documents were not relevant. Theranking of all runs by mean evaluation score when evaluated using the officialTREC-8 relevance judgment set and the newly expanded relevance set are almostidentical, with Kendall's tau correlations greater than 0.99. Correlations forindividual topics are also high. The TREC-8 ad hoc collection was originallyconstructed using deep pools over a diverse set of runs, including severaleffective manual runs. Its judgment budget, and hence construction cost, wasrelatively large. However, it does appear that the expense was well-spent: evenwith the advent of neural techniques, the collection has stood the test of timeand remains a reliable evaluation instrument as retrieval techniques haveadvanced.

Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?