English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In evaluation campaigns, participants often explore variations of popular,state-of-the-art baselines as a low-risk strategy to achieve competitiveresults. While effective, this can lead to local "hill climbing" rather thanmore radical and innovative departure from standard methods. Moreover, if manyparticipants build on similar baselines, the overall diversity of approachesconsidered may be limited. In this work, we propose a new class of IRevaluation metrics intended to promote greater diversity of approaches inevaluation campaigns. Whereas traditional IR metrics focus on user experience,our two "innovation" metrics instead reward exploration of more divergent,higher-risk strategies finding relevant documents missed by other systems.Experiments on four TREC collections show that our metrics do change systemrankings by rewarding systems that find such rare, relevant documents. Thisresult is further supported by a controlled, synthetic data experiment, and aqualitative analysis. In addition, we show that our metrics achieve higherevaluation stability and discriminative power than the standard metrics wemodify. To support reproducibility, we share our source code.

New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches