English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this position paper, we argue that the classical evaluation on NaturalLanguage Processing (NLP) tasks using annotated benchmarks is in trouble. Theworst kind of data contamination happens when a Large Language Model (LLM) istrained on the test split of a benchmark, and then evaluated in the samebenchmark. The extent of the problem is unknown, as it is not straightforwardto measure. Contamination causes an overestimation of the performance of acontaminated model in a target benchmark and associated task with respect totheir non-contaminated counterparts. The consequences can be very harmful, withwrong scientific conclusions being published while other correct ones arediscarded. This position paper defines different levels of data contaminationand argues for a community effort, including the development of automatic andsemi-automatic measures to detect when data from a benchmark was exposed to amodel, and suggestions for flagging papers with conclusions that arecompromised by data contamination.

NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark