English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose the Data Contamination Quiz, a simple and effective approach todetect data contamination in large language models (LLMs) and estimate theamount of it. Specifically, we frame data contamination detection as a seriesof multiple-choice questions. We devise a quiz format wherein three perturbedversions of each dataset instance are created. These changes only includeword-level perturbations, replacing words with their contextual synonyms,ensuring both the semantic and sentence structure remain exactly the same asthe original instance. Together with the original instance, these perturbedversions constitute the choices in the quiz. Given that the only distinguishingsignal among these choices is the exact wording, an LLM, when tasked withidentifying the original instance from the choices, opts for the original if ithas memorized it in its pre-training phase--a trait intrinsic to LLMs. Adataset partition is then marked as contaminated if the LLM's performance onthe quiz surpasses what random chance suggests. Our evaluation spans sevendatasets and their respective splits (train and test/validation) on twostate-of-the-art LLMs: GPT-4 and GPT-3.5. While lacking access to thepre-training data, our results suggest that our approach not only enhances thedetection of data contamination but also provides an accurate estimation of itsextent, even when the contamination signal is weak.

Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models