English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent observations have underscored a disparity between the inflatedbenchmark scores and the actual performance of LLMs, raising concerns aboutpotential contamination of evaluation benchmarks. This issue is especiallycritical for closed-source models and certain open-source models where trainingdata transparency is lacking. In this paper we study data contamination byproposing two methods tailored for both open-source and proprietary LLMs. Wefirst introduce a retrieval-based system to explore potential overlaps betweenevaluation benchmarks and pretraining corpora. We further present a novelinvestigation protocol named \textbf{T}estset \textbf{S}lot Guessing(\textit{TS-Guessing}), applicable to both open and proprietary models. Thisapproach entails masking a wrong answer in a multiple-choice question andprompting the model to fill in the gap. Additionally, it involves obscuring anunlikely word in an evaluation example and asking the model to produce it. Wefind that certain commercial LLMs could surprisingly guess the missing optionin various test sets. Specifically, in the TruthfulQA benchmark, we find thatLLMs exhibit notable performance improvement when provided with additionalmetadata in the benchmark. Further, in the MMLU benchmark, ChatGPT and GPT-4demonstrated an exact match rate of 52\% and 57\%, respectively, in guessingthe missing options in benchmark test data. We hope these results underscorethe need for more robust evaluation methodologies and benchmarks in the field.

Investigating Data Contamination in Modern Benchmarks for Large Language Models