English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

With large language models (LLMs) poised to become embedded in our dailylives, questions are starting to be raised about the dataset(s) they learnedfrom. These questions range from potential bias or misinformation LLMs couldretain from their training data to questions of copyright and fair use ofhuman-generated text. However, while these questions emerge, developers of therecent state-of-the-art LLMs become increasingly reluctant to disclose detailson their training corpus. We here introduce the task of document-levelmembership inference for real-world LLMs, i.e. inferring whether the LLM hasseen a given document during training or not. First, we propose a procedure forthe development and evaluation of document-level membership inference for LLMsby leveraging commonly used data sources for training and the model releasedate. We then propose a practical, black-box method to predict document-levelmembership and instantiate it on OpenLLaMA-7B with both books and academicpapers. We show our methodology to perform very well, reaching an impressiveAUC of 0.856 for books and 0.678 for papers. We then show our approach tooutperform the sentence-level membership inference attacks used in the privacyliterature for the document-level membership task. We finally evaluate whethersmaller models might be less sensitive to document-level inference and showOpenLLaMA-3B to be approximately as sensitive as OpenLLaMA-7B to our approach.Taken together, our results show that accurate document-level membership can beinferred for LLMs, increasing the transparency of technology poised to changeour lives.

Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models