English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained large language models, such as GPT\nobreakdash-2 and BERT, areoften fine-tuned to achieve state-of-the-art performance on a downstream task.One natural example is the ``Smart Reply'' application where a pre-trainedmodel is tuned to provide suggested responses for a given query message. Sincethe tuning data is often sensitive data such as emails or chat transcripts, itis important to understand and mitigate the risk that the model leaks itstuning data. We investigate potential information leakage vulnerabilities in atypical Smart Reply pipeline. We consider a realistic setting where theadversary can only interact with the underlying model through a front-endinterface that constrains what types of queries can be sent to the model.Previous attacks do not work in these settings, but require the ability to sendunconstrained queries directly to the model. Even when there are no constraintson the queries, previous attacks typically require thousands, or even millions,of queries to extract useful information, while our attacks can extractsensitive data in just a handful of queries. We introduce a new type of activeextraction attack that exploits canonical patterns in text containing sensitivedata. We show experimentally that it is possible for an adversary to extractsensitive user information present in the training data, even in realisticsettings where all interactions with the model must go through a front-end thatlimits the types of queries. We explore potential mitigation strategies anddemonstrate empirically how differential privacy appears to be a reasonablyeffective defense mechanism to such pattern extraction attacks.

Combing for Credentials: Active Pattern Extraction from Smart Reply