English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised learning (SSL) algorithms can produce useful imagerepresentations by learning to associate different parts of natural images withone another. However, when taken to the extreme, SSL models can unintendedlymemorize specific parts in individual training samples rather than learningsemantically meaningful associations. In this work, we perform a systematicstudy of the unintended memorization of image-specific information in SSLmodels -- which we refer to as d\'ej\`a vu memorization. Concretely, we showthat given the trained model and a crop of a training image containing only thebackground (e.g., water, sky, grass), it is possible to infer the foregroundobject with high accuracy or even visually reconstruct it. Furthermore, we showthat d\'ej\`a vu memorization is common to different SSL algorithms, isexacerbated by certain design choices, and cannot be detected by conventionaltechniques for evaluating representation quality. Our study of d\'ej\`a vumemorization reveals previously unknown privacy risks in SSL models, as well assuggests potential practical mitigation strategies. Code is available athttps://github.com/facebookresearch/DejaVu.

Do SSL Models Have D\'ej\`a Vu? A Case of Unintended Memorization in Self-supervised Learning