English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Artificial intelligence (AI) has seen a tremendous surge in capabilitiesthanks to the use of foundation models trained on internet-scale data. On theflip side, the uncurated nature of internet-scale data also poses significantprivacy and legal risks, as they often contain personal information orcopyrighted material that should not be trained on without permission. In thiswork, we propose as a mitigation measure a recipe to train foundation visionmodels with differential privacy (DP) guarantee. We identify maskedautoencoders as a suitable learning algorithm that aligns well with DP-SGD, andtrain ViP -- a Vision transformer with differential Privacy -- under a strictprivacy budget of $\epsilon=8$ on the LAION400M dataset. We evaluate thequality of representation learned by ViP using standard downstream visiontasks; in particular, ViP achieves a (non-private) linear probing accuracy of$55.7\%$ on ImageNet, comparable to that of end-to-end trained AlexNet (trainedand evaluated on ImageNet). Our result suggests that scaling to internet-scaledata can be practical for private learning. Code is available at\url{https://github.com/facebookresearch/ViP-MAE}.

ViP: A Differentially Private Foundation Model for Computer Vision