English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we revisit the problem of using in-distribution public data toimprove the privacy/utility trade-offs for differentially private (DP) modeltraining. (Here, public data refers to auxiliary data sets that have no privacyconcerns.) We design a natural variant of DP mirror descent, where the DPgradients of the private/sensitive data act as the linear term, and the lossgenerated by the public data as the mirror map.  We show that, for linear regression with feature vectors drawn from anon-isotropic sub-Gaussian distribution, our algorithm, PDA-DPMD (a variant ofmirror descent), provides population risk guarantees that are asymptoticallybetter than the best known guarantees under DP (without having access to publicdata), when the number of public data samples ($n_{\sf pub}$) is sufficientlylarge. We further show that our algorithm has natural "noise stability"properties that control the variance due to noise added to ensure DP.  We demonstrate the efficacy of our algorithm by showing privacy/utilitytrade-offs on four benchmark datasets (StackOverflow, WikiText-2, CIFAR-10, andEMNIST). We show that our algorithm not only significantly improves overtraditional DP-SGD, which does not have access to public data, but to ourknowledge is the first to improve over DP-SGD on models that have beenpre-trained with public data.

Public Data-Assisted Mirror Descent for Private Model Training