English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The Exponential Mechanism (ExpM), a differentially private optimizationmethod, promises many advantages over Differentially Private StochasticGradient Descent (DPSGD), the state-of-the-art (SOTA) and de facto method fordifferentially private machine learning (ML). Yet, ExpM has been historicallystymied from differentially private training of modern ML algorithms by twoobstructions: ExpM requires a sensitivity bound for the given loss function;ExpM requires sampling from a historically intractable density. We prove asensitivity bound for $\ell(2)$ loss, and investigate using Normalizing Flows(NFs), deep networks furnishing approximate sampling from the otherwiseintractable ExpM distribution. We prove that as the NF output converges to ExpMdistribution, the privacy ($\varepsilon$) of an NF sample converges to that ofthe ExpM distribution. Under the assumption that the NF output distribution isthe ExpM distribution, we empirically test ExpM+NF against DPSGD using the SOTAimplementation (Opacus \cite{opacus} with PRV accounting) in multipleclassification tasks on the Adult Dataset (census data) and MIMIC-III Dataset(healthcare records) using Logistic Regression and GRU-D, a deep learningrecurrent neural network with \smallsim 20K-100K parameters. In all experimentswe find ExpM+NF achieves greater than 94\% of the non-private training accuracy(AUC) with $\varepsilon$-DP for $\varepsilon$ a low as $1\mathrm{e}{-3}$ --three orders of magnitude stronger privacy with similar accuracy. Further,performance results show ExpM+NF training time is comparable to (slightly less)than DPSGD. Limitations and future directions are provided; notably, researchon NF approximation accuracy and its effect on privacy are a promising avenueto substantially advancing the field. Code for these experiments \hl{will beprovided after review}.

ExpM+NF Tractable Exponential Mechanism via Normalizing Flow, A Path through the Accuracy-Privacy Ceiling Constraining Differentially Private ML