English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep neural networks have strong capabilities of memorizing the underlyingtraining data, which can be a serious privacy concern. An effective solution tothis problem is to train models with differential privacy, which providesrigorous privacy guarantees by injecting random noise to the gradients. Thispaper focuses on the scenario where sensitive data are distributed amongmultiple participants, who jointly train a model through federated learning(FL), using both secure multiparty computation (MPC) to ensure theconfidentiality of each gradient update, and differential privacy to avoid dataleakage in the resulting model. A major challenge in this setting is thatcommon mechanisms for enforcing DP in deep learning, which inject real-valuednoise, are fundamentally incompatible with MPC, which exchanges finite-fieldintegers among the participants. Consequently, most existing DP mechanismsrequire rather high noise levels, leading to poor model utility. Motivated bythis, we propose Skellam mixture mechanism (SMM), an approach to enforce DP onmodels built via FL. Compared to existing methods, SMM eliminates theassumption that the input gradients must be integer-valued, and, thus, reducesthe amount of noise injected to preserve DP. Further, SMM allows tight privacyaccounting due to the nice composition and sub-sampling properties of theSkellam distribution, which are key to accurate deep learning with DP. Thetheoretical analysis of SMM is highly non-trivial, especially considering (i)the complicated math of differentially private deep learning in general and(ii) the fact that the mixture of two Skellam distributions is rather complex,and to our knowledge, has not been studied in the DP literature. Extensiveexperiments on various practical settings demonstrate that SMM consistently andsignificantly outperforms existing solutions in terms of the utility of theresulting model.

Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacy