English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While differential privacy and gradient compression are separatelywell-researched topics in machine learning, the study of interaction betweenthese two topics is still relatively new. We perform a detailed empirical studyon how the Gaussian mechanism for differential privacy and gradient compressionjointly impact test accuracy in deep learning. The existing literature ingradient compression mostly evaluates compression in the absence ofdifferential privacy guarantees, and demonstrate that sufficiently highcompression rates reduce accuracy. Similarly, existing literature indifferential privacy evaluates privacy mechanisms in the absence ofcompression, and demonstrates that sufficiently strong privacy guaranteesreduce accuracy. In this work, we observe while gradient compression generallyhas a negative impact on test accuracy in non-private training, it cansometimes improve test accuracy in differentially private training.Specifically, we observe that when employing aggressive sparsification or rankreduction to the gradients, test accuracy is less affected by the Gaussiannoise added for differential privacy. These observations are explained throughan analysis how differential privacy and compression effects the bias andvariance in estimating the average gradient. We follow this study with arecommendation on how to improve test accuracy under the context ofdifferentially private deep learning and gradient compression. We evaluate thisproposal and find that it can reduce the negative impact of noise added bydifferential privacy mechanisms on test accuracy by up to 24.6%, and reduce thenegative impact of gradient sparsification on test accuracy by up to 15.1%.

On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning