English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Differentially Private Stochastic Gradient Descent (DP-SGD) limits the amountof private information deep learning models can memorize during training. Thisis achieved by clipping and adding noise to the model's gradients, and thusnetworks with more parameters require proportionally stronger perturbation. Asa result, large models have difficulties learning useful information, renderingtraining with DP-SGD exceedingly difficult on more challenging training tasks.Recent research has focused on combating this challenge through trainingadaptations such as heavy data augmentation and large batch sizes. However,these techniques further increase the computational overhead of DP-SGD andreduce its practical applicability. In this work, we propose using theprinciple of sparse model design to solve precisely such complex tasks withfewer parameters, higher accuracy, and in less time, thus serving as apromising direction for DP-SGD. We achieve such sparsity by design byintroducing equivariant convolutional networks for model training withDifferential Privacy. Using equivariant networks, we show that small andefficient architecture design can outperform current state-of-the-art modelswith substantially lower computational requirements. On CIFAR-10, we achieve anincrease of up to $9\%$ in accuracy while reducing the computation time by morethan $85\%$. Our results are a step towards efficient model architectures thatmake optimal use of their parameters and bridge the privacy-utility gap betweenprivate and non-private deep learning for computer vision.

Equivariant Differentially Private Deep Learning: Why DP-SGD Needs Sparser Models