English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large convolutional neural networks (CNN) can be difficult to train in thedifferentially private (DP) regime, since the optimization algorithms require acomputationally expensive operation, known as the per-sample gradient clipping.We propose an efficient and scalable implementation of this clipping onconvolutional layers, termed as the mixed ghost clipping, that significantlyeases the private training in terms of both time and space complexities,without affecting the accuracy. The improvement in efficiency is rigorouslystudied through the first complexity analysis for the mixed ghost clipping andexisting DP training algorithms.  Extensive experiments on vision classification tasks, with large ResNet, VGG,and Vision Transformers, demonstrate that DP training with mixed ghost clippingadds $1\sim 10\%$ memory overhead and $<2\times$ slowdown to the standardnon-private training. Specifically, when training VGG19 on CIFAR10, the mixedghost clipping is $3\times$ faster than state-of-the-art Opacus library with$18\times$ larger maximum batch size. To emphasize the significance ofefficient DP training on convolutional layers, we achieve 96.7\% accuracy onCIFAR10 and 83.0\% on CIFAR100 at $\epsilon=1$ using BEiT, while the previousbest results are 94.8\% and 67.4\%, respectively. We open-source a privacyengine (\url{https://github.com/woodyx218/private_vision}) that implements DPtraining of CNN with a few lines of code.

Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy