English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Differentially private SGD (DP-SGD) holds the promise of enabling the safeand responsible application of machine learning to sensitive datasets. However,DP-SGD only provides a biased, noisy estimate of a mini-batch gradient. Thisrenders optimisation steps less effective and limits model utility as a result.With this work, we show a connection between per-sample gradient norms and theestimation bias of the private gradient oracle used in DP-SGD. Here, we proposeBias-Aware Minimisation (BAM) that allows for the provable reduction of privategradient estimator bias. We show how to efficiently compute quantities neededfor BAM to scale to large neural networks and highlight similarities to closelyrelated methods such as Sharpness-Aware Minimisation. Finally, we provideempirical evidence that BAM not only reduces bias but also substantiallyimproves privacy-utility trade-offs on the CIFAR-10, CIFAR-100, and ImageNet-32datasets.

Bias-Aware Minimisation: Understanding and Mitigating Estimator Bias in Private SGD