English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The success of large neural networks is crucially determined by theavailability of data. It has been observed that training only on a small amountof public data, or privately on the abundant private data can lead toundesirable degradation of accuracy. In this work, we leverage both private andpublic data to improve the optimization, by coupling their gradients via aweighted linear combination. We formulate an optimal solution for the optimalweight in the convex setting to indicate that the weighting coefficient shouldbe hyperparameter-dependent. Then, we prove the acceleration in the convergenceof non-convex loss and the effects of hyper-parameters such as privacy budget,number of iterations, batch size, and model size on the choice of the weightingcoefficient. We support our analysis with empirical experiments across languageand vision benchmarks, and provide a guideline for choosing the optimal weightof the gradient coupling.

Coupling public and private gradient provably helps optimization