English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper presents a new generalization error analysis for the DecentralizedStochastic Gradient Descent (D-SGD) algorithm based on algorithmic stability.The obtained results largely improve upon state-of-the-art results, and eveninvalidate their claims that the communication graph has a detrimental effecton generalization. For instance, we show that in convex settings, D-SGD has thesame generalization bounds as the classical SGD algorithm, no matter the choiceof graph. We exhibit that this counter-intuitive result comes from consideringthe average of local parameters, which hides a final global averaging stepincompatible with the decentralized scenario. In light of this observation, weadvocate to analyze the supremum over local parameters and show that in thiscase, the graph does have an impact on the generalization. Unlike priorresults, our analysis yields non-vacuous bounds even for non-connected graphs.

Improved Stability and Generalization Analysis of the Decentralized SGD Algorithm