English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The convergence speed of machine learning models trained with FederatedLearning is significantly affected by heterogeneous data partitions, even moreso in a fully decentralized setting without a central server. In this paper, weshow that the impact of label distribution skew, an important type of dataheterogeneity, can be significantly reduced by carefully designing theunderlying communication topology. We present D-Cliques, a novel topology thatreduces gradient bias by grouping nodes in sparsely interconnected cliques suchthat the label distribution in a clique is representative of the global labeldistribution. We also show how to adapt the updates of decentralized SGD toobtain unbiased gradients and implement an effective momentum with D-Cliques.Our extensive empirical evaluation on MNIST and CIFAR10 demonstrates that ourapproach provides similar convergence speed as a fully-connected topology,which provides the best convergence in a data heterogeneous setting, with asignificant reduction in the number of edges and messages. In a 1000-nodetopology, D-Cliques require 98% less edges and 96% less total messages, withfurther possible gains using a small-world topology across cliques.

D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning