English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We investigate the generalization error of statistical learning models in aFederated Learning (FL) setting. Specifically, we study the evolution of thegeneralization error with the number of communication rounds between theclients and the parameter server, i.e., the effect on the generalization errorof how often the local models as computed by the clients are aggregated at theparameter server. We establish PAC-Bayes and rate-distortion theoretic boundson the generalization error that account explicitly for the effect of thenumber of rounds, say $ R \in \mathbb{N}$, in addition to the number ofparticipating devices $K$ and individual datasets size $n$. The bounds, whichapply in their generality for a large class of loss functions and learningalgorithms, appear to be the first of their kind for the FL setting.Furthermore, we apply our bounds to FL-type Support Vector Machines (FSVM); andwe derive (more) explicit bounds on the generalization error in this case. Inparticular, we show that the generalization error of FSVM increases with $R$,suggesting that more frequent communication with the parameter serverdiminishes the generalization power of such learning algorithms. Combined withthat the empirical risk generally decreases for larger values of $R$, thisindicates that $R$ might be a parameter to optimize in order to minimize thepopulation risk of FL algorithms. Moreover, specialized to the case $R=1$(sometimes referred to as "one-shot" FL or distributed learning) our boundssuggest that the generalization error of the FL setting decreases faster thanthat of centralized learning by a factor of $\mathcal{O}(\sqrt{\log(K)/K})$,thereby generalizing recent findings in this direction to arbitrary lossfunctions and algorithms. The results of this paper are also validated on someexperiments.

Federated Learning You May Communicate Less Often!