English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We investigate the generalization error of statistical learning models in aFederated Learning (FL) setting. Specifically, we study the evolution of thegeneralization error with the number of communication rounds $R$ between $K$clients and a parameter server (PS), i.e., the effect on the generalizationerror of how often the clients' local models are aggregated at PS. In oursetup, the more the clients communicate with PS the less data they use forlocal training in each round, such that the amount of training data per clientis identical for distinct values of $R$. We establish PAC-Bayes andrate-distortion theoretic bounds on the generalization error that accountexplicitly for the effect of the number of rounds $R$, in addition to thenumber of participating devices $K$ and individual datasets size $n$. Thebounds, which apply to a large class of loss functions and learning algorithms,appear to be the first of their kind for the FL setting. Furthermore, we applyour bounds to FL-type Support Vector Machines (FSVM); and derive (more)explicit bounds in this case. In particular, we show that the generalizationbound of FSVM increases with $R$, suggesting that more frequent communicationwith PS diminishes the generalization power. This implies that the populationrisk decreases less fast with $R$ than does the empirical risk. Moreover, ourbound suggests that the generalization error of FSVM decreases faster than thatof centralized learning by a factor of $\mathcal{O}(\sqrt{\log(K)/K})$.Finally, we provide experimental results obtained using neural networks(ResNet-56) which show evidence that not only may our observations for FSVMhold more generally but also that the population risk may even start toincrease beyond some value of $R$.

Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!