English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the problem of a Parameter Server (PS) that wishes to learn amodel that fits data distributed on the nodes of a graph. We focus on FederatedLearning (FL) as a canonical application. One of the main challenges of FL isthe communication bottleneck between the nodes and the parameter server. Apopular solution in the literature is to allow each node to do several localupdates on the model in each iteration before sending it back to the PS. Whilethis mitigates the communication bottleneck, the statistical heterogeneity ofthe data owned by the different nodes has proven to delay convergence and biasthe model. In this work, we study random walk (RW) learning algorithms fortackling the communication and data heterogeneity problems. The main idea is toleverage available direct connections among the nodes themselves, which aretypically "cheaper" than the communication to the PS. In a random walk, themodel is thought of as a "baton" that is passed from a node to one of itsneighbors after being updated in each iteration. The challenge in designing theRW is the data heterogeneity and the uncertainty about the data distributions.Ideally, we would want to visit more often nodes that hold more informativedata. We cast this problem as a sleeping multi-armed bandit (MAB) to design anear-optimal node sampling strategy that achieves variance-reduced gradientestimates and approaches sub-linearly the optimal sampling strategy. Based onthis framework, we present an adaptive random walk learning algorithm. Weprovide theoretical guarantees on its convergence. Our numerical resultsvalidate our theoretical findings and show that our algorithm outperformsexisting random walk algorithms.

Walk for Learning: A Random Walk Approach for Federated Learning from Heterogeneous Data