English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Federated learning (FL) is a distributed paradigm that coordinates massivelocal clients to collaboratively train a global model via stage-wise localtraining processes on the heterogeneous dataset. Previous works have implicitlystudied that FL suffers from the ``client-drift'' problem, which is caused bythe inconsistent optimum across local clients. However, till now it still lackssolid theoretical analysis to explain the impact of this local inconsistency.To alleviate the negative impact of the ``client drift'' and explore itssubstance in FL, in this paper, we first design an efficient FL algorithm\textit{FedInit}, which allows employing the personalized relaxedinitialization state at the beginning of each local training stage.Specifically, \textit{FedInit} initializes the local state by moving away fromthe current global state towards the reverse direction of the latest localstate. This relaxed initialization helps to revise the local divergence andenhance the local consistency level. Moreover, to further understand howinconsistency disrupts performance in FL, we introduce the excess risk analysisand study the divergence term to investigate the test error of the proposed\textit{FedInit} method. Our studies show that optimization error is notsensitive to this local inconsistency, while it mainly affects thegeneralization error bound in \textit{FedInit}. Extensive experiments areconducted to validate this conclusion. Our proposed \textit{FedInit} couldachieve state-of-the-art~(SOTA) results compared to several advanced benchmarkswithout any additional costs. Meanwhile, stage-wise relaxed initializationcould also be incorporated into the current advanced algorithms to achievehigher performance in the FL paradigm.

Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization