English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Federated Averaging (FedAvg), also known as Local SGD, is one of the mostpopular algorithms in Federated Learning (FL). Despite its simplicity andpopularity, the convergence rate of FedAvg has thus far been undetermined. Evenunder the simplest assumptions (convex, smooth, homogeneous, and boundedcovariance), the best-known upper and lower bounds do not match, and it is notclear whether the existing analysis captures the capacity of the algorithm. Inthis work, we first resolve this question by providing a lower bound for FedAvgthat matches the existing upper bound, which shows the existing FedAvg upperbound analysis is not improvable. Additionally, we establish a lower bound in aheterogeneous setting that nearly matches the existing upper bound. While ourlower bounds show the limitations of FedAvg, under an additional assumption ofthird-order smoothness, we prove more optimistic state-of-the-art convergenceresults in both convex and non-convex settings. Our analysis stems from anotion we call iterate bias, which is defined by the deviation of theexpectation of the SGD trajectory from the noiseless gradient descenttrajectory with the same initialization. We prove novel sharp bounds on thisquantity, and show intuitively how to analyze this quantity from a StochasticDifferential Equation (SDE) perspective.

Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective