English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Communication between workers and the master node to collect local stochasticgradients is a key bottleneck in a large-scale federated learning system.Various recent works have proposed to compress the local stochastic gradientsto mitigate the communication overhead. However, robustness to maliciousattacks is rarely considered in such a setting. In this work, we investigatethe problem of Byzantine-robust compressed federated learning, where theattacks from Byzantine workers can be arbitrarily malicious. We theoreticallypoint out that different to the attacks-free compressed stochastic gradientdescent (SGD), its vanilla combination with geometric median-based robustaggregation seriously suffers from the compression noise in the presence ofByzantine attacks. In light of this observation, we propose to reduce thecompression noise with gradient difference compression so as to improve theByzantine-robustness. We also observe the impact of the intrinsic stochasticnoise caused by selecting random samples, and adopt the stochastic averagegradient algorithm (SAGA) to gradually eliminate the inner variations ofregular workers. We theoretically prove that the proposed algorithm reaches aneighborhood of the optimal solution at a linear convergence rate, and theasymptotic learning error is in the same order as that of the state-of-the-artuncompressed method. Finally, numerical experiments demonstrate theeffectiveness of the proposed method.

BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learning