English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper considers the Byzantine fault-tolerance problem in distributedstochastic gradient descent (D-SGD) method - a popular algorithm fordistributed multi-agent machine learning. In this problem, each agent samplesdata points independently from a certain data-generating distribution. In thefault-free case, the D-SGD method allows all the agents to learn a mathematicalmodel best fitting the data collectively sampled by all agents. We consider thecase when a fraction of agents may be Byzantine faulty. Such faulty agents maynot follow a prescribed algorithm correctly, and may render traditional D-SGDmethod ineffective by sharing arbitrary incorrect stochastic gradients. Wepropose a norm-based gradient-filter, named comparative gradient elimination(CGE), that robustifies the D-SGD method against Byzantine agents. We show thatthe CGE gradient-filter guarantees fault-tolerance against a bounded fractionof Byzantine agents under standard stochastic assumptions, and iscomputationally simpler compared to many existing gradient-filters such asmulti-KRUM, geometric median-of-means, and the spectral filters. We empiricallyshow, by simulating distributed learning on neural networks, that thefault-tolerance of CGE is comparable to that of existing gradient-filters. Wealso empirically show that exponential averaging of stochastic gradientsimproves the fault-tolerance of a generic gradient-filter.

Byzantine Fault-Tolerant Distributed Machine Learning Using Stochastic Gradient Descent (SGD) and Norm-Based Comparative Gradient Elimination (CGE)