English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A plethora of modern machine learning tasks require the utilization oflarge-scale distributed clusters as a critical component of the trainingpipeline. However, abnormal Byzantine behavior of the worker nodes can derailthe training and compromise the quality of the inference. Such behavior can beattributed to unintentional system malfunctions or orchestrated attacks; as aresult, some nodes may return arbitrary results to the parameter server (PS)that coordinates the training. Recent work considers a wide range of attackmodels and has explored robust aggregation and/or computational redundancy tocorrect the distorted gradients.  In this work, we consider attack models ranging from strong ones: $q$omniscient adversaries with full knowledge of the defense protocol that canchange from iteration to iteration to weak ones: $q$ randomly chosenadversaries with limited collusion abilities which only change every fewiterations at a time. Our algorithms rely on redundant task assignments coupledwith detection of adversarial behavior. We also show the convergence of ourmethod to the optimal point under common assumptions and settings considered inliterature. For strong attacks, we demonstrate a reduction in the fraction ofdistorted gradients ranging from 16%-99% as compared to the priorstate-of-the-art. Our top-1 classification accuracy results on the CIFAR-10data set demonstrate 25% advantage in accuracy (averaged over strong and weakscenarios) under the most sophisticated attacks compared to state-of-the-artmethods.

Detection and Mitigation of Byzantine Attacks in Distributed Training