English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Gradient coding is a technique for straggler mitigation in distributedlearning. In this paper we design novel gradient codes using tools fromclassical coding theory, namely, cyclic MDS codes, which compare favorably withexisting solutions, both in the applicable range of parameters and in thecomplexity of the involved algorithms. Second, we introduce an approximatevariant of the gradient coding problem, in which we settle for approximategradient computation instead of the exact one. This approach enables gracefuldegradation, i.e., the $\ell_2$ error of the approximate gradient is adecreasing function of the number of stragglers. Our main result is thatnormalized adjacency matrices of expander graphs yield excellent approximategradient codes, which enable significantly less computation compared to exactgradient coding, and guarantee faster convergence than trivial solutions understandard assumptions. We experimentally test our approach on Amazon EC2, andshow that the generalization error of approximate gradient coding is very closeto the full gradient while requiring significantly less computation from theworkers.

Gradient Coding from Cyclic MDS Codes and Expander Graphs