English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advances in distributed optimization and learning have shown thatcommunication compression is one of the most effective means of reducingcommunication. While there have been many results on convergence rates undercommunication compression, a theoretical lower bound is still missing.  Analyses of algorithms with communication compression have attributedconvergence to two abstract properties: the unbiased property or thecontractive property. They can be applied with either unidirectionalcompression (only messages from workers to server are compressed) orbidirectional compression. In this paper, we consider distributed stochasticalgorithms for minimizing smooth and non-convex objective functions undercommunication compression. We establish a convergence lower bound foralgorithms whether using unbiased or contractive compressors in unidirection orbidirection. To close the gap between the lower bound and the existing upperbounds, we further propose an algorithm, NEOLITHIC, which almost reaches ourlower bound (up to logarithm factors) under mild conditions. Our results alsoshow that using contractive bidirectional compression can yield iterativemethods that converge as fast as those using unbiased unidirectionalcompression. The experimental results validate our findings.

Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression