English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We develop and analyze MARINA: a new communication efficient method fornon-convex distributed learning over heterogeneous datasets. MARINA employs anovel communication compression strategy based on the compression of gradientdifferences that is reminiscent of but different from the strategy employed inthe DIANA method of Mishchenko et al. (2019). Unlike virtually all competingdistributed first-order methods, including DIANA, ours is based on a carefullydesigned biased gradient estimator, which is the key to its superiortheoretical and practical performance. The communication complexity bounds weprove for MARINA are evidently better than those of all previous first-ordermethods. Further, we develop and analyze two variants of MARINA: VR-MARINA andPP-MARINA. The first method is designed for the case when the local lossfunctions owned by clients are either of a finite sum or of an expectationform, and the second method allows for a partial participation of clients -- afeature important in federated learning. All our methods are superior toprevious state-of-the-art methods in terms of oracle/communication complexity.Finally, we provide a convergence analysis of all methods for problemssatisfying the Polyak-Lojasiewicz condition.

MARINA: Faster Non-Convex Distributed Learning with Compression