English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

First proposed by Seide (2014) as a heuristic, error feedback (EF) is a verypopular mechanism for enforcing convergence of distributed gradient-basedoptimization methods enhanced with communication compression strategies basedon the application of contractive compression operators. However, existingtheory of EF relies on very strong assumptions (e.g., bounded gradients), andprovides pessimistic convergence rates (e.g., while the best known rate for EFin the smooth nonconvex regime, and when full gradients are compressed, is$O(1/T^{2/3})$, the rate of gradient descent in the same regime is $O(1/T)$).Recently, Richt\'{a}rik et al. (2021) proposed a new error feedback mechanism,EF21, based on the construction of a Markov compressor induced by a contractivecompressor. EF21 removes the aforementioned theoretical deficiencies of EF andat the same time works better in practice. In this work we propose sixpractical extensions of EF21, all supported by strong convergence theory:partial participation, stochastic approximation, variance reduction, proximalsetting, momentum and bidirectional compression. Several of these techniqueswere never analyzed in conjunction with EF before, and in cases where they were(e.g., bidirectional compression), our rates are vastly superior.

EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback