English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Decentralized optimization is effective to save communication in large-scalemachine learning. Although numerous algorithms have been proposed withtheoretical guarantees and empirical successes, the performance limits indecentralized optimization, especially the influence of network topology andits associated weight matrix on the optimal convergence rate, have not beenfully understood. While (Lu and Sa, 2021) have recently provided an optimalrate for non-convex stochastic decentralized optimization with weight matricesdefined over linear graphs, the optimal rate with general weight matricesremains unclear.  This paper revisits non-convex stochastic decentralized optimization andestablishes an optimal convergence rate with general weight matrices. Inaddition, we also establish the optimal rate when non-convex loss functionsfurther satisfy the Polyak-Lojasiewicz (PL) condition. Following existing linesof analysis in literature cannot achieve these results. Instead, we leveragethe Ring-Lattice graph to admit general weight matrices while maintaining theoptimal relation between the graph diameter and weight matrix connectivity.Lastly, we develop a new decentralized algorithm to nearly attain the above twooptimal rates under additional mild conditions.

Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization