English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Decentralized nonconvex optimization has received increasing attention inrecent years in machine learning due to its advantages in system robustness,data privacy, and implementation simplicity. However, three fundamentalchallenges in designing decentralized optimization algorithms are how to reducetheir sample, communication, and memory complexities. In this paper, we proposea \underline{g}radient-\underline{t}racking-based \underline{sto}chastic\underline{r}ecursive \underline{m}omentum (GT-STORM) algorithm for efficientlysolving nonconvex optimization problems. We show that to reach an$\epsilon^2$-stationary solution, the total number of sample evaluations of ouralgorithm is $\tilde{O}(m^{1/2}\epsilon^{-3})$ and the number of communicationrounds is $\tilde{O}(m^{-1/2}\epsilon^{-3})$, which improve the$O(\epsilon^{-4})$ costs of sample evaluations and communications for theexisting decentralized stochastic gradient algorithms. We conduct extensiveexperiments with a variety of learning models, including non-convex logisticalregression and convolutional neural networks, to verify our theoreticalfindings. Collectively, our results contribute to the state of the art oftheories and algorithms for decentralized network optimization.

GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning