English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

There is growing interest in large-scale machine learning and optimizationover decentralized networks, e.g. in the context of multi-agent learning andfederated learning. Due to the imminent need to alleviate the communicationburden, the investigation of communication-efficient distributed optimizationalgorithms - particularly for empirical risk minimization - has flourished inrecent years. A large fraction of these algorithms have been developed for themaster/slave setting, relying on a central parameter server that cancommunicate with all agents. This paper focuses on distributed optimizationover networks, or decentralized optimization, where each agent is only allowedto aggregate information from its neighbors. By properly adjusting the globalgradient estimate via local averaging in conjunction with proper correction, wedevelop a communication-efficient approximate Newton-type method Network-DANE,which generalizes DANE to the decentralized scenarios. Our key ideas can beapplied in a systematic manner to obtain decentralized versions of othermaster/slave distributed algorithms. A notable development isNetwork-SVRG/SARAH, which employs variance reduction to further acceleratelocal computation. We establish linear convergence of Network-DANE andNetwork-SVRG for strongly convex losses, and Network-SARAH for quadraticlosses, which shed light on the impacts of data homogeneity, networkconnectivity, and local averaging upon the rate of convergence. We furtherextend Network-DANE to composite optimization by allowing a nonsmooth penaltyterm. Numerical evidence is provided to demonstrate the appealing performanceof our algorithms over competitive baselines, in terms of both communicationand computation efficiency. Our work suggests that performing a certain amountof local communications and computations per iteration can substantiallyimprove the overall efficiency.

Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction