English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Fueled by advances in distributed deep learning (DDL), recent years havewitnessed a rapidly growing demand for resource-intensive distributed/parallelcomputing to process DDL computing jobs. To resolve network communicationbottleneck and load balancing issues in distributed computing, the so-called``ring-all-reduce'' decentralized architecture has been increasingly adopted toremove the need for dedicated parameter servers. To date, however, thereremains a lack of theoretical understanding on how to design resourceoptimization algorithms for efficiently scheduling ring-all-reduce DDL jobs incomputing clusters. This motivates us to fill this gap by proposing a series ofnew resource scheduling designs for ring-all-reduce DDL jobs. Our contributionsin this paper are three-fold: i) We propose a new resource schedulinganalytical model for ring-all-reduce deep learning, which covers a wide rangeof objectives in DDL performance optimization (e.g., excessive trainingavoidance, energy efficiency, fairness); ii) Based on the proposed performanceanalytical model, we develop an efficient resource scheduling algorithm calledGADGET (greedy ring-all-reduce distributed graph embedding technique), whichenjoys a provable strong performance guarantee; iii) We conduct extensivetrace-driven experiments to demonstrate the effectiveness of the GADGETapproach and its superiority over the state of the art.

GADGET: Online Resource Optimization for Scheduling Ring-All-Reduce Learning Jobs