English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Task-based distributed frameworks (e.g., Ray, Dask, Hydro) have becomeincreasingly popular for distributed applications that contain asynchronous anddynamic workloads, including asynchronous gradient descent, reinforcementlearning, and model serving. As more data-intensive applications move to run ontop of task-based systems, collective communication efficiency has become animportant problem. Unfortunately, traditional collective communicationlibraries (e.g., MPI, Horovod, NCCL) are an ill fit, because they require thecommunication schedule to be known before runtime and they do not provide faulttolerance.  We design and implement Hoplite, an efficient and fault-tolerant collectivecommunication layer for task-based distributed systems. Our key technique is tocompute data transfer schedules on the fly and execute the schedulesefficiently through fine-grained pipelining. At the same time, when a taskfails, the data transfer schedule adapts quickly to allow other tasks to keepmaking progress. We apply Hoplite to a popular task-based distributedframework, Ray. We show that Hoplite speeds up asynchronous stochastic gradientdescent, reinforcement learning, and serving an ensemble of machine learningmodels that are difficult to execute efficiently with traditional collectivecommunication by up to 7.8x, 3.9x, and 3.3x, respectively.

Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems