English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Geo-distributed ML training can benefit many emerging ML scenarios (e.g.,large model training, federated learning) with multi-regional cloud resourcesand wide area network. However, its efficiency is limited due to 2 challenges.First, efficient elastic scheduling of multi-regional cloud resources isusually missing, affecting resource utilization and performance of training.Second, training communication on WAN is still the main overhead, easilysubjected to low bandwidth and high fluctuations of WAN. In this paper, wepropose a framework, Cloudless-Training, to realize efficient PS-basedgeo-distributed ML training in 3 aspects. First, it uses a two-layerarchitecture with control and physical training planes to support elasticscheduling and communication for multi-regional clouds in a serverlessmaner.Second, it provides an elastic scheduling strategy that can deploytraining workflows adaptively according to the heterogeneity of available cloudresources and distribution of pre-existing training datasets. Third, itprovides 2 new synchronization strategies for training partitions among clouds,including asynchronous SGD with gradient accumulation (ASGD-GA) and inter-PSmodel averaging (MA). It is implemented with OpenFaaS and evaluated on TencentCloud. Experiments show that Cloudless-Training can support general ML trainingin a geo-distributed way, greatly improve resource utilization (e.g.,9.2%-24.0% training cost reduction) and synchronization efficiency (e.g., 1.7xtraining speedup over baseline at most) with model correctness guarantees.

Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training