English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Distributing Neural Network training is of particular interest for severalreasons including scaling using computing clusters, training at data sourcessuch as IOT devices and edge servers, utilizing underutilized resources acrossheterogeneous environments, and so on. Most contemporary approaches primarilyaddress scaling using computing clusters and require high network bandwidth andfrequent communication. This thesis presents an overview of standard approachesto distribute training and proposes a novel technique involvingpairwise-communication using Gossip-like protocols, called Elastic Gossip. Thisapproach builds upon an existing technique known as Elastic Averaging SGD(EASGD), and is similar to another technique called Gossiping SGD which alsouses Gossip-like protocols. Elastic Gossip is empirically evaluated againstGossiping SGD using the MNIST digit recognition and CIFAR-10 classificationtasks, using commonly used Neural Network architectures spanning Multi-LayerPerceptrons (MLPs) and Convolutional Neural Networks (CNNs). It is found thatElastic Gossip, Gossiping SGD, and All-reduce SGD perform quite comparably,even though the latter entails a substantially higher communication cost. WhileElastic Gossip performs better than Gossiping SGD in these experiments, it ispossible that a more thorough search over hyper-parameter space, specific to agiven application, may yield configurations of Gossiping SGD that work betterthan Elastic Gossip.

Elastic Gossip: Distributing Neural Network Training Using Gossip-like Protocols