English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning (DL) has achieved notable successes in many machine learningtasks. A number of frameworks have been developed to expedite the process ofdesigning and training deep neural networks (DNNs), such as Caffe, Torch andTheano. Currently they can harness multiple GPUs on a single machine, but areunable to use GPUs that are distributed across multiple machines; as evenaverage-sized DNNs can take days to train on a single GPU with 100s of GBs toTBs of data, distributed GPUs present a prime opportunity for scaling up DL.However, the limited bandwidth available on commodity Ethernet networkspresents a bottleneck to distributed GPU training, and prevents its trivialrealization.  To investigate how to adapt existing frameworks to efficiently supportdistributed GPUs, we propose Poseidon, a scalable system architecture fordistributed inter-machine communication in existing DL frameworks. We integratePoseidon with Caffe and evaluate its performance at training DNNs for objectrecognition. Poseidon features three key contributions that accelerate DNNtraining on clusters: (1) a three-level hybrid architecture that allowsPoseidon to support both CPU-only and GPU-equipped clusters, (2) a distributedwait-free backpropagation (DWBP) algorithm to improve GPU utilization and tobalance communication, and (3) a structure-aware communication protocol (SACP)to minimize communication overheads. We empirically show that Poseidonconverges to same objectives as a single machine, and achieves state-of-arttraining speedup across multiple models and well-established datasets using acommodity GPU cluster of 8 nodes (e.g. 4.5x speedup on AlexNet, 4x onGoogLeNet, 4x on CIFAR-10). On the much larger ImageNet22K dataset, Poseidonwith 8 nodes achieves better speedup and competitive accuracy to recentCPU-based distributed systems such as Adam and Le et al., which use 10s to1000s of nodes.

Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines