English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning systems have become vital tools across many fields, but theincreasing model sizes mean that training must be accelerated to maintain suchsystems' utility. Current systems like Tensorflow and MXNet focus on onespecific parallelization strategy, data parallelism, which requires largetraining batch sizes in order to scale. We cast the problem of finding the bestparallelization strategy as the problem of finding the best tiling to partitiontensors with the least overall communication. We propose an algorithm that canfind the optimal tiling. Our resulting parallelization solution is a hybrid ofdata parallelism and model parallelism. We build the SoyBean system thatperforms automatic parallelization. SoyBean automatically transforms a serialdataflow graph captured by an existing deep learning system frontend into aparallel dataflow graph based on the optimal tiling it has found. Ourevaluations show that SoyBean is 1.5x-4x faster than pure data parallelism forAlexNet and VGG. We present this automatic tiling in a new system, SoyBean,that can act as a backend for Tensorflow, MXNet, and others.

Unifying Data, Model and Hybrid Parallelism in Deep Learning via Tensor Tiling