English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern neural networks require long training to reach decent performance onmassive datasets. One common approach to speed up training is modelparallelization, where large neural networks are split across multiple devices.However, different device placements of the same neural network lead todifferent training times. Most of the existing device placement solutions treatthe problem as sequential decision-making by traversing neural network graphsand assigning their neurons to different devices. This work studies the impactof graph traversal order on device placement. In particular, we empiricallystudy how different graph traversal order leads to different device placement,which in turn affects the training execution time. Our experiment results showthat the best graph traversal order depends on the type of neural networks andtheir computation graphs features. In this work, we also providerecommendations on choosing graph traversal order in device placement forvarious neural network families to improve the training time in modelparallelization.

Accelerate Model Parallel Training by Using Efficient Graph Traversal Order in Device Placement