English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Model parallelism has become necessary to train large neural networks.However, finding a suitable model parallel schedule for an arbitrary neuralnetwork is a non-trivial task due to the exploding search space. In this work,we present a model parallelism framework TAP that automatically searches forthe best data and tensor parallel schedules. Leveraging the key insight that aneural network can be represented as a directed acyclic graph, within which mayonly exist a limited set of frequent subgraphs, we design a graph pruningalgorithm to fold the search space efficiently. TAP runs at sub-linearcomplexity concerning the neural network size. Experiments show that TAP is$20\times- 160\times$ faster than the state-of-the-art automatic parallelismframework, and the performance of its discovered schedules is competitive withthe expert-engineered ones.

TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation