English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning models have demonstrated impressive performance in variousdomains. However, the prolonged training time of these models remains acritical problem. Manually designed parallel training strategies could enhanceefficiency but require considerable time and deliver little flexibility. Hence,automatic parallelism is proposed to automate the parallel strategy searchingprocess. Even so, existing approaches suffer from sub-optimal strategy spacebecause they treat automatic parallelism as two independent stages, namelyinter- and intra-layer parallelism. To address this issue, we propose UniAP,which utilizes mixed integer quadratic programming to unify inter- andintra-layer automatic parallelism. To the best of our knowledge, UniAP is thefirst work to unify these two categories to search for a globally optimalstrategy. The experimental results show that UniAP outperforms state-of-the-artmethods by up to 1.70$\times$ in throughput and reduces strategy searching timeby up to 16$\times$ across four Transformer-like models.

UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming