English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We propose TopoOpt, a novel direct-connect fabric for deep neural network(DNN) training workloads. TopoOpt co-optimizes the distributed training processacross three dimensions: computation, communication, and network topology. Wedemonstrate the mutability of AllReduce traffic, and leverage this property toconstruct efficient network topologies for DNN training jobs. TopoOpt then usesan alternating optimization technique and a group theory-inspired algorithmcalled TotientPerms to find the best network topology and routing plan,together with a parallelization strategy. We build a fully functional 12-nodedirect-connect prototype with remote direct memory access (RDMA) forwarding at100 Gbps. Large-scale simulations on real distributed training models showthat, compared to similar-cost Fat-Tree interconnects, TopoOpt reduces DNNtraining time by up to 3.4x.

TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs