English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The allreduce collective operation accounts for a significant fraction of theruntime of workloads running on distributed systems. One factor determining itsperformance is the distance between communicating nodes, especially on networkslike torus, where a higher distance implies multiple messages being forwardedon the same link, thus reducing the allreduce bandwidth. Torus networks arewidely used on systems optimized for machine learning workloads (e.g., GoogleTPUs and Amazon Trainium devices), as well as on some of the Top500supercomputers. To improve allreduce performance on torus networks we introduceSwing, a new algorithm that keeps a low distance between communicating nodes byswinging between torus directions. Our analysis and experimental evaluationshow that Swing outperforms by up to 3x existing allreduce algorithms forvectors ranging from 32B to 128MiB, on different types of torus and torus-liketopologies, regardless of their shape and size.

Swing: Short-cutting Rings for Higher Bandwidth Allreduce