English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Neural machine translation - using neural networks to translate humanlanguage - is an area of active research exploring new neuron types and networktopologies with the goal of dramatically improving machine translationperformance. Current state-of-the-art approaches, such as the multi-headattention-based transformer, require very large translation corpuses and manyepochs to produce models of reasonable quality. Recent attempts to parallelizethe official TensorFlow "Transformer" model across multiple nodes have hitroadblocks due to excessive memory use and resulting out of memory errors whenperforming MPI collectives. This paper describes modifications made to theHorovod MPI-based distributed training framework to reduce memory usage fortransformer models by converting assumed-sparse tensors to dense tensors, andsubsequently replacing sparse gradient gather with dense gradient reduction.The result is a dramatic increase in scale-out capability, with CPU-onlyscaling tests achieving 91% weak scaling efficiency up to 1200 MPI processes(300 nodes), and up to 65% strong scaling efficiency up to 400 MPI processes(200 nodes) using the Stampede2 supercomputer.

Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI  Collective Performance during Tensor Accumulation for Parallelized Training  of Neural Machine Translation Models