English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Neural network language model (NNLM) is an essential component of industrialASR systems. One important challenge of training an NNLM is to leverage betweenscaling the learning process and handling big data. Conventional approachessuch as block momentum provides a blockwise model update filtering (BMUF)process and achieves almost linear speedups with no performance degradation forspeech recognition. However, it needs to calculate the model average from allcomputing nodes (e.g., GPUs) and when the number of computing nodes is large,the learning suffers from the severe communication latency. As a consequence,BMUF is not suitable under restricted network conditions. In this paper, wepresent a decentralized BMUF process, in which the model is split intodifferent components, each of which is updated by communicating to somerandomly chosen neighbor nodes with the same component, followed by a BMUF-likeprocess. We apply this method to several LSTM language modeling tasks.Experimental results show that our approach achieves consistently betterperformance than conventional BMUF. In particular, we obtain a lower perplexitythan the single-GPU baseline on the wiki-text-103 benchmark using 4 GPUs. Inaddition, no performance degradation is observed when scaling to 8 and 16 GPUs.

A Random Gossip BMUF Process for Neural Language Modeling