English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Non-autoregressive Transformer (NAT) is a family of text generation models,which aims to reduce the decoding latency by predicting the whole sentences inparallel. However, such latency reduction sacrifices the ability to captureleft-to-right dependencies, thereby making NAT learning very challenging. Inthis paper, we present theoretical and empirical analyses to reveal thechallenges of NAT learning and propose a unified perspective to understandexisting successes. First, we show that simply training NAT by maximizing thelikelihood can lead to an approximation of marginal distributions but drops alldependencies between tokens, where the dropped information can be measured bythe dataset's conditional total correlation. Second, we formalize many previousobjectives in a unified framework and show that their success can be concludedas maximizing the likelihood on a proxy distribution, leading to a reducedinformation loss. Empirical studies show that our perspective can explain thephenomena in NAT learning and guide the design of new training methods.

On the Learning of Non-Autoregressive Transformers