English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Attention-based autoregressive models have achieved state-of-the-artperformance in various sequence-to-sequence tasks, including Text-To-Speech(TTS) and Neural Machine Translation (NMT), but can be difficult to train. Thestandard training approach, teacher forcing, guides a model with the referenceback-history. During inference, the generated back-history must be used. Thismismatch limits the evaluation performance. Attention forcing has beenintroduced to address the mismatch, guiding the model with the generatedback-history and reference attention. While successful in tasks with continuousoutputs like TTS, attention forcing faces additional challenges in tasks withdiscrete outputs like NMT. This paper introduces the two extensions ofattention forcing to tackle these challenges. (1) Scheduled attention forcingautomatically turns attention forcing on and off, which is essential for taskswith discrete outputs. (2) Parallel attention forcing makes training parallel,and is applicable to Transformer-based models. The experiments show that theproposed approaches improve the performance of models based on RNNs andTransformers.

Parallel Attention Forcing for Machine Translation