English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recent work has shown that the encoder-decoder attention mechanisms in neuralmachine translation (NMT) are different from the word alignment in statisticalmachine translation. In this paper, we focus on analyzing encoder-decoderattention mechanisms, in the case of word sense disambiguation (WSD) in NMTmodels. We hypothesize that attention mechanisms pay more attention to contexttokens when translating ambiguous words. We explore the attention distributionpatterns when translating ambiguous nouns. Counter-intuitively, we find thatattention mechanisms are likely to distribute more attention to the ambiguousnoun itself rather than context tokens, in comparison to other nouns. Weconclude that attention mechanism is not the main mechanism used by NMT modelsto incorporate contextual information for WSD. The experimental results suggestthat NMT models learn to encode contextual information necessary for WSD in theencoder hidden states. For the attention mechanism in Transformer models, wereveal that the first few layers gradually learn to "align" source and targettokens and the last few layers learn to extract features from the related butunaligned context tokens.

An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation