Open AccessParagraph-Parallel based Neural Machine Translation Model with Hierarchical Attention
Author(s) -
Yuqi Zhang,
Gongshen Liu
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1453/1/012006
Subject(s) - computer science , machine translation , transformer , paragraph , encoder , natural language processing , artificial intelligence , example based machine translation , speech recognition , voltage , engineering , world wide web , electrical engineering , operating system
Neural Machine Translation (NMT) has achieved great developments in recent years, but we still have to face two challenges: establishing a high-quality corpus and exploring optimal parameters of models for long text translation. In this paper, we first attempt to set up a paragraph-parallel corpus based on English and Chinese versions of the novels and then design a hierarchical attention model for it to handle these two challenges. Our encoder and decoder take segmented clauses as input to process the words, clauses, paragraphs at different levels, particularly with a two-layer transformer to capture the context from both source and target languages. The output of the model based on the original transformer is used as another level of abstraction, conditioning on its own previous hidden states. During this process, inter-clause and intra-clause contexts from source and target sides are introduced for translation prediction and both encoder and decoder can profit from contexts in complementary ways. Experimental results show that our hierarchical attention model significantly outperforms six competitive baselines, including ensembles.