Open AccessImproving Word Alignment by Adding Gromov-Wasserstein into Attention Neural Network
Author(s) -
Yan Huang,
Tianyuan Zhang,
Hui Zhu
Publication year - 2022
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/2171/1/012043
Subject(s) - computer science , word (group theory) , machine translation , sentence , translation (biology) , task (project management) , artificial intelligence , layer (electronics) , generative grammar , natural language processing , artificial neural network , head (geology) , generative model , linguistics , biochemistry , chemistry , management , organic chemistry , geomorphology , messenger rna , economics , gene , geology , philosophy
Statistical machine translation systems usually break the translation task into two or more subtasks and an important one is finding word alignments over a parallel sentence bilingual corpus. We address the problem of introducing word alignment for language pairs by developing a novel neural network model that can applied to other generative alignment models. We use Multi-layer attention model and multi-layer model with multi-head-attention mechanism on each layer provides superior translation quality. It can be trained on bilingual data without relying on word alignment. In this paper, we cast the correspondence problem directly as an optimal distance problem. We use the Gromov-Wasserstein distance to calculated how similarities between word pairs are related across languages. The resulting alignments dramatically outperform the GIZA++ and FastAlign approach, these alignments are comparable on public data sets.