Open AccessS 2 ‐Net: Machine reading comprehension with SRU‐based self‐matching networks
Author(s) -
Park Cheoneum,
Lee Changki,
Hong Lynn,
Hwang Yigyu,
Yoo Taejoon,
Jang Jaeyong,
Hong Yunki,
Bae KyungHoon,
Kim HyunKi
Publication year - 2019
Publication title -
etri journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.295
H-Index - 46
eISSN - 2233-7326
pISSN - 1225-6463
DOI - 10.4218/etrij.2017-0279
Subject(s) - recurrent neural network , computer science , context (archaeology) , artificial intelligence , matching (statistics) , artificial neural network , reading comprehension , encoder , comprehension , coreference , deep learning , reading (process) , machine learning , natural language processing , resolution (logic) , mathematics , statistics , paleontology , political science , law , biology , operating system , programming language
Machine reading comprehension is the task of understanding a given context and finding the correct response in that context. A simple recurrent unit (SRU) is a model that solves the vanishing gradient problem in a recurrent neural network (RNN) using a neural gate, such as a gated recurrent unit (GRU) and long short‐term memory (LSTM); moreover, it removes the previous hidden state from the input gate to improve the speed compared to GRU and LSTM. A self‐matching network, used in R‐Net, can have a similar effect to coreference resolution because the self‐matching network can obtain context information of a similar meaning by calculating the attention weight for its own RNN sequence. In this paper, we construct a dataset for Korean machine reading comprehension and propose an S 2 ‐Net model that adds a self‐matching layer to an encoder RNN using multilayer SRU. The experimental results show that the proposed S 2 ‐Net model has performance of single 68.82% EM and 81.25% F1, and ensemble 70.81% EM, 82.48% F1 in the Korean machine reading comprehension test dataset, and has single 71.30% EM and 80.37% F1 and ensemble 73.29% EM and 81.54% F1 performance in the SQuAD dev dataset.