Open AccessAsk me in your own words: paraphrasing for multitask question answering
Author(s) -
G. Thomas Hudson,
Noura Al Moubayed
Publication year - 2021
Publication title -
peerj. computer science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.927
H-Index - 70
ISSN - 2376-5992
DOI - 10.7717/peerj-cs.759
Subject(s) - paraphrase , question answering , computer science , natural language processing , artificial intelligence , robustness (evolution) , sentence , leverage (statistics) , task (project management) , ask price , natural language understanding , transfer of learning , natural language , machine learning , biochemistry , chemistry , management , economics , gene , economy
Multitask learning has led to significant advances in Natural Language Processing, including the decaNLP benchmark where question answering is used to frame 10 natural language understanding tasks in a single model. In this work we show how models trained to solve decaNLP fail with simple paraphrasing of the question. We contribute a crowd-sourced corpus of paraphrased questions (PQ-decaNLP), annotated with paraphrase phenomena. This enables analysis of how transformations such as swapping the class labels and changing the sentence modality lead to a large performance degradation. Training both MQAN and the newer T5 model using PQ-decaNLP improves their robustness and for some tasks improves the performance on the original questions, demonstrating the benefits of a model which is more robust to paraphrasing. Additionally, we explore how paraphrasing knowledge is transferred between tasks, with the aim of exploiting the multitask property to improve the robustness of the models. We explore the addition of paraphrase detection and paraphrase generation tasks, and find that while both models are able to learn these new tasks, knowledge about paraphrasing does not transfer to other decaNLP tasks.