Open AccessA data-driven caption for L2 listening
Author(s) -
Maryam Sadat Mirzaei,
Kourosh Meshgi
Publication year - 2020
Language(s) - English
Resource type - Conference proceedings
DOI - 10.36505/exling-2020/11/0042/000457
Subject(s) - computer science , classifier (uml) , artificial intelligence , false positive paradox , word (group theory) , feature selection , labeled data , task (project management) , natural language processing , feature (linguistics) , speech recognition , pattern recognition (psychology) , philosophy , linguistics , management , economics
Partial and Synchronized Caption (PSC) is a tool that automatically detects difficult segments for the second language (L2) listeners and displays them in the caption while omitting easy-to-recognize cases to reduce cognitive load. Given that the number of words to be shown in this caption is limited, the main challenge lies in selecting and prioritizing difficult words. Since partialization is a classifying task, we made a dataset of labeled words in TED talks (easy vs. difficult) for a target proficiency-level. A deep classifier is trained on this dataset to automate the detection of difficult words/phrases without explicitly extracting word features. This proposed data-driven PSC outperforms its feature-based versions by adopting a selection pattern that is more similar to the annotations, capturing more complicated cases, and minimizing the false positives.