z-logo
open-access-imgOpen Access
Localizing category-related information in speech with multi-scale analyses
Author(s) -
Sam Tilsen,
Seung-Eun Kim,
Claire Wang
Publication year - 2021
Publication title -
plos one
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.99
H-Index - 332
ISSN - 1932-6203
DOI - 10.1371/journal.pone.0258178
Subject(s) - phrase , computer science , set (abstract data type) , linear discriminant analysis , vocal tract , mutual information , artificial intelligence , discriminant , speech recognition , natural language processing , scale (ratio) , artificial neural network , redundancy (engineering) , pattern recognition (psychology) , physics , quantum mechanics , programming language , operating system
Measurements of the physical outputs of speech—vocal tract geometry and acoustic energy—are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.
Having issues? You can contact us here