Premium
Comparison of remote and in‐person digital speech‐based measures of cognition
Author(s) -
Stegmann Gabriela M,
Hahn Shira,
Liss Julie,
Berisha Visar,
Mueller Kimberly D
Publication year - 2021
Publication title -
alzheimer's and dementia
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.713
H-Index - 118
eISSN - 1552-5279
pISSN - 1552-5260
DOI - 10.1002/alz.056438
Subject(s) - pronoun , noun , relevance (law) , test (biology) , cognition , psychology , task (project management) , proper noun , neuropsychology , computer science , cognitive psychology , natural language processing , artificial intelligence , linguistics , paleontology , philosophy , management , neuroscience , political science , law , economics , biology
Background Neuropsychological testing requires an in‐person visit with a trained administrator using standard/fixed materials. Speech‐based cognitive testing on mobile devices enables more frequent and timely test administration, but head‐to‐head comparisons of in‐person and remote versions of tests are rare. We compare responses to a well‐validated task conducted under supervised (in‐person) and unsupervised (remote) conditions. Method We used two data sets containing Cookie Theft picture descriptions (BDAE), one collected in‐person, under supervision (Wisconsin Registry for Alzheimer’s Prevention (WRAP); N = 912; age = 62.2 (SD = 6.7), 70% F) and one collected remotely, without supervision (participants online; N = 93, age = 36.5 (SD = 11.5), 65% F). WRAP participants were judged to be cognitively normal, and online participants self‐reported as healthy. Each participant provided one picture description, yielding 93 remote and 912 in‐person transcribed descriptions. Language metrics previously used in dementia research were extracted: word count (number of words spoken), MATTR (ratio of unique words to total number of words), pronoun‐to‐noun ratio, and semantic relevance. Comparing MATTR, pronoun‐to‐noun ratio, and semantic relevance values elicited in‐person and remotely is important because these characteristics have been found to be impacted by declines in cognition. Differences in word counts may reflect different levels of motivation in the two settings. Result Using Cohen’s d effect sizes, differences in mean word count between in‐person and remote participant transcripts was negligible (112 words in‐person and 118 words remote; d = .10, p = .21), the difference for semantic relevance was negligible (.19 in‐person and .18 remote, scale = 0 to 1; d = .14, p = .19), the difference for MATTR was small (.75 in‐person and .74 remote, scale = 0 to 1; d = .37, p<.05), and the difference for pronoun‐to‐noun ratio was moderate (.48 in‐person and .35 remote, scale = 0 to 1; d = .56, p <.05). Conclusion Results show that response length and semantic relevance of responses to the Cookie Theft picture description task are comparable under supervised, in‐person and unsupervised, remote conditions. Small to moderate differences in vocabulary (MATTR) and pronoun‐to‐noun ratio may be due to differences in age and merit further investigation.