Open AccessEffect of Speech Recognition on Problem Solving and Recall in Consumer Digital Health Tasks: Controlled Laboratory Experiment
Author(s) -
Jessica Chen,
David Lyell,
Liliana Laranjo,
Farah Magrabi
Publication year - 2020
Publication title -
jmir. journal of medical internet research/journal of medical internet research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.446
H-Index - 142
eISSN - 1439-4456
pISSN - 1438-8871
DOI - 10.2196/14827
Subject(s) - recall , computer science , usability , task (project management) , cognitive load , cognition , speech recognition , human–computer interaction , natural language processing , psychology , cognitive psychology , engineering , systems engineering , neuroscience
Background Recent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However, little is known about the use of speech recognition in consumer health applications and few studies have evaluated the efficacy of conversational agents in the hands of consumers. In other consumer-facing tools, cognitive load has been observed to be an important factor affecting the use of speech recognition technologies in tasks involving problem solving and recall. Users find it more difficult to think and speak at the same time when compared to typing, pointing, and clicking. However, the effects of speech recognition on cognitive load when performing health tasks has not yet been explored. Objective The aim of this study was to evaluate the use of speech recognition for documentation in consumer digital health tasks involving problem solving and recall. Methods Fifty university staff and students were recruited to undertake four documentation tasks with a simulated conversational agent in a computer laboratory. The tasks varied in complexity determined by the amount of problem solving and recall required (simple and complex) and the input modality (speech recognition vs keyboard and mouse). Cognitive load, task completion time, error rate, and usability were measured. Results Compared to using a keyboard and mouse, speech recognition significantly increased the cognitive load for complex tasks ( Z =–4.08, P <.001) and simple tasks ( Z =–2.24, P =.03). Complex tasks took significantly longer to complete ( Z =–2.52, P =.01) and speech recognition was found to be overall less usable than a keyboard and mouse ( Z =–3.30, P =.001). However, there was no effect on errors. Conclusions Use of a keyboard and mouse was preferable to speech recognition for complex tasks involving problem solving and recall. Further studies using a broader variety of consumer digital health tasks of varying complexity are needed to investigate the contexts in which use of speech recognition is most appropriate. The effects of cognitive load on task performance and its significance also need to be investigated.