Premium
Digital Module 12: Think‐aloud Interviews and Cognitive Labs https://ncme.elevate.commpartners.com
Author(s) -
Leighton Jacqueline P.,
Lehman Blair
Publication year - 2020
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/emip.12321
Subject(s) - credibility , think aloud protocol , cognitive interview , glossary , interview , cognition , data collection , psychology , computer science , protocol analysis , comprehension , reliability (semiconductor) , test (biology) , multimedia , human–computer interaction , cognitive science , usability , linguistics , philosophy , mathematics , law , biology , paleontology , power (physics) , quantum mechanics , political science , programming language , statistics , physics , neuroscience
Abstract In this digital ITEMS module, Dr. Jacqueline Leighton and Dr. Blair Lehman review differences between think‐aloud interviews to measure problem‐solving processes and cognitive labs to measure comprehension processes. Learners are introduced to historical, theoretical, and procedural differences between these methods and how to use and analyze distinct types of verbal reports in the collection of evidence of test‐taker response processes. The module includes details on (a) the different types of cognition that are tapped by different interviewer probes, (b) traditional interviewing methods and new automated tools for collecting verbal reports, and (c) options for analyses of verbal reports. This includes a discussion of reliability and validity issues such as potential bias in the collection of verbal reports, ways to mitigate bias, and inter‐rater agreement to enhance credibility of analysis. A novel digital tool for data collection called the ABC tool is presented via illustrative videos. As always, the module contains audio‐narrated slides, quiz questions with feedback, a glossary, and curated resources.