Premium
How do cognitive processes influence script concordance test responses?
Author(s) -
Gawad Nada,
Wood Timothy J.,
Cowley Lindsay,
Raiche Isabelle
Publication year - 2021
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/medu.14416
Subject(s) - psychology , test (biology) , cognition , deliberation , concordance , comprehension , social psychology , cognitive psychology , scripting language , process (computing) , applied psychology , developmental psychology , computer science , medicine , paleontology , neuroscience , politics , political science , law , biology , programming language , operating system
Abstract Introduction The script concordance test (SCT) is a test of clinical decision‐making (CDM) that compares the thought process of learners to that of experts to determine to what extent their cognitive ‘scripts’ align. Without understanding test‐takers’ cognitive process, however, it is unclear what influences their responses. The objective of this study was to gather response process validity evidence by studying the cognitive process of test‐takers to determine whether the SCT tests CDM and what cognitive processes may influence SCT responses. Methods Cases from an SCT used in a national validation study were administered and semi‐structured cognitive interviews were conducted with ten residents and five staff surgeons. A retrospective verbal probing technique was used. Data was independently analysed and coded by two analysts. Themes were identified as factors that influence SCT responses during the cognitive interview. Results Cognitive interviews demonstrated variability in CDM among test‐takers. Consistent with dual process theory, test‐takers relied on scripts formed through past experiences, when available, to make decisions and used conscious deliberation in the absence of experience. However, test‐takers’ response process was also influenced by their comprehension of specific terms, desire for additional information, disagreement with the planned management, underlying knowledge gaps and desire to demonstrate confidence or humility. Conclusion The rationale behind SCT answers may be influenced by comprehension, underlying knowledge and social desirability in addition to formed scripts and/or conscious deliberation. Having test‐takers verbalise their rationale for responses provides a depth of assessment that is otherwise lost in the SCT’s current format. With the improved ability to standardise CDM assessment using the SCT, consideration of test‐makers improving the SCT construction process and combining the SCT question format with verbal responses may improve the use of the SCT for CDM assessment.