Premium
Effect of Eliciting Verbal Reports of Thinking on Critical Thinking Test Performance
Author(s) -
Norris Stephen P.
Publication year - 1990
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.1990.tb00733.x
Subject(s) - psychology , interview , test (biology) , critical thinking , generality , social psychology , developmental psychology , mathematics education , paleontology , political science , law , psychotherapist , biology
Verbal reports of examinees' thinking on multiple‐choice critical thinking test items can provide useful validation data only if the verbal reporting does not change the course of examinees' thinking and performance. Using a completely randomized factorial design, 343 senior high school students were divided into five groups. In four of the groups, different procedures were used to elicit students' thinking as they worked through Part A of a critical thinking test of observation appraisal (Norris & King, 1983). In the control group, students took the same test in paper‐and‐pencil format. There were no significant differences in test performance among the five groups nor in the quality of thinking among the four groups from whom verbal reports of thinking were elicited. These results are evidence that verbal reports of thinking can meet one of the necessary conditions of useful validation data—namely, that collecting the data does not alter examinees' thinking and performance. Some analyses found significant interviewer main effects and sex‐by‐interviewer and elicitation‐level‐by‐interviewer‐by‐sex‐by‐grade interaction effects. Analysis of these interactions suggested that the role of the interviewer might limit the generality of the technique.