Premium
Application of Think Aloud Protocols for Examining and Confirming Sources of Differential Item Functioning Identified by Expert Reviews
Author(s) -
Ercikan Kadriye,
Arim Rubab,
Law Danielle,
Domene Jose,
Gag France,
Lacroix Serge
Publication year - 2010
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/j.1745-3992.2010.00173.x
Subject(s) - differential item functioning , think aloud protocol , psychology , empirical evidence , cognitive psychology , expert opinion , applied psychology , social psychology , item response theory , computer science , developmental psychology , psychometrics , epistemology , medicine , human–computer interaction , philosophy , usability , intensive care medicine
This paper demonstrates and discusses the use of think aloud protocols (TAPs) as an approach for examining and confirming sources of differential item functioning ( DIF). The TAPs are used to investigate to what extent surface characteristics of the items that are identified by expert reviews as sources of DIF are supported by empirical evidence from examinee thinking processes in the English and French versions of a Canadian national assessment. In this research, the TAPs confirmed sources of DIF identified by expert reviews for 10 out of 20 DIF items. The moderate agreement between TAPs and expert reviews indicates that evidence from expert reviews cannot be considered sufficient in deciding whether DIF items are biased and such judgments need to include evidence from examinee thinking processes.