Premium
What sources contribute to variance in observer ratings? Using generalizability theory to assess construct validity of psychological measures
Author(s) -
Lakes Kimberley D.,
Hoyt William T.
Publication year - 2008
Publication title -
infant and child development
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.87
H-Index - 57
eISSN - 1522-7219
pISSN - 1522-7227
DOI - 10.1002/icd.551
Subject(s) - generalizability theory , psychology , situational ethics , variance (accounting) , construct (python library) , construct validity , context (archaeology) , social psychology , external validity , cognitive psychology , applied psychology , developmental psychology , psychometrics , accounting , computer science , business , programming language , paleontology , biology
Cronbach and Meehl ( Psychol. Bull . 1955; 52:281–302) stated that the key question to be addressed when assessing construct validity is ‘What sources contribute to variance in test performance?’ We illustrate the utility of generalizability theory (GT) as a conceptual framework that encourages psychological researchers to address this question and as a flexible set of analytic tools that can provide answers to inform both substantive theory and measurement practice. To illustrate these capabilities, we analyze observer ratings of 27 caregiver–child dyads, focusing on the importance of situational (contextual) factors as sources of variance in observer ratings of caregiver–child behaviors. Cross‐situational consistency was relatively low for the categories of behavior analyzed, indicating that dyads vary greatly in their interactional patterns from one situation to the next, so that it is difficult to predict behavioral frequencies in one context from behaviors observed in a different context. Our findings suggest that single‐situation behavioral measures may have limited generalizability, either to behavior in other contexts or as measures of global interaction tendencies. We discuss the implications of these findings for research and measurement design in developmental psychology. Copyright © 2008 John Wiley & Sons, Ltd.