Premium
A Multivariate Generalizability Analysis of Data from a Performance Assessment of Physicians' Clinical Skills
Author(s) -
Clauser Brian E.,
Harik Polina,
Margolis Melissa J.
Publication year - 2006
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.2006.00012.x
Subject(s) - generalizability theory , multivariate statistics , context (archaeology) , multivariate analysis , multivariate analysis of variance , facet (psychology) , variance (accounting) , psychology , test (biology) , variance components , statistics , computer science , econometrics , mathematics , social psychology , accounting , personality , big five personality traits , business , biology , paleontology
Although multivariate generalizability theory was developed more than 30 years ago, little published research utilizing this framework exists and most of what does exist examines tests built from tables of specifications. In this context, it is assumed that the universe scores from levels of the fixed multivariate facet will be correlated, but the error terms will be uncorrelated because subscores result from mutually exclusive sets of test items. This paper reports on an application in which multiple subscores are derived from each task completed by the examinee. In this context, both universe scores and errors may be correlated across levels of the fixed multivariate facet. The data described come from the United States Medical Licensing Examination ® Step 2 Clinical Skills Examination. In this test, each examinee interacts with a series of standardized patients and each interaction results in four component scores. The paper focuses on the application of multivariate generalizability theory in this context and on the practical interpretation of the resulting estimated variance and covariance components.