Premium
A Simulation Study on the Performance of the Simple Difference and Covariance‐Adjusted Scores in Randomized Experimental Designs
Author(s) -
Petscher Yaacov,
Schatschneider Christopher
Publication year - 2011
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.2010.00129.x
Subject(s) - analysis of covariance , randomized experiment , statistics , covariance , completely randomized design , mathematics , sample size determination , normality , research design , repeated measures design , analysis of variance , variance (accounting) , correlation , econometrics , accounting , business , geometry
Research by Huck and McLean (1975) demonstrated that the covariance‐adjusted score is more powerful than the simple difference score, yet recent reviews indicate researchers are equally likely to use either score type in two‐wave randomized experimental designs. A Monte Carlo simulation was conducted to examine the conditions under which the simple difference and covariance‐adjusted scores were more or less powerful to detect treatment effects when relaxing certain assumptions made by Huck and McLean (1975) . Four factors were manipulated in the design including sample size, normality of the pretest and posttest distributions, the correlation between pretest and posttest, and posttest variance. A 5 × 5 × 4 × 3 mostly crossed design was run with 1,000 replications per condition, resulting in 226,000 unique samples. The gain score was nearly as powerful as the covariance‐adjusted score when pretest and posttest variances were equal, and as powerful in fan‐spread growth conditions; thus, under certain circumstances the gain score could be used in two‐wave randomized experimental designs.