Premium
Estimating effect size: Bias resulting from the significance criterion in editorial decisions
Author(s) -
Lane David M.,
Dunlap William P.
Publication year - 1978
Publication title -
british journal of mathematical and statistical psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.157
H-Index - 51
eISSN - 2044-8317
pISSN - 0007-1102
DOI - 10.1111/j.2044-8317.1978.tb00578.x
Subject(s) - statistics , sample size determination , econometrics , statistical significance , mathematics , population , population size , demography , sociology
Experiments that find larger differences between groups than actually exist in the population are more likely to pass stringent tests of significance and be published than experiments that find smaller differences. Published measures of the magnitude of experimental effects will therefore tend to overestimate these effects. This bias was investigated as a function of sample size, actual population difference, and alpha level. The overestimation of experimental effects was found to be quite large with the commonly employed significance levels of 5 per cent and 1 per cent. Further, the recently recommended measure, ω 2 , was found to depend much more heavily on the alpha level employed than the true population ω 2 value. Hence, it was concluded that effect size estimation is impractical unless scientific journals drop the consideration of statistical significance as one of the criteria of publication.