Premium
Theory Testing Using Quantitative Predictions of Effect Size
Author(s) -
Velicer Wayne F.,
Cumming Geoff,
Fava Joseph L.,
Rossi Joseph S.,
Prochaska James O.,
Johnson Janet
Publication year - 2008
Publication title -
applied psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.497
H-Index - 88
eISSN - 1464-0597
pISSN - 0269-994X
DOI - 10.1111/j.1464-0597.2008.00348.x
Subject(s) - null hypothesis , transtheoretical model , statistics , sample size determination , confidence interval , ordinal data , statistical hypothesis testing , econometrics , mathematics , psychology , social psychology , behavior change
Traditional Null Hypothesis Testing procedures are poorly adapted to theory testing. The methodology can mislead researchers in several ways, including: (a) a lack of power can result in an erroneous rejection of the theory; (b) the focus on directionality (ordinal tests) rather than more precise quantitative predictions limits the information gained; and (c) the misuse of probability values to indicate effect size. An alternative approach is proposed which involves employing the theory to generate explicit effect size predictions that are compared to the effect size estimates and related confidence intervals to test the theoretical predictions. This procedure is illustrated employing the Transtheoretical Model. Data from a sample ( N = 3,967) of smokers from a large New England HMO system were used to test the model. There were a total of 15 predictions evaluated, each involving the relation between Stage of Change and one of the other 15 Transtheoretical Model variables. For each variable, omega‐squared and the related confidence interval were calculated and compared to the predicted effect sizes. Eleven of the 15 predictions were confirmed, providing support for the theoretical model. Quantitative predictions represent a much more direct, informative, and strong test of a theory than the traditional test of significance.