Premium
Testing What Matters (If You Must Test at All): A Context‐Driven Approach to Substantive and Statistical Significance
Author(s) -
Gross Justin H.
Publication year - 2015
Publication title -
american journal of political science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.347
H-Index - 170
eISSN - 1540-5907
pISSN - 0092-5853
DOI - 10.1111/ajps.12149
Subject(s) - statistical hypothesis testing , null hypothesis , test (biology) , presentation (obstetrics) , statistical inference , context (archaeology) , statistical significance , psychology , interpretation (philosophy) , epistemology , cognitive psychology , computer science , econometrics , statistics , mathematics , medicine , paleontology , philosophy , radiology , biology , programming language
For over a half century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation and assessment of research results. A long list of criticisms has fueled the so‐called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (e.g., scientific, substantive, social, political) significance. I introduce a simple, intuitive approach that grounds testing in subject‐area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to test for real‐world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation.