Premium
“ statcheck ”: Automatically detect statistical reporting inconsistencies to increase reproducibility of meta‐analyses
Author(s) -
Nuijten Michèle B.,
Polanin Joshua R.
Publication year - 2020
Publication title -
research synthesis methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.376
H-Index - 35
eISSN - 1759-2887
pISSN - 1759-2879
DOI - 10.1002/jrsm.1408
Subject(s) - statistic , computer science , meta analysis , value (mathematics) , statistical hypothesis testing , statistical analysis , meaning (existential) , reproducibility , statistics , data science , information retrieval , data mining , psychology , medicine , machine learning , mathematics , pathology , psychotherapist
We present the R package and web app statcheck to automatically detect statistical reporting inconsistencies in primary studies and meta‐analyses. Previous research has shown a high prevalence of reported p ‐values that are inconsistent ‐ meaning a re‐calculated p‐ value, based on the reported test statistic and degrees of freedom, does not match the author‐reported p ‐value. Such inconsistencies affect the reproducibility and evidential value of published findings. The tool statcheck can help researchers to identify statistical inconsistencies so that they may correct them. In this paper, we provide an overview of the prevalence and consequences of statistical reporting inconsistencies. We also discuss the tool statcheck in more detail and give an example of how it can be used in a meta‐analysis. We end with some recommendations concerning the use of statcheck in meta‐analyses and make a case for better reporting standards of statistical results.