Premium
The influence of hidden researcher decisions in applied microeconomics
Author(s) -
HuntingtonKlein Nick,
Arenas Andreu,
Beam Emily,
Bertoni Marco,
Bloem Jeffrey R.,
Burli Pralhad,
Chen Naibin,
Grieco Paul,
Ekpe Godwin,
Pugatch Todd,
Saavedra Martin,
Stopnitzky Yaniv
Publication year - 2021
Publication title -
economic inquiry
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.823
H-Index - 72
eISSN - 1465-7295
pISSN - 0095-2583
DOI - 10.1111/ecin.12992
Subject(s) - economics , microeconomics , mathematical economics , neoclassical economics
Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many‐analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3–4 times the mean reported standard error.