Premium
Specification Searching and Significance Inflation Across Time, Methods and Disciplines
Author(s) -
Vivalt Eva
Publication year - 2019
Publication title -
oxford bulletin of economics and statistics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.131
H-Index - 73
eISSN - 1468-0084
pISSN - 0305-9049
DOI - 10.1111/obes.12289
Subject(s) - inflation (cosmology) , statistical significance , set (abstract data type) , randomized controlled trial , randomized experiment , econometrics , economics , psychology , statistics , medicine , computer science , mathematics , physics , surgery , theoretical physics , programming language
This paper examines how significance inflation has varied across time, methods and disciplines. Leveraging a unique data set of impact evaluations on 20 kinds of development programmes, I find that results from randomized controlled trials exhibit less significance inflation than results from studies using other methods. Further, randomized controlled trials have exhibited less significance inflation over time, but quasi‐experimental studies have not. There is no robust difference between results from researchers affiliated with economics departments and those from researchers affiliated with other predominantly health‐related departments. Overall, the biases found appear much smaller than those previously observed in other social sciences.