Premium
Mostly harmless simulations? Using Monte Carlo studies for estimator selection
Author(s) -
Advani Arun,
Kitagawa Toru,
Słoczyński Tymon
Publication year - 2019
Publication title -
journal of applied econometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.878
H-Index - 99
eISSN - 1099-1255
pISSN - 0883-7252
DOI - 10.1002/jae.2724
Subject(s) - estimator , monte carlo method , robustness (evolution) , computer science , econometrics , mean squared error , selection (genetic algorithm) , range (aeronautics) , statistics , mathematics , machine learning , engineering , biochemistry , chemistry , gene , aerospace engineering
Summary We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be satisfied in many contexts. To test empirical relevance, we also apply the approaches to a real‐world setting where estimator performance is known. Both approaches are worse than random at selecting estimators that minimize absolute bias. They are better when selecting estimators that minimize mean squared error. However, using a simple bootstrap is at least as good and often better. For now, researchers would be best advised to use a range of estimators and compare estimates for robustness.