Premium
Randomization inference for treatment effect variation
Author(s) -
Ding Peng,
Feller Avi,
Miratrix Luke
Publication year - 2016
Publication title -
journal of the royal statistical society: series b (statistical methodology)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.523
H-Index - 137
eISSN - 1467-9868
pISSN - 1369-7412
DOI - 10.1111/rssb.12124
Subject(s) - randomized experiment , variation (astronomy) , randomization , inference , econometrics , treatment effect , covariate , randomized controlled trial , causal inference , nuisance parameter , treatment and control groups , computer science , restricted randomization , statistical hypothesis testing , statistics , mathematics , artificial intelligence , medicine , physics , surgery , estimator , astrophysics , traditional medicine
Summary Applied researchers are increasingly interested in whether and how treatment effects vary in randomized evaluations, especially variation that is not explained by observed covariates. We propose a model‐free approach for testing for the presence of such unexplained variation. To use this randomization‐based approach, we must address the fact that the average treatment effect, which is generally the object of interest in randomized experiments, actually acts as a nuisance parameter in this setting. We explore potential solutions and advocate for a method that guarantees valid tests in finite samples despite this nuisance. We also show how this method readily extends to testing for heterogeneity beyond a given model, which can be useful for assessing the sufficiency of a given scientific theory. We finally apply our method to the National Head Start impact study, which is a large‐scale randomized evaluation of a Federal preschool programme, finding that there is indeed significant unexplained treatment effect variation.