Premium
The Coverage Properties of Confidence Regions After Model Selection
Author(s) -
Kabaila Paul
Publication year - 2009
Publication title -
international statistical review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.051
H-Index - 54
eISSN - 1751-5823
pISSN - 0306-7734
DOI - 10.1111/j.1751-5823.2009.00089.x
Subject(s) - frequentist inference , inference , model selection , statistical inference , selection (genetic algorithm) , computer science , confidence interval , a priori and a posteriori , confidence distribution , statistical model , fiducial inference , coverage probability , statistical hypothesis testing , econometrics , multiple comparisons problem , data mining , statistics , machine learning , bayesian inference , artificial intelligence , mathematics , bayesian probability , philosophy , epistemology
Summary It is very common in applied frequentist (“classical”) statistics to carry out a preliminary statistical (i.e. data‐based) model selection by, for example, using preliminary hypothesis tests or minimizing AIC. This is usually followed by the inference of interest, using the same data, based on the assumption that the selected model had been given to us a priori . This assumption is false and it can lead to an inaccurate and misleading inference. We consider the important case that the inference of interest is a confidence region. We review the literature that shows that the resulting confidence regions typically have very poor coverage properties. We also briefly review the closely related literature that describes the coverage properties of prediction intervals after preliminary statistical model selection. A possible motivation for preliminary statistical model selection is a wish to utilize uncertain prior information in the inference of interest. We review the literature in which the aim is to utilize uncertain prior information directly in the construction of confidence regions, without requiring the intermediate step of a preliminary statistical model selection. We also point out this aim as a future direction for research.