Premium
Comparing ROC curves derived from regression models
Author(s) -
Seshan Venkatraman E.,
Gönen Mithat,
Begg Colin B.
Publication year - 2012
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.5648
Subject(s) - wald test , statistics , resampling , receiver operating characteristic , score test , regression , predictive power , statistic , regression analysis , variance (accounting) , test statistic , econometrics , computer science , likelihood ratio test , mathematics , statistical hypothesis testing , philosophy , accounting , epistemology , business
In constructing predictive models, investigators frequently assess the incremental value of a predictive marker by comparing the ROC curve generated from the predictive model including the new marker with the ROC curve from the model excluding the new marker. Many commentators have noticed empirically that a test of the two ROC areas often produces a non‐significant result when a corresponding Wald test from the underlying regression model is significant. A recent article showed using simulations that the widely used ROC area test produces exceptionally conservative test size and extremely low power. In this article, we demonstrate that both the test statistic and its estimated variance are seriously biased when predictions from nested regression models are used as data inputs for the test, and we examine in detail the reasons for these problems. Although it is possible to create a test reference distribution by resampling that removes these biases, Wald or likelihood ratio tests remain the preferred approach for testing the incremental contribution of a new marker. Copyright © 2012 John Wiley & Sons, Ltd.