Premium
An evaluation of resampling methods for assessment of survival risk prediction in high‐dimensional settings
Author(s) -
Subramanian Jyothi,
Simon Richard
Publication year - 2010
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.4106
Subject(s) - resampling , statistics , computer science , jackknife resampling , sample size determination , data mining , sample (material) , range (aeronautics) , mathematics , chemistry , materials science , chromatography , estimator , composite material
Resampling techniques are often used to provide an initial assessment of accuracy for prognostic prediction models developed using high‐dimensional genomic data with binary outcomes. Risk prediction is most important, however, in medical applications and frequently the outcome measure is a right‐censored time‐to‐event variable such as survival. Although several methods have been developed for survival risk prediction with high‐dimensional genomic data, there has been little evaluation of the use of resampling techniques for the assessment of such models. Using real and simulated datasets, we compared several resampling techniques for their ability to estimate the accuracy of risk prediction models. Our study showed that accuracy estimates for popular resampling methods, such as sample splitting and leave‐one‐out cross validation (Loo CV), have a higher mean square error than for other methods. Moreover, the large variability of the split‐sample and Loo CV may make the point estimates of accuracy obtained using these methods unreliable and hence should be interpreted carefully. A k ‐fold cross‐validation with k = 5 or 10 was seen to provide a good balance between bias and variability for a wide range of data settings and should be more widely adopted in practice. Published in 2010 by John Wiley & Sons, Ltd.