Premium
Comparison of typical evaluation methods for computer‐aided diagnostic schemes: Monte Carlo simulation study
Author(s) -
Li Qiang,
Doi Kunio
Publication year - 2007
Publication title -
medical physics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.473
H-Index - 180
eISSN - 2473-4209
pISSN - 0094-2405
DOI - 10.1118/1.2437130
Subject(s) - cad , computer science , monte carlo method , medical physics , scheme (mathematics) , generalization , data mining , machine learning , reliability engineering , statistics , mathematics , engineering drawing , medicine , engineering , mathematical analysis
Computer‐aided diagnostic (CAD) schemes have been developed for assisting radiologists in the detection of various lesions in medical images. Many evaluation approaches, such as the resubstitution, leave‐one‐out, cross‐validation, and hold‐out methods, have been employed for the assessment of the performance of various CAD schemes. For these evaluation methods, some investigators have studied their bias in the estimated performance levels of CAD schemes trained with finite samples. However, systematical study has not been conducted for the comparison of these common evaluation methods in terms of multiple important characteristics such as the bias of the estimated performance, the generalization performance, and the uniqueness of the trained CAD scheme. Therefore, in this study, we examined and compared these important characteristics for various evaluation methods and attempted to provide a guideline for investigators to select appropriate evaluation methods for the assessment of CAD schemes in typical practical situations.