Premium
Confidence intervals around Bayes Cost in multi‐state diagnostic settings to estimate optimal performance
Author(s) -
Batterton Katherine A.,
Schubert Christine M.
Publication year - 2014
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.6174
Subject(s) - confidence interval , youden's j statistic , bayes' theorem , statistics , computer science , a priori and a posteriori , statistical hypothesis testing , feature (linguistics) , bayes classifier , artificial intelligence , mathematics , receiver operating characteristic , pattern recognition (psychology) , bayesian probability , philosophy , linguistics , epistemology
A critical feature of diagnostic testing is correctly classifying subjects based upon specified thresholds of some measure. The commonly employed Youden index determines a test's optimal thresholds by maximizing the correct classification rates for a diagnostic scenario. An alternative to the Youden index is the cost function, Bayes Cost ( BC ). BC determines a test's optimal setting by minimizing the sum of all misclassification rates from the test. Unlike the Youden index, BC can consider a priori costs of all the diagnostic outcomes including class specific misclassifications regardless of the number of classes. Delta method approximate confidence intervals around BC are derived under the assumption of normally distributed classes as a means for quantifying a test's performance and comparing classifiers at their optimal settings in a multi‐state diagnostic framework. A simulation study is conducted to demonstrate the performance of the derived confidence intervals that are found to perform well, especially for sample sizes of 50 or larger in each diagnostic class. Finally, the proposed methods are applied to a four‐class breast tissue classification problem, where four possible discriminatory features are compared under varying decision cost structures. Using the confidence intervals around BC , the best feature for classification is selected, and the optimal thresholds and their 95% confidence intervals are determined. Published 2014. This article is a U.S. Government work and is in the public domain in the USA.