Receiver operating characteristic analysis under tree orderings of disease classes

Receiver operating characteristic (ROC) curve and its summary statistics (e.g., the area under curve (AUC)) are commonly used to evaluate the diagnostic accuracy for disease processes with binary classification. The ROC curve has been extended to ROC surface for scenarios with three ordinal classes or to hyper‐surface for scenarios with more than three classes. For classifier under tree or umbrella ordering in which the marker measurement for one class is lower or higher than those for the other classes, the commonly adopted diagnostic measures are the naive AUC ( NAUC ) based on a pooled class of all the unordered classes and the umbrella volume ( UV ) based on the concept of volume under surface. However, both NAUC and UV have some limitations. For example, NAUC depends on the sampling weights for all the classes in population, and UV has only been introduced for three‐class settings. In this article, we initiate the idea of a new ROC framework for tree or umbrella ordering (denoted as TROC ) and propose the area under TROC curve (denoted as TAUC ) as an appropriate diagnostic measure. The proposed TROC and TAUC share many nice features with the traditional ROC and AUC. Both parametric and nonparametric approaches are explored to construct the confidence interval estimation of TAUC . The performances of these methods are compared in simulation studies under a variety settings. At the end, the proposed methods are applied to a published microarray data set. Copyright © 2015 John Wiley & Sons, Ltd.