Premium
The relationship between the C‐statistic and the accuracy of program‐specific evaluations
Author(s) -
Wey Andrew,
Salkowski Nicholas,
Kasiske Bertram L.,
Skeans Melissa A.,
Gustafson Sally K.,
Israni Ajay K.,
Snyder Jon J.
Publication year - 2019
Publication title -
american journal of transplantation
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.89
H-Index - 188
eISSN - 1600-6143
pISSN - 1600-6135
DOI - 10.1111/ajt.15132
Subject(s) - statistic , medicine , statistics , cohen's kappa , metric (unit) , hazard ratio , hazard , confidence interval , mathematics , operations management , chemistry , organic chemistry , economics
The C‐statistic of the risk‐adjustment model is often used to judge the accuracy of program evaluations. However, the C‐statistic depends on the variability in risk for individual transplants and may be inappropriate for determining the accuracy of program evaluations. A simulation study investigated the association of the C‐statistic with several metrics of program evaluation accuracy, including categorizing programs into the 5‐tier system and identifying programs for regulatory review. The simulation study used data from deceased donor kidney‐alone transplants for adult recipients in the program‐specific reports released January 2018. A range of C‐statistics was generated by changing the variability in risk for individual transplants. The C‐statistic had no association with any metric of program evaluation accuracy. Instead, the number of expected events at a program was the most important factor. For example, Spearman's rho, which is the correlation of ranks, was −0.27 and −0.72 between the true program‐specific hazard ratios and assigned tiers for programs with, respectively, <3 and >10 expected events. Presence of unadjusted risk factors did not modify the associations, although the accuracy of program evaluations was systematically lower. Therefore, the C‐statistic provides no information on the accuracy of program evaluations.