Premium
Use of statistical analysis of cytologic interpretation to determine the causes of interobserver disagreement and in quality improvement
Author(s) -
Renshaw Andrew A.,
Lee Kenneth R.,
Granter Scott R.
Publication year - 1997
Publication title -
cancer cytopathology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.052
H-Index - 304
eISSN - 1097-0142
pISSN - 0008-543X
DOI - 10.1002/(sici)1097-0142(19970825)81:4<212::aid-cncr3>3.0.co;2-r
Subject(s) - ascus (bryozoa) , medicine , receiver operating characteristic , squamous intraepithelial lesion , kappa , cytology , medical diagnosis , cohen's kappa , statistics , cancer , radiology , pathology , cervical cancer , cervical intraepithelial neoplasia , mathematics , botany , geometry , ascospore , spore , biology
BACKGROUND Disagreements in cytologic interpretation can have several causes, including differences in diagnostic threshold and diagnostic accuracy. These can be distinguished by a combination of statistical analyses. METHODS For demonstration purposes, a nonrandom collection of 80 cervicovaginal smears, the majority of which (74) were originally diagnosed as atypical cells of undetermined significance (ASCUS), were reviewed by 3 separate observers and classified as either negative, negative and reactive, ASCUS favor reactive, ASCUS not otherwise specified, ASCUS suggestive of a squamous intraepithelial lesion (SIL), low grade SIL, or high grade SIL. The results were compared with corresponding biopsies and analyzed with distribution analysis, the kappa statistic, threshold analysis, and receiver operating characteristic (ROC) curve analysis. RESULTS Distribution analysis of diagnoses from the three observers demonstrated statistically significant differences in how cases were classified and a low level of agreement. Kappa analysis confirmed a very poor interobserver agreement. Threshold analysis revealed that one observer used a threshold between negative and ASCUS that was statistically more specific but less sensitive than the other observers. ROC curve analysis showed that another observer was more accurate than this observer. CONCLUSIONS Variation in cytologic interpretation may have several causes. Distribution, threshold, and ROC analysis allow distinction between differences in diagnostic accuracy and diagnostic thresholds. This approach to analyzing cytologic interpretation may be useful for quality improvement efforts. Cancer (Cancer Cytopathol) 1997; 81:212‐9. © 1997 American Cancer Society.