Open AccessThe Use of “Overall Accuracy” to Evaluate the Validity of Screening or Diagnostic Tests
Author(s) -
Alberg Anthony J.,
Park Ji Wan,
Hager Brant W.,
Brock Malcolm V.,
DienerWest Marie
Publication year - 2004
Publication title -
journal of general internal medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.746
H-Index - 180
eISSN - 1525-1497
pISSN - 0884-8734
DOI - 10.1111/j.1525-1497.2004.30091.x
Subject(s) - medicine , contingency table , diagnostic accuracy , test (biology) , statistics , criterion validity , sensitivity (control systems) , psychometrics , clinical psychology , construct validity , mathematics , paleontology , biology , radiology , electronic engineering , engineering
OBJECTIVE: Evaluations of screening or diagnostic tests sometimes incorporate measures of overall accuracy , diagnostic accuracy , or test efficiency . These terms refer to a single summary measurement calculated from 2 × 2 contingency tables that is the overall probability that a patient will be correctly classified by a screening or diagnostic test. We assessed the value of overall accuracy in studies of test validity, a topic that has not received adequate emphasis in the clinical literature. DESIGN: Guided by previous reports, we summarize the issues concerning the use of overall accuracy. To document its use in contemporary studies, a search was performed for test evaluation studies published in the clinical literature from 2000 to 2002 in which overall accuracy derived from a 2 × 2 contingency table was reported. MEASUREMENTS AND MAIN RESULTS: Overall accuracy is the weighted average of a test's sensitivity and specificity, where sensitivity is weighted by prevalence and specificity is weighted by the complement of prevalence. Overall accuracy becomes particularly problematic as a measure of validity as 1) the difference between sensitivity and specificity increases and/or 2) the prevalence deviates away from 50%. Both situations lead to an increasing deviation between overall accuracy and either sensitivity or specificity. A summary of results from published studies ( N = 25) illustrated that the prevalence‐dependent nature of overall accuracy has potentially negative consequences that can lead to a distorted impression of the validity of a screening or diagnostic test. CONCLUSIONS: Despite the intuitive appeal of overall accuracy as a single measure of test validity, its dependence on prevalence renders it inferior to the careful and balanced consideration of sensitivity and specificity.