Premium
Research synthesis of information theory measures of uncertainty: Meta‐analysis of entropy and mutual information of diagnostic tests
Author(s) -
Tsalatsanis Athanasios,
Hozo Iztok,
Djulbegovic Benjamin
Publication year - 2021
Publication title -
journal of evaluation in clinical practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.737
H-Index - 73
eISSN - 1365-2753
pISSN - 1356-1294
DOI - 10.1111/jep.13475
Subject(s) - meta analysis , diagnostic test , metric (unit) , entropy (arrow of time) , mutual information , statistical hypothesis testing , data mining , medical physics , diagnostic accuracy , computer science , medicine , statistics , artificial intelligence , radiology , mathematics , emergency medicine , operations management , physics , quantum mechanics , economics
Rationale, aims, and objectives Assessing the performance of diagnostic tests requires evaluation of the amount of diagnostic uncertainty a test reduces. Statistical measures, such as sensitivity and specificity, currently dominating the evidence‐based medicine (EBM) and related fields, cannot explicitly measure this reduction in diagnostic uncertainty. Mutual information (MI), an information theory statistic, explicitly quantifies diagnostic uncertainty by measuring information gain before vs after diagnostic testing. In this paper, we propose the use of MI as a single measure to express diagnostic test performance and demonstrate how it can be used in the meta‐analysis of diagnostic test studies. Methods We use two case studies from the literature to demonstrate the applicability of MI meta‐analysis in assessing diagnostic performance. Meta‐analysis of studies evaluating (a) ultrasonography (US) to detect endometrial cancer and (b) magnetic resonance angiography to detect arterial stenosis. Results The results of MI meta‐analyses are comparable to those of traditional statistical measures' meta‐analyses. However, the results of MI are easier to understand as it relates directly to the extent of uncertainty a diagnostic test can reduce. For example, the US test, diagnosing endometrial cancer, is 40% specific and 94% sensitive. The combination of these values is difficult to interpret and may lead to inappropriate assessment (eg, one could favour the test due to its high sensitivity, ignoring its low specificity). In terms of MI, however, a single metric shows that the test reduces diagnostic uncertainty by 10%, which many users may consider small under most circumstances. Conclusions We have demonstrated the suitability of MI in assessing the performance of diagnostic tests, which can facilitate easier interpretation of the true utility of diagnostic tests. Similarly, to the guidance for interpretation of effect size of treatment interventions, we also propose the guidelines for interpretation of the utility of diagnostic tests based on the magnitude of reduction in diagnostic uncertainty.