Premium
Sensitivity of Fisher's exact test to minor perturbations in 2 × 2 contingency tables
Author(s) -
Dupont William D.
Publication year - 1986
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.4780050610
Subject(s) - contingency table , exact test , statistics , mathematics , sample size determination , statistical significance , p value , drop (telecommunication) , econometrics , statistical hypothesis testing , computer science , telecommunications
The two tailed Fisher's exact P value is extremely sensitive to small perturbations in 2 × 2 contingency tables. An example indicates that a 1 per cent increase in the denominator of one treatment group results in a 32 per cent drop in the exact P value, but a mere 0.1 per cent decrease in the treatment success rate. This is equivalent to the increase in significance obtained by a 20 per cent increase in the sample size of both treatments without changing the observed success rates. This drop results from small changes in the probabilities of unobserved events. A systematic evaluation of 920 pairs of similar contingency tables shows that these fluctuations occur frequently over a wide range of sample sizes and significance levels. Doubling the one tailed exact P value provides a more consistent measure of inferential strength. We discuss various chi‐squared continuity corrections.