Premium
Methods correcting for multiple testing: operating characteristics
Author(s) -
Brown Barry W.,
Russell Kathy
Publication year - 1997
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/(sici)1097-0258(19971130)16:22<2511::aid-sim693>3.0.co;2-4
Subject(s) - statistics , null hypothesis , false discovery rate , multiple comparisons problem , statistical hypothesis testing , computer science , p value , type i and type ii errors , mathematics , biology , biochemistry , gene
We examine the operating characteristics of 17 methods for correcting p ‐values for multiple testing on synthetic data with known statistical properties. These methods are derived p ‐values only and not the raw data. With the test cases, we systematically varied the number of p ‐values, the proportion of false null hypotheses, the probability that a false null hypothesis would result in a p ‐value less than 5 per cent and the degree of correlation between p ‐values. We examined the effect of each of these factors on familywise and false negative error rates and compared the false negative error rates of methods with an acceptable familywise error. Only four methods were not bettered in this comparison. Unfortunately, however, a uniformly best method of those examined does not exist. A suggested strategy for examining corrections uses a succession of methods that are increasingly lax in familywise error. A computer program for these corrections is available. © 1997 by John Wiley & Sons, Ltd.