Premium
MULTIPLE COMPARISON PROCEDURES UPDATED
Author(s) -
Ludbrook John
Publication year - 1998
Publication title -
clinical and experimental pharmacology and physiology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.752
H-Index - 103
eISSN - 1440-1681
pISSN - 0305-1870
DOI - 10.1111/j.1440-1681.1998.tb02179.x
Subject(s) - bonferroni correction , pairwise comparison , type i and type ii errors , multiple comparisons problem , statistics , null hypothesis , class (philosophy) , mathematics , statistical hypothesis testing , false discovery rate , inference , statistical inference , sort , computer science , artificial intelligence , biology , arithmetic , genetics , gene
SUMMARY 1. A common statistical flaw in articles submitted to or published in biomedical research journals is to test multiple null hypotheses that originate from the results of a single experiment without correcting for the inflated risk of type 1 error (false positive statistical inference) that results from this. Multiple comparison procedures (MCP) are designed to minimize this risk. The present review focuses on pairwise contrasts, the most common sort of multiple comparisons made by biomedical investigators. 2. In an earlier review a variety of MCP were described and evaluated. It was concluded that an effective MCP should control the risk of family‐wise type 1 error, so as to ensure that not more than one hypothesis within a single family is falsely rejected. One‐step procedures based on the Bonferroni or Sidak inequalities do this. For continuous data and under normal distribution theory, so does the Tukey‐Kramer procedure for all possible pairwise contrasts of means and the Dunnett procedure for all possible pairwise contrasts of means with a control mean. 3. There is now a new class of MCP, based on the Bonferroni or Šidák inequalities but performed in a step‐wise fashion. The members of this class have certain desirable properties. They: (i) control the family‐wise type 1 error rate as effectively as the one‐step procedures; (ii) are more powerful than the one‐step Bonferroni or Sidak procedures, especially when hypotheses are logically related; and (iii) can be applied not only to continuous data but also to ordinal or categorical data. 4. Of the new step‐wise MCP, Holm's step‐down procedures are commended for their combination of accuracy, power and versatility. They also have the virtue of simplicity. Given the raw P values that result from conventional tests of significance, the adjustments for multiple comparisons can be made by hand or hand‐held calculator. 5. Despite the corrective abilities of the new step‐wise MCP, investigators should try to design their experiments and analyses to test a single, global hypothesis rather than multiple ones.