Premium
A comparison of one‐sided methods to identify significant individual outcomes in a multiple outcome setting: stepwise tests or global tests with closed testing
Author(s) -
Troendle James F.,
Legler Julie M.
Publication year - 1998
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/(sici)1097-0258(19980615)17:11<1245::aid-sim833>3.0.co;2-z
Subject(s) - univariate , type i and type ii errors , statistical hypothesis testing , statistics , null hypothesis , multiple comparisons problem , mathematics , outcome (game theory) , ordinary least squares , econometrics , multivariate statistics , mathematical economics
We compare two approaches to the identification of individual significant outcomes when a comparison of two groups involves multiple outcome variables. The approaches are all designed to control the familywise error rate (FWE) with any subset of the null hypotheses being true (in the strong sense). The first approach is initially to use a global test of the overall hypothesis that the groups are equivalent for all variables, followed by an application of the closed testing algorithm of Marcus, Peritz and Gabriel. The global tests considered here are ordinary least squares (OLS), generalized least squares (GLS), an approximation to a likelihood ratio test (ALR), and a new test based on an approximation to the most powerful similar test for simple alternatives. The second approach is that of stepwise testing, which tests the univariate hypotheses in a specific order with appropriate adjustment to the univariate p ‐values for multiplicity. The stepwise tests considered include both step‐down and step‐up tests of a general type, and likewise permutation tests that incorporate the dependence structure of the data. We illustrate the tests with two examples of birth outcomes: a comparison of cocaine‐exposed new‐borns to control new‐borns on neurobehavioural and physical growth variables, and, in a separate study, a comparison of babies born to diabetic mothers and babies born to non‐diabetic mothers on minor malformations. After describing the methods and analysing the birth outcome data, we use simulations on Gaussian data to provide guidelines for the use of these procedures in terms of power and computation. © 1998 John Wiley & Sons, Ltd.