Premium
THE “ CAUTION‐LEVEL ” IN MULTIPLE TESTS OF SIGNIFICANCE
Author(s) -
Verhagen A. M. W.
Publication year - 1963
Publication title -
australian journal of statistics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.434
H-Index - 41
eISSN - 1467-842X
pISSN - 0004-9581
DOI - 10.1111/j.1467-842x.1963.tb00139.x
Subject(s) - statistics , mathematics , statistical significance , test (biology) , p value , statistical hypothesis testing , null hypothesis , assertion , type i and type ii errors , reliability (semiconductor) , range (aeronautics) , econometrics , computer science , engineering , power (physics) , paleontology , physics , quantum mechanics , biology , programming language , aerospace engineering
Summary The paper produces a measure for the reliability of multiple assertions arrived at with multile tests of significance. it is called the caution level and in the case of single tests it reduces to the well know significance level. The caution level is introduced and discussed in relation to n independent test statistics for n different null hypotheses. Its value c is given by c = l ‐(l ‐α) n‐(r‐1) where α is the level of significance of the individual tests and r is the observed number of significant test statistics. The caution level is applied to some experimental data and worked out in detail for the case n=2. The caution level depends on the observed number of significant test‐statistics and is therefore closely geared to the actual outcome of the experiment. This gives it an advantage over the “experimentwise error rate” proposed by Tukey (1953), since the latter becomes more unrealistic as the number r of significant test‐statistics increases. The caution level turns out to be well suited for designing multiple tests in which the caution level is the same irrespective of the possible multiple assertion to which the test may lead. The construction of tests with the caution level equalized over all possible assertions involves only the distribution of the largest of ν independent rectangular variates in the range (0, 1). Such a test is illustrated graphically for thc case n=2. Just as the significance level in the case of a single test, so the caution level provides a, starting point for the study of errors of the second kind, both in the absence and in the presence of prior distribuions, for the more general multiple‐test situation and thus opens up a wide field of investigation. At a later stage the author proposes to extend the use of the caution level to more complicated situations involving studentized test‐statistics dependent through a common denominator and to correlated test statistics consisting of differences between means.