Premium
Exact approaches for testing hypotheses based on the intra‐class kappa coefficient
Author(s) -
Wilding Gregory E.,
Consiglio Joseph D.,
Shan Guogen
Publication year - 2014
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.6135
Subject(s) - null distribution , mathematics , nuisance parameter , categorical variable , null hypothesis , test statistic , statistical hypothesis testing , sample size determination , statistics , type i and type ii errors , asymptotic distribution , statistic , parameter space , sampling distribution , sample (material) , chemistry , chromatography , estimator
Testing involving the intra‐class kappa coefficient is commonly performed in order to assess agreement involving categorical ratings. A number of procedures have been proposed, which make use of the limiting null distribution as the sample size goes to infinity in order to compute the observed significance. As with many tests based on asymptotic null distributions, these tests are associated with problematic type I error control for selected sample sizes and points in the parameter space. We propose and study a collection of exact testing approaches for both the one‐sample and K ‐sample scenarios. For the one‐sample case, p ‐values are obtained using the exact distribution of the test statistic conditional on a sufficient statistic. In addition, unconditional approaches are considered on the basis of maximization across the nuisance parameter space. Numerical evaluation reveals advantages with the exact unconditional procedures. Copyright © 2014 John Wiley & Sons, Ltd.