Premium
Inference Procedures for Assessing Interobserver Agreement among Multiple Raters
Author(s) -
Altaye Mekibib,
Dormer Allan,
Klar Neil
Publication year - 2001
Publication title -
biometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.298
H-Index - 130
eISSN - 1541-0420
pISSN - 0006-341X
DOI - 10.1111/j.0006-341x.2001.00584.x
Subject(s) - statistics , goodness of fit , mathematics , nominal level , confidence interval , binomial distribution , range (aeronautics) , inference , sample size determination , binomial (polynomial) , multiple comparisons problem , statistical inference , computer science , artificial intelligence , materials science , composite material
Summary. We propose a new procedure for constructing inferences about a measure of interobserver agreement in studies involving a binary outcome and multiple raters. The proposed procedure, based on a chi‐square goodness‐of‐fit test as applied to the correlated binomial model (Bahadur, 1961, in Studies in Item Analysis and Prediction , 158–176), is an extension of the goodness‐of‐fit procedure developed by Donner and Eliasziw (1992, Statistics in Medicine 11 , 1511–1519) for the case of two raters. The new procedure is shown to provide confidence‐interval coverage levels that are close to nominal over a wide range of parameter combinations. The procedure also provides a sample‐size formula that may be used to determine the required number of subjects and raters for such studies.