Premium
Raking Kappa: Describing Potential Impact of Marginal Distributions on Measures of Agreement
Author(s) -
Agresti Alan,
Ghosh Atalanta,
Bini Matilde
Publication year - 1995
Publication title -
biometrical journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.108
H-Index - 63
eISSN - 1521-4036
pISSN - 0323-3847
DOI - 10.1002/bimj.4710370705
Subject(s) - contingency table , kappa , statistics , mathematics , marginal distribution , econometrics , cohen's kappa , marginal model , random variable , regression analysis , geometry
Several authors have noted the dependence of kappa measures of inter‐rater agreement on the marginal distributions of contingency tables displaying the joint ratings. This paper introduces a smoothed version of kappa computed after raking the table to achieve pre‐specified marginal distributions. A comparison of kappa with raked kappa for various margins can indicate the extent of the dependence on the margins, and can indicate how much of the lack of agreement is due to marginal heterogeneity.