Premium
A Coefficient of Agreement Adjusted for Bias in Paired Ordered Categorical Data
Author(s) -
Svensson Elisabeth,
Svensson Elisabeth
Publication year - 1997
Publication title -
biometrical journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.108
H-Index - 63
eISSN - 1521-4036
pISSN - 0323-3847
DOI - 10.1002/bimj.4710390602
Subject(s) - categorical variable , statistics , mathematics , rank correlation , spearman's rank correlation coefficient , correlation , ranking (information retrieval) , econometrics , artificial intelligence , computer science , geometry
The quality of ordered categorical recordings is determined from repeated measurements on the same subject in order to assess the level of agreement between raters, scales or occasions. The presented rating‐invariant method for ordered categorical data provides means of analysing the quality of single‐item rating scales, irrespective of the number of possible response values and the marginal distributions. Marginal heterogeneity implies systematic disagreement, so‐called bias. An augmented ranking approach is the basis for the separation of inter‐rater disagreement into systematic and random components. Correlation between pairs of augmented rank values provides a measure of agreement to the best common ordering of paired classifications, given inter‐rater bias. The essential differences in interpretation and applicability of the proposed coefficient of agreement and the Spearman rank‐order correlation for ordered categorical data are discussed.