Premium
Assessing inter‐rater reliability when the raters are fixed: Two concepts and two estimates
Author(s) -
Rousson Valentin
Publication year - 2011
Publication title -
biometrical journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.108
H-Index - 63
eISSN - 1521-4036
pISSN - 0323-3847
DOI - 10.1002/bimj.201000066
Subject(s) - intraclass correlation , reliability (semiconductor) , inter rater reliability , statistics , context (archaeology) , sample (material) , population , psychology , sample size determination , econometrics , mathematics , statistical model , psychometrics , demography , geography , sociology , rating scale , power (physics) , physics , chemistry , archaeology , chromatography , quantum mechanics
Intraclass correlation (ICC) is an established tool to assess inter‐rater reliability. In a seminal paper published in 1979, Shrout and Fleiss considered three statistical models for inter‐rater reliability data with a balanced design. In their first two models, an infinite population of raters was considered, whereas in their third model, the raters in the sample were considered to be the whole population of raters. In the present paper, we show that the two distinct estimates of ICC developed for the first two models can both be applied to the third model and we discuss their different interpretations in this context.