Premium
A family of measures to evaluate scale reliability in a longitudinal setting
Author(s) -
Laenen Annouschka,
Alonso Ariel,
Molenberghs Geert,
Vangeneugden Tony
Publication year - 2009
Publication title -
journal of the royal statistical society: series a (statistics in society)
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.103
H-Index - 84
eISSN - 1467-985X
pISSN - 0964-1998
DOI - 10.1111/j.1467-985x.2008.00554.x
Subject(s) - generalizability theory , reliability (semiconductor) , scale (ratio) , intraclass correlation , replicate , classical test theory , psychology , population , reliability theory , computer science , statistics , reliability engineering , econometrics , psychometrics , item response theory , mathematics , developmental psychology , engineering , power (physics) , physics , demography , quantum mechanics , sociology
Summary. The concept of reliability denotes one of the most important psychometric properties of a measurement scale. Reliability refers to the capacity of the scale to discriminate between subjects in a given population. In classical test theory, it is often estimated by using the intraclass correlation coefficient based on two replicate measurements. However, the modelling framework that is used in this theory is often too narrow when applied in practical situations. Generalizability theory has extended reliability theory to a much broader framework but is confronted with some limitations when applied in a longitudinal setting. We explore how the definition of reliability can be generalized to a setting where subjects are measured repeatedly over time. On the basis of four defining properties for the concept of reliability, we propose a family of reliability measures which circumscribes the area in which reliability measures should be sought. It is shown how different members assess different aspects of the problem and that the reliability of the instrument can depend on the way that it is used. The methodology is motivated by and illustrated on data from a clinical study on schizophrenia. On the basis of this study, we estimate and compare the reliabilities of two different rating scales to evaluate the severity of the disorder.