Stable unstable reliability theory

Classical reliability theory assumes that individuals have identical true scores on both testing occasions, a condition described as stable. If some individuals’ true scores are different on different testing occasions, described as unstable, the estimated reliability can be misleading. A model called stable unstable reliability theory (SURT) frames stability or instability as an empirically testable question. SURT assumes a mixed population of stable and unstable individuals in unknown proportions, with w i the probability that individual i is stable. w i becomes i ’s test score weight which is used to form a weighted correlation coefficient r w which is reliability under SURT. If all w i = 1 then r w is the classical reliability coefficient; thus classical theory is a special case of SURT. Typically r w is larger than the conventional reliability r , and confidence intervals on true scores are typically shorter than conventional intervals. r w is computed with routines in a publicly available R package.