Measuring Agreement, More Complicated Than It Seems

In medicine, before replacing an old device by a new one, we need to know whether the results of the old and new device are similar. This is called determining the agreement between methods. In this paper, we will first discuss various ways to determine the agreement between methods to measure continuous variables, including the t test, the correlation coefficient and the Bland-Altman plot. In the second part, we will discuss methods to determine the agreement between categorical variables, like the χ(2) test and Cohen's ĸ. The latter are often used when studying the agreement between clinicians, definitions, formulas or different data sources.