Factors That Influence Reliability of the Mouse Clinical Frailty Index

Dear Editor, We thank Kane and colleagues for their careful attention to our report recently published in the Journal of Gerontology Biological Sciences (1), and for the opportunity to provide some additional description of our study. In their letter, they noted that, without a special procedure for assessing reliability, and without feedback of the results, they saw no trend toward increasing inter-rater reliability in their series, which included both the long-lived C57BL/6JJ mice and the short-lived DBA/2J mouse model. By contrast, both their recently published report (2) and our original study (1), which each included attempts to improve reliability, yielded better results. Kane and colleagues also call to attention that the professional background of the rater may be important. In their data, scientists and trainees showed greater inter-rater reliability than did animal technicians. Our data appear to support the former conclusion; without data from technicians we must be silent in the latter. The two who did the ratings in our original submission (H.F. and M.S.) were both graduate students at the time, with 4–5 years of experience in the research laboratory. Importantly, they achieved a high level of inter-rater reliability (intra-class correlation coefficient = 0.77; 95% confidence interval = 0.67–0.83) by the final set of ratings. The work of Kane and colleagues also strongly suggests that it is the professional background of the rater rather than the number of years of experience that are important. We cannot comment on this based on our study (1), as both raters in our study had similar backgrounds and years of experience. Kane and colleagues are right to draw to attention to the fact that reliability of laboratory protocols should not be presumed. Instead, reliability merits both careful selection of personnel and use of specific procedures for its verification.