Premium
Inter‐rater reliability of three adult handwriting legibility instruments
Author(s) -
Au Eunice H.,
McCluskey Annie,
Lannin Natasha A.
Publication year - 2012
Publication title -
australian occupational therapy journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.595
H-Index - 44
eISSN - 1440-1630
pISSN - 0045-0766
DOI - 10.1111/j.1440-1630.2012.01035.x
Subject(s) - concordance , handwriting , cronbach's alpha , legibility , inter rater reliability , rasch model , kappa , reliability (semiconductor) , psychology , rating scale , scale (ratio) , physical medicine and rehabilitation , physical therapy , medicine , psychometrics , clinical psychology , computer science , developmental psychology , mathematics , artificial intelligence , art , power (physics) , geometry , physics , quantum mechanics , visual arts
Background/aim Legibility is important for functional, handwritten communication. Deficits in legibility can impair occupational performance following stroke or trauma. Few instruments are available to assess adult handwriting legibility during rehabilitation. The aim of this study was to compare inter‐rater reliability of a new four‐point legibility rating instrument with two existing instruments, and describe scale structure and item difficulty of each instrument. Methods Three trained raters scored 60 handwriting samples using: (i) a Four‐Point Scale ( FPS ); (ii) the modified Evaluation Tool of Children's Handwriting ( mETCH ); and (iii) the new modified FPS . Rater concordance and exact agreement were investigated using the intra‐class correlation coefficient ( ICC 3,1 ), multi‐rater kappa (κ) and Krippendorff's alpha (α). Cronbach's alpha was calculated to examine internal consistency reliability, and Rasch modelling was used to examine scale structure and item difficulty. Results Rater concordance for the FPS was fair (ICC 3,1 = 0.37); exact agreement was poor (κ = 0.19; α = 0.19). Rater concordance for the two mETCH subtests was fair to moderate (ICC 3,1 = 0.39–0.50), but with no exact agreement (κ < 0.00). Rater concordance for three subtests of the modified FPS was slight to moderate ( ICC 3,1 = 0.16–0.51); exact agreement ranged from nil to fair (κ = −0.06–0.30; α = −0.05–0.30). Rasch modelling confirmed internal consistency of instruments, but was low between‐rater consistency (rater severity variability). Conclusions A reliable instrument for measuring change over time remains elusive. In the meantime, these instruments can be used by individual clinicians to diagnose and rate legibility.