Premium
Examining the inter‐rater and test–retest reliability of the Student Practice Evaluation Form‐Revised ( SPEF ‐R) for occupational therapy students
Author(s) -
Rodger Sylvia,
Coleman Allison,
Caine AnneMaree,
Chien ChiWen,
Copley Jodie,
Turpin Merrill,
Brown Ted
Publication year - 2014
Publication title -
australian occupational therapy journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.595
H-Index - 44
eISSN - 1440-1630
pISSN - 0045-0766
DOI - 10.1111/1440-1630.12151
Subject(s) - rasch model , rating scale , psychology , reliability (semiconductor) , occupational therapy , test (biology) , multidisciplinary approach , inter rater reliability , medical education , likert scale , vignette , medicine , clinical psychology , social psychology , paleontology , developmental psychology , power (physics) , social science , physics , quantum mechanics , sociology , biology , psychiatry
Background/aim The Student Practice Evaluation Form‐Revised Edition ( SPEF ‐R) is widely used in Australian universities to evaluate occupational therapy students’ performance in practice education. Reliable completion of the SPEF ‐R by practice educators is critical for students and universities from a quality assurance perspective. This study used standardised video vignettes to examine the inter‐rater and test–retest reliability of practice educators when completing the SPEF ‐R. Methods Nine vignettes were developed with three levels of student performance (below expectations, adequate‐to‐proficient, and outstanding) for each of three scenarios which depicted a multidisciplinary team meeting, a supervision session, and a home visit respectively. Seventy‐five occupational therapy practice educators viewed the vignettes and completed an online survey to rate student performance in each vignette using five selected SPEF ‐R items and using the five‐point rating scale. Twenty of these practice educators completed the rating process twice for test–retest reliability purposes. Percentage agreement, t‐tests and Rasch Measurement Model were used for analysis. Results Practice educators exhibited consistent ratings on most of the SPEF ‐R items, except for rating adequate‐to‐proficient student performance on three items for inter‐rater reliability and eight items for test–retest reliability. Rasch analysis found that the majority of practice educators (96–98.7%) exhibited consistency in their use of the SPEF ‐R rating scales. Practice educators also demonstrated satisfactory test–retest agreement of severity/leniency in rating student performance in the multidisciplinary team meeting scenario. Conclusion The study findings suggest that the SPEF ‐R could be used reliably and interpreted consistently by practice educators with diverse backgrounds and levels of experience.