Open AccessExamining Rater Biases of Peer Assessors in Different Assessment Environments
Author(s) -
Sabahattin Yeşilçınar,
Mehmet Şata
Publication year - 2021
Publication title -
international journal of psychology and educational studies
Language(s) - English
Resource type - Journals
ISSN - 2148-9378
DOI - 10.52380/ijpes.2021.8.4.622
Subject(s) - rubric , psychology , rasch model , peer assessment , inter rater reliability , turkish , peer feedback , presentation (obstetrics) , applied psychology , social psychology , mathematics education , developmental psychology , rating scale , medicine , linguistics , philosophy , radiology
The current study employed many-facet Rasch measurement (MFRM) to explain the rater bias patterns of EFL student teachers (hereafter students) when they rate the teaching performance of their peers in three assessment environments: online, face-to-face, and anonymous. Twenty-four students and two instructors rated 72 micro-teachings performed by senior Turkish students. The performance was assessed using a five-category analytic rubric developed by the researchers (Lesson Presentation, Classroom Management, Communication, Material, and Instructional Feedback). MFRM revealed the severity and leniency biases in all three assessment environments at the group and individual levels, drawing attention to the less occurrence of biases anonymous assessment. The central tendency and halo effects were observed only at the individual level in all three assessment environments, and these errors were similar to each other. Semi-structured interviews with peer raters (n = 24) documented their perspectives about how the anonymous assessment affected the severity, leniency, central tendency, and halo effects. Besides, the findings displayed that hiding the identity of the peers develops the reliability and validity of the measurements performed during peer assessment.