Premium
Senior‐executive performance: Interrater reliability and rater effects in multi‐source ratings *
Author(s) -
Hutchison Ann,
Burch Giles St J
Publication year - 2011
Publication title -
asia pacific journal of human resources
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.825
H-Index - 33
eISSN - 1744-7941
pISSN - 1038-4111
DOI - 10.1177/1038411111423187
Subject(s) - inter rater reliability , psychology , trait , reliability (semiconductor) , social psychology , affect (linguistics) , developmental psychology , rating scale , computer science , power (physics) , physics , communication , quantum mechanics , programming language
This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We present a study of 189 senior executives in New Zealand and Australia, whose performance was rated by an average of 4.23 raters: superiors, peers, and subordinates. Intra‐class correlation coefficients revealed strong rater effects, and a multi‐trait multi‐method analysis showed that those effects came from individual raters, rather than rater source (i.e. superior, peer, or subordinate). The findings suggest that it may be unwise to aggregate performance ratings at the senior‐executive level, and to use such ratings, whether aggregated or single, to make critical decisions.