Premium
A FIELD STUDY OF THE EFFECTS OF RATING PURPOSE ON THE QUALITY OF MULTISOURCE RATINGS
Author(s) -
GREGURAS GARY J.,
ROBIE CHET,
SCHLEICHER DEIDRA J.,
III MAYNARD GOFF
Publication year - 2003
Publication title -
personnel psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.076
H-Index - 142
eISSN - 1744-6570
pISSN - 0031-5826
DOI - 10.1111/j.1744-6570.2003.tb00141.x
Subject(s) - psychology , generalizability theory , variance (accounting) , dependability , reliability (semiconductor) , quality (philosophy) , sample (material) , applied psychology , field (mathematics) , social psychology , statistics , developmental psychology , reliability engineering , mathematics , power (physics) , philosophy , chemistry , accounting , epistemology , chromatography , quantum mechanics , business , engineering , pure mathematics , physics
Using a field sample of peers and subordinates, the current study employed generalizability theory to estimate sources of systematic variability associated with both developmental and administrative ratings (variance due to items, raters, etc.) and then used these values to estimate the dependability (i.e., reliability) of the performance ratings under various conditions. Results indicated that the combined rater and rater‐by‐ratee interaction effect and the residual effect were substantially larger than the person effect (i.e., object of measurement) for both rater sources across both purpose conditions. For subordinates, the person effect accounted for a significantly greater percentage of total variance in developmental ratings than in administrative ratings; however, no differences were observed for peer ratings as a function of rating purpose. These results suggest that subordinate ratings are of significantly better quality when made for developmental than for administrative purposes, but the same is not true for peer ratings.