Premium
Rate it again: Using the wisdom of many to improve performance evaluations
Author(s) -
Barneron Meir,
Allalouf Avi,
Yaniv Ilan
Publication year - 2019
Publication title -
journal of behavioral decision making
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.136
H-Index - 76
eISSN - 1099-0771
pISSN - 0894-3257
DOI - 10.1002/bdm.2127
Subject(s) - psychology , field (mathematics) , social psychology , class (philosophy) , econometrics , applied psychology , computer science , artificial intelligence , economics , mathematics , pure mathematics
Much research shows that judgmental estimation could be improved by combining estimates from independent judges as well as within judges. These results have been obtained mostly with judgments about matters of fact, that is, for which there are objective truth criteria. In the present research, we extend these findings to performance evaluations. In a controlled field study, expert judges provided evaluations of a large number of essays written by college applicants taking college entrance tests. The judges were asked to evaluate each essay twice—on two occasions, a week apart. This design allowed us to assess the benefits of two methods of combining evaluations: within rater and across raters. Accuracy gains were obtained with both methods. Although the within‐rater combinations yielded fewer gains than the across‐rater ones, they were still appreciable in comparison with the across rater ones. Our findings extend the class of judgments to which the “wisdom of many” could be applied. These findings are potentially applicable to performance evaluations in social, educational, and employment settings.