Premium
Assessing the merits and faults of holistic and disaggregated judgments
Author(s) -
Arkes Hal R.,
GonzálezVallejo Claudia,
Bonham Aaron J.,
Kung YiHan,
Bailey Nathan
Publication year - 2010
Publication title -
journal of behavioral decision making
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.136
H-Index - 76
eISSN - 1099-0771
pISSN - 0894-3257
DOI - 10.1002/bdm.655
Subject(s) - variance (accounting) , set (abstract data type) , preference , psychology , paired comparison , social psychology , cognitive psychology , task (project management) , statistics , econometrics , computer science , mathematics , economics , accounting , management , programming language
Three studies explored both the advantages of and subjects' preferences for a disaggregated judgment procedure and a holistic one. The task in our first two studies consisted of evaluating colleges; the third study asked participants to evaluate job applicants. Holistic ratings consisted of providing an overall evaluation while considering all of the characteristics of the evaluation objects; disaggregated ratings consisted of evaluating each cue independently. Participants also made paired comparisons of the evaluation objects. We constructed preference orders for the disaggregated method by aggregating these ratings (unweighted or weighted characteristics). To compare the holistic, disaggregated, and weighted‐disaggregated method we regressed the four cues on the participant's holistic rating, on the linearly aggregated disaggregated ratings, and on the average weighted disaggregated rating, using the participant's “importance points” for each cue as weights. Both types of combined disaggregated ratings related more closely to the cues in terms of proportion of variance accounted for in Experiments 1 and 2. In addition, the disaggregated ratings were more closely related to the paired‐comparison orderings, but Experiment 2 showed that this was true for a small set (10) but not a large set (60) of evaluation objects. Experiment 3 tested the “gamesmanship” hypothesis: People prefer holistic ratings because it is easier to incorporate illegitimate but appealing criteria into one's judgment. The results suggested that the disaggregated procedure generally produced sharper distinctions between the most relevant and least relevant cues. Participants in all three of these studies preferred the holistic ratings despite their statistical inferiority. Copyright © 2009 John Wiley & Sons, Ltd.