Premium
Averaging probability judgments: Monte Carlo analyses of asymptotic diagnostic value
Author(s) -
Johnson Timothy R.,
Budescu David V.,
Wallsten Thomas S.
Publication year - 2001
Publication title -
journal of behavioral decision making
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.136
H-Index - 76
eISSN - 1099-0771
pISSN - 0894-3257
DOI - 10.1002/bdm.369
Subject(s) - mathematics , pairwise comparison , pooling , independence (probability theory) , conditional independence , conditional probability , covert , event (particle physics) , statistics , econometrics , computer science , artificial intelligence , linguistics , philosophy , physics , quantum mechanics
Wallsten et al. (1997) developed a general framework for assessing the quality of aggregated probability judgments. Within this framework they presented a theorem regarding the effects of pooling multiple probability judgments regarding unique binary events. The theorem states that under reasonable conditions, and assuming conditional pairwise independence of the judgments, the average probability estimate is asymptotically perfectly diagnostic of the true event state as the number of estimates pooled goes to infinity. The purpose of the present study was to examine by simulation (1) the rate of convergence of averaged judgments to perfect diagnostic value under various conditions and (2) the robustness of the theorem to violations of its assumption that the covert probability judgments are conditionally pairwise independent. The results suggest that while the rate of convergence is sensitive to violations of the conditional pairwise independence, the asymptotic properties remain relatively robust under a large variety of conditions. The practical implications of these results are discussed. Copyright © 2001 John Wiley & Sons, Ltd.