Premium
Eliciting reliable uncertainty estimates
Author(s) -
Tonn Bruce E.,
Goeltz Richard,
Travis Cheryl
Publication year - 1992
Publication title -
expert systems
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.365
H-Index - 38
eISSN - 1468-0394
pISSN - 0266-4720
DOI - 10.1111/j.1468-0394.1992.tb00382.x
Subject(s) - modalities , computer science , reliability (semiconductor) , range (aeronautics) , social science , power (physics) , physics , materials science , quantum mechanics , sociology , composite material
Knowledge engineers often find that experts’ estimates of uncertainty change from one day to the next and therefore often seem unreliable. This article explores the unreliability of probability estimates. Forty undergraduates answered 31 questions concerning the probability of daily events at two separate times. Three answer modalities were available: probability, chances and percent of the time. This paper finds that reliability decreases when answer modalities change from one time to the next. It is hypothesized that problem‐solving approaches changed along with the modalities. The subjects represented a wide range in reliability, with the most reliable subjects changing answer modalities less than the unreliable subjects. These results suggest that knowledge engineers should endeavor to have experts express their uncertainty estimates using consistent modalities.