Premium
Elicitation of Quantitative Data from a Heterogeneous Expert Panel: Formal Process and Application in Animal Health
Author(s) -
Van Der FelsKlerx Ine H. J.,
Goossens Louis H. J.,
Saatkamp Helmut W.,
Horst Suzan H. S.
Publication year - 2002
Publication title -
risk analysis
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.972
H-Index - 130
eISSN - 1539-6924
pISSN - 0272-4332
DOI - 10.1111/0272-4332.t01-1-00007
Subject(s) - expert elicitation , computer science , protocol (science) , process (computing) , weighting , normative , task (project management) , construct (python library) , delphi method , management science , data mining , risk analysis (engineering) , artificial intelligence , statistics , mathematics , engineering , medicine , philosophy , alternative medicine , systems engineering , epistemology , pathology , radiology , programming language , operating system
This paper presents a protocol for a formal expert judgment process using a heterogeneous expert panel aimed at the quantification of continuous variables. The emphasis is on the process's requirements related to the nature of expertise within the panel, in particular the heterogeneity of both substantive and normative expertise. The process provides the opportunity for interaction among the experts so that they fully understand and agree upon the problem at hand, including qualitative aspects relevant to the variables of interest, prior to the actual quantification task. Individual experts' assessments on the variables of interest, cast in the form of subjective probability density functions, are elicited with a minimal demand for normative expertise. The individual experts' assessments are aggregated into a single probability density function per variable, thereby weighting the experts according to their expertise. Elicitation techniques proposed include the Delphi technique for the qualitative assessment task and the ELI method for the actual quantitative assessment task. Appropriately, the Classical model was used to weight the experts' assessments in order to construct a single distribution per variable. Applying this model, the experts' quality typically was based on their performance on seed variables. An application of the proposed protocol in the broad and multidisciplinary field of animal health is presented. Results of this expert judgment process showed that the proposed protocol in combination with the proposed elicitation and analysis techniques resulted in valid data on the (continuous) variables of interest. In conclusion, the proposed protocol for a formal expert judgment process aimed at the elicitation of quantitative data from a heterogeneous expert panel provided satisfactory results. Hence, this protocol might be useful for expert judgment studies in other broad and/or multidisciplinary fields of interest.