Applying Subject Matter Expertise (SME) Elicitation Techniques to TRAC Studies

: We are interested in quantifying measures which are qualitative and reside as knowledge held by a Subject Matter Expert (SME). This knowledge must be elicited from the SME and quantified in a logical manner to be used in analysis. Difficulties arise when trying to convey results in a mathematically rigorous way. To add to the complexity, we often combine the qualitative responses through estimating or averaging a single value that encompasses all information. We examine the applicable decision theory which supports mathematical representation of agreement among SMEs when they give ordinal responses. We pose a process in a logical and methodical way that draws from this theory. We follow this by demonstrating the application of these measures through three examples of SME collections in support of the Capability Portfolio Review (CPR) process, the Tactical Wheeled Vehicle (TWV) reduction study V, and the Army's selection of a portfolio of future technologies in which to invest.