Premium
Valuing Structured Professional Judgment: Predictive Validity, Decision‐making, and the Clinical‐Actuarial Conflict
Author(s) -
Falzer Paul R.
Publication year - 2013
Publication title -
behavioral sciences and the law
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.649
H-Index - 74
eISSN - 1099-0798
pISSN - 0735-3936
DOI - 10.1002/bsl.2043
Subject(s) - receiver operating characteristic , computer science , predictive validity , risk assessment , actuarial science , data mining , risk analysis (engineering) , psychology , machine learning , medicine , clinical psychology , computer security , business
Structured professional judgment (SPJ) has received considerable attention as an alternative to unstructured clinical judgment and actuarial assessment, and as a means of resolving their ongoing conflict. However, predictive validity studies have typically relied on receiver operating characteristic (ROC) analysis, the same technique commonly used to validate actuarial assessment tools. This paper presents SPJ as distinct from both unstructured clinical judgment and actuarial assessment. A key distinguishing feature of SPJ is the contribution of modifiable factors, either dynamic or protective, to summary risk ratings. With modifiable factors, the summary rating scheme serves as a prognostic model rather than a classification procedure. However, prognostic models require more extensive and thorough predictive validity testing than can be provided by ROC analysis. It is proposed that validation should include calibration and reclassification techniques, as well as additional measures of discrimination. Several techniques and measures are described and illustrated. The paper concludes by tracing the limitations of ROC analysis to its philosophical foundation and its origin as a statistical theory of decision‐making. This foundation inhibits the performance of crucial tasks, such as determining the sufficiency of a risk assessment and examining the evidentiary value of statistical findings. The paper closes by noting a current effort to establish a viable and complementary relationship between SPJ and decision‐making theory. Copyright © 2013 John Wiley & Sons, Ltd.