Premium
Evaluating Risk Assessments Using Receiver Operating Characteristic Analysis: Rationale, Advantages, Insights, and Limitations
Author(s) -
Mossman Douglas
Publication year - 2013
Publication title -
behavioral sciences and the law
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.649
H-Index - 74
eISSN - 1099-0798
pISSN - 0735-3936
DOI - 10.1002/bsl.2050
Subject(s) - receiver operating characteristic , feeling , meaning (existential) , risk assessment , poison control , sensitivity (control systems) , human factors and ergonomics , psychology , social psychology , cognitive psychology , computer science , risk analysis (engineering) , medicine , machine learning , computer security , medical emergency , engineering , psychotherapist , electronic engineering
The last two decades have witnessed major changes in the way that mental health professionals assess, describe, and think about persons' risk for future violence. Psychiatrists and psychologists have gone from believing that they could not predict violence to feeling certain they can assess violence risk with well‐above‐chance accuracy. Receiver operating characteristic (ROC) analysis has played a central role in changing this view. This article reviews the key concepts underlying ROC methods, the meaning of the area under the ROC curve (AUC), the relationship between AUC and effect size d , and what these two indices tell us about evaluations of violence risk. The area under the ROC curve and d provide succinct but incomplete descriptions of discrimination capacity. These indices do not provide details about sensitivity–specificity trade‐offs; they do not tell us how to balance false‐positive and false‐negative errors; and they do not determine whether a diagnostic system is accurate enough to make practically useful distinctions between violent and non‐violent subject groups. Justifying choices or clinical practices requires a contextual investigation of outcomes, a process that takes us beyond simply knowing global indices of accuracy. Copyright © 2013 John Wiley & Sons, Ltd.