The clinical utility of imaging-defined biotypes of depression and transcranial magnetic stimulation: A decision curve analysis

Fig. 1. Decision Curve Analysis (DCA) for neuroimaging models predictive of responsiveness to transcranial magnetic stimulation (TMS). The threshold probability (Pt) represents the point at which positive treatment response to TMS is valued equally to avoiding unnecessary treatment. Net benefit is defined as the percentage of individuals who receive TMS and achieve response minus a weighted percentage of treated individuals who do not respond. The dotted lines represent potential treatment strategies based on functional MRI-defined biotypes of depression from Drysdale et al. [7] When compared with alternative strategies of “treat all” or “treat none” (solid lines), the neuroimaging based strategies provide greater net benefit over a wide range of Pt values above 0.14. Psychiatry has been waiting for a neuroimaging test that can provide practical information to guide treatment decisions [1]. Despite decades of using magnetic resonance imaging to characterize psychiatric neurobiology, no imaging tests have been translated into clinical practice. Challenges to this goal include costs, relevant applications, reproducibility, and the absence of a simple method to evaluate a test’s clinical utility. The traditional statistical metrics reported in neuroimaging studies, such as sensitivity, specificity, and area under the curve, do not provide direct information about whether a test would change a clinical decision. To this end, we introduce the approach of decision curve analysis (DCA) to evaluate predictive neuroimaging models and demonstrate how DCA could be applied to the prediction of treatment response to transcranial magnetic stimulation (TMS) for major depression. DCA provides a framework to evaluate predictive models that incorporates the balancing of risks and benefits of treatment across a range of clinician and patient preferences [2]. DCA has been used to evaluate the clinical utility of predictive tests in oncology, cardiology, and other areas of medicine [3e5], but has yet to be adopted in psychiatry. The core componentofDCA is the conceptof “thresholdprobability” (Pt), or the probability at which an individual values the benefits of treatment equally to avoiding unnecessary treatment. If the probability of a condition being presentwere above the threshold probability, individuals would opt for treatment. Conversely, if this probability were below their threshold, individuals would avoid treatment. DCA calculates the net benefit of predictive models over a range of threshold probabilities and therefore a precise estimate of threshold probability is not required. The unit of net benefit in DCA is equal to the percentage of individuals appropriately treated (“true positives”) minus a weighted percentage of those inappropriately treated (“false positives”) given by a ratio of the threshold probability over its complement (as shown in Equation (1)). Therefore, at low threshold probabilities, the potential harm of false positives is considered low compared to the benefit of treatment. But, if the cost or risk of false positives were high, threshold probability increases and treatment would be reserved to individuals with a higher probability of the condition. The net benefit is then calculated over a range of threshold probabilities and is compared to “treat all” and “treat none” models. The strategy with the highest net benefit over a range of reasonable threshold probabilities is considered superior.