SASSI: A REPLY TO THE CRITIQUE OF FELDSTEIN & MILLER (2007)

The journal publishes both invited and unsolicited letters. It is at the editor's discretion whether such letters are sent out throught peer review In their review of the Substance Abuse Subtle Screening Inventory (SASSI), Feldstein & Miller use incorrect methods to calculate accuracy and make unsubstantiated claims of bias in the SASSI test classifications [1]. The authors reviewed 36 studies, only nine of which included a criterion variable and, of these, only six reported enough information to calculate sensitivity and specificity of the test classifications for the adult SASSI. Across studies there was substantial variation in prevalence rates (12–78%), samples (e.g. traumatic brain injury, college students, criminal offenders) and criterion variables. The adult SASSI was developed and validated against clinicians' DSM diagnoses of life-time substance dependence disorders [2]. By contrast, the studies reviewed utilized a nurse's query regarding substance use [3], DSM diagnoses of abuse, dependence or both, Composite International Diagnostic Interview (CIDI) diagnoses of only current alcohol dependence [4], Diagnostic Interview Schedule (DIS) substance dependence diagnoses [5] and a dichotomized variable created from justice system staff ratings on the Addiction Severity Index [6,7]. Further, only three of these studies used the current version of the instrument (SASSI-3), which differs from its earlier versions in scale compositions, cut-offs and decision rules. Feldstein & Miller ignore the impact of different revisions to the SASSI. They presented findings from SASSI-2 studies as ‘failures to replicate’ SASSI-3 findings and calculated average performance statistics across all the aforementioned variables to arrive at their conclusions. We differ with their procedures, calculations and conclusions. In reviewing the data presented to evaluate criterion validity, first recall that six of the nine studies employed earlier versions of the adult SASSI. Secondly, in an attempt to summarize sensitivity and specificity across studies, Feldstein & Miller multiplied observed sensitivity and specificity for each study by the total study n, and then averaged. This distorts the meaning of sensitivity and specificity, i.e. accuracy with respect to the criterion status of the case, and produces artifactually lower values. For example, studies with small sample sizes that show high sensitivity are weighted less than larger studies showing less sensitivity, even though the actual number of criterion positive cases, and thus the ‘opportunity’ to demonstrate sensitivity might be the same. Similarly, low sensitivity in a large study has substantial influence on the weighted average calculated by Feldstein and Miller, even if the actual prevalence, i.e. ‘opportunity value’ for demonstrating sensitivity is low. The outcome of utilizing this weighting strategy is that their reported average sensitivity underestimated observed sensitivity in the six studies with full criterion information by 14%, and underestimated observed specificity by 12%. Weighting n by observed prevalence (or 1-prevalence for specificity) instead provides the number of criterion positive and criterion negative cases for each study and allows for the standard calculation of sensitivity and specificity. Again, these standard performance statistics calculate identification with respect to the criterion positive and criterion negative ‘opportunities’ in each study, rather than the total sample n. When the studies that contained full criterion positive and criterion negative information are combined and these standard equations applied, the percent of criterion positives that are test positive (sensitivity) is 84%, and the percent of criterion negatives that are test negative (specificity) is 74%. These findings can be contrasted with findings for direct screening instruments regarding current and life-time diagnoses of alcohol dependence: Alcohol Use Disorders Identification Test (AUDIT): sensitivity: 0.74 (current), 0.54 (life-time), specificity: 0.86, 0.86; Michigan Alcoholism Screening Test (MAST) sensitivity: 0.63, 0.56, specificity: 0.80, 0.81; Cut-down, Annoyed, Guilt, Eye-opener (CAGE), sensitivity: 0.37, 0.42, specificity: 0.89, 0.91 ([8]; see also [9,10]). The authors also interpret associations between SASSI scores and demographic characteristics as suggesting ‘an overclassification bias for the SASSI when used with ethnic minorities’ ([1], p. 49). A substance use disorder (SUD) screening instrument that shows higher scale scores for one ethnic group than another is not biased if the two groups show different prevalence rates and classification accuracy does not differ for the groups (cf. [11,12]). The one study evidencing differences in SASSI test classifications as a function of ethnicity used the original adolescent SASSI to screen learning-disabled students for chemical dependence [13]. Findingsindicated that a significantly higher proportion of non-Caucasian students were learning-disabled and that learning disability predicted whether students tested positive on the SASSI, but that ethnicity, while correlated, did not. None of the findings cited show bias in SASSI scores as a function of subject characteristics. A central theme is that the authors believe the SASSI subtle scales do not add enough sensitivity to warrant their use. They misquote the SASSI-3 manual [14] to make this point: ‘The test manual reports that the direct scales [rules 1–3] detected only 79% [actually 74%] of actual SUDs, whereas adding the indirect [subtle] scales increased sensitivity to 94%’. Also, although the positive predictive power of the face valid scales was 100%, their negative predictive power was 50%. That the subtle scales improve detection appears to be evidence for using, not discarding them. Evidence of the advantage of the subtle scales has also been shown in other research. Myerholtz & Rosenberg [15] found that scores on the SASSI face valid alcohol (FVA) and drug (FVOD) scales dropped between one and two standard deviations to average scores for the normative population under instructions to fake good. Scores on the Subtle Attributes scale, designed to resist faking, did not change with attempts to fake good, and scores on the Defensiveness (DEF) scale, developed to identify response sets to minimize problems, increased nearly two standard deviations. These scores indicate that none of the fake good subjects would have been identified by the face valid scales, and yet nearly all subjects would have been recognized as having extreme DEF scores. Access to a defensiveness scale allows one to examine possible minimization. A final advantage to using both direct and indirect scales goes beyond mere screening. Just as a diagnostic interview to determine if a client has an SUD provides information beyond the presence or absence of SUD, so the SASSI can provide information in addition to classification. Just as two eyes can serve not just as an independent check on what each eye can see but also provide depth perception, subtle and direct scales give a more complete picture. What clinician would be indifferent to the degree to which a new client is unwilling to recognize the impact of alcohol and drug use in his or her life?