This article was downloaded by: [Michigan State University] On: 11 January 2015, At: 03:53 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Substance Abuse Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wsub20
Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients a
ab
ab
a
Laura E. Winters MS , Anne Karow MD , Jens Reimer MD , Susanne Fricke PhD , Olaf a
ab
Kuhnigk MD & Ingo Schäfer MD, MPH a
Department of Psychiatry and Psychotherapy, University Medical Center, HamburgEppendorf, Hamburg, Germany b
Center for Interdisciplinary Addiction Research, University of Hamburg, Hamburg, Germany Accepted author version posted online: 17 Mar 2014.Published online: 08 Aug 2014.
Click for updates To cite this article: Laura E. Winters MS, Anne Karow MD, Jens Reimer MD, Susanne Fricke PhD, Olaf Kuhnigk MD & Ingo Schäfer MD, MPH (2014) Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients, Substance Abuse, 35:3, 262-267, DOI: 10.1080/08897077.2014.891555 To link to this article: http://dx.doi.org/10.1080/08897077.2014.891555
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
SUBSTANCE ABUSE, 35: 262–267, 2014 Copyright Ó Taylor & Francis Group, LLC ISSN: 0889-7077 print / 1547-0164 online DOI: 10.1080/08897077.2014.891555
Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients Laura E. Winters, MS,1 Anne Karow, MD,1,2 Jens Reimer, MD,1,2 Susanne Fricke, PhD,1 Olaf Kuhnigk, MD,1 and Ingo Sch€afer, MD, MPH1,2 Downloaded by [Michigan State University] at 03:53 11 January 2015
ABSTRACT. Background: A high prevalence of comorbid posttraumatic stress disorder (PTSD) is found in patients with substance use disorders (SUDs). In the few existing studies, mixed results regarding the psychometric properties of common screening instruments for PTSD have been reported for patients with SUDs. No results are available for the Posttraumatic Diagnostic Scale (PDS), an established self-report measure for PTSD.Methods: The authors assessed 105 patients with alcohol dependence according to DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) (70% male) 2 weeks after their admission to an inpatient detoxification unit. Participants were administered the PDS, the PTSD module of the Structured Clinical Interview for DSM-IV (SCID), as well as measures of depression and anxiety. Patients with other substance use disorders were excluded as were patients reporting no traumatic event. Results: Internal consistencies were good to very good for the total scale (.93) and the subscales of the PDS (.82–.91). In our sample, the PDS had a high specificity (.89) but only moderate sensitivity (.57). Diagnostic agreement with the SCID was 83% (.46). The results of a receiver operating characteristic (ROC) analysis suggested that a PDS score of 8 was the optimal cutoff to screen for PTSD. The highest diagnostic agreement between PDS and SCID (89%; .60) was achieved using a cutoff score of 24. Conclusions: These findings confirm previous results suggesting that the psychometric properties of self-report measures of PTSD in patients with SUDs might differ from those in the general population. When the PDS is used in recently detoxified patients with alcohol dependence, it seems advisable to modify the cutoff score of this instrument to improve its sensitivity and diagnostic accuracy.
Keywords: Alcohol dependence, Posttraumatic Diagnostic Scale, PTSD, sensitivity, specificity
INTRODUCTION Over the past decade, the importance of co-occurring posttraumatic stress disorder (PTSD) and substance use disorder (SUD) has become increasingly apparent. Of all patients treated for SUDs, 26%–52% have a lifetime diagnosis of PTSD, and 15%– 41% currently meet PTSD criteria.1,2 SUD patients with PTSD have a more severe clinical profile compared with patients without 1
Department of Psychiatry and Psychotherapy, University Medical Center, Hamburg-Eppendorf, Hamburg, Germany 2 Center for Interdisciplinary Addiction Research, University of Hamburg, Hamburg, Germany Correspondence should be addressed to Ingo Sch€afer, MD, MPH, Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany. E-mail: i. [email protected]
PTSD. They have an earlier onset of substance abuse and more years of problematic use,3,4 they report more polydrug use,5,6 and they have greater severity of current substance use.6,7 Moreover, PTSD interferes with patients’ ability to benefit from SUD treatment. Studies in different samples of SUD patients suggest that those with PTSD have a poorer adherence to treatment and a shorter duration of abstinence,8,9 and they have consistently worse outcomes across a variety of measures.10 Despite the high prevalence of PTSD in patients with SUDs, and the obvious need for specific treatment,11 few diagnostic instruments have been evaluated for their utility in diagnosing PTSD in SUD populations. The few existing studies yielded inconsistent results regarding the psychometric properties of wellestablished measures of PTSD in patients with SUDs. Kimerling et al.12 found that the Primary Care-PTSD Scale (PC-PTSD13), a 4-item screener for PTSD, had a higher sensitivity and a specificity that was only slightly lower in patients with SUDs (.91 and
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL.
.80, respectively) as compared with a primary care sample (.78 and .87, respectively). Similar results were reported for the modified version of the PTSD Symptom Scale (PSS-SR14), a predecessor version of the instrument that is the focus of this paper. Coffey and colleagues15 found a comparable sensitivity of the PSS-SR and a slightly lower specificity in a substance abuse sample (.89 and .65, respectively) as compared with a community sample (.91 and .72, respectively16). Several other studies reported weaker psychometric properties of established measures of PTSD in patients with SUDs. Rash et al.17 compared the Impact of Events Scale—Revised (IES-R18) in patients with either alcohol dependence or drug dependence to a clinical interview (Clinician Administered PTSD Scale19). They found that the diagnostic agreement between both instruments was only moderate (.50), and that the sensitivity and specificity of the IES-R were markedly lower in their sample (.73 and .72, respectively) than had been reported for a community sample (.91 and .82, respectively20). In contrast to the proposed cutscore of 33,20 Rash et al.,17 therefore, suggested a lower cutoff score of 22 for screening purposes in patients with SUDs, resulting in a sensitivity of .92 and a specificity of .57 in their sample. Harrington and Newman21 reported similar results for the PTSD Checklist—Civilian Version (PCL-C22), and the Penn Inventory for Posttraumatic Stress Disorder.23 Although a cutoff score of 44 on the PCL-C resulted in a sensitivity of .90 and a specificity of .95 in a sample of college students,24 both parameters were markedly lower in their sample of female patients with SUDs (.76 and .79, respectively). A lower cutoff score of 38 resulted in a sensitivity of .82 and a specificity of .66 in this sample. Similarly, the Penn Inventory for Posttraumatic Stress Disorder23 had a high sensitivity and specificity (both .90) in a population consisting of veterans and civilians,23 but a much lower sensitivity and specificity in the SUD sample (.86 and .53, respectively). In contrast to the cutoff score of 35 applied by Hammarberg,23 Harrington and Newman,21 therefore, proposed a lower cutoff score of 25, resulting in a sensitivity of .82 and a specificity of .64 in their patients. The findings of these studies underline the need to evaluate the psychometric properties of established instruments for PTSD when used in patients with SUDs. The weaker diagnostic accuracy of these instruments seems to be due to both a lower sensitivity and a lower specificity. Harrington and Newman21 suggested that patients with SUDs might minimize their PTSD symptoms, making lower cutoff scores necessary to achieve an acceptable sensitivity. Another explanation could be that the degree of symptoms in the 3 symptom clusters of PTSD (i.e., intrusions, avoidance, and hyperarousal) might differ between patients with and without SUDs, and even between SUD patients with different types of substance abuse.25 A question of high clinical relevance, finally, is how accurately a diagnosis of PTSD can be made when self-report questionnaires are used among patients in detoxification treatment, given the high overlap of some PTSD symptoms with symptoms of acute withdrawal (e.g., feeling irritable or having trouble to fall asleep) and the overlap with comorbid conditions, namely, depression. Depressive symptoms are present in many patients when entering detoxification treatment and seem to remit in most of them in the course of the treatment.26 Even if PTSD symptoms remain stable over time, the changes in comorbid symptoms or the medication that might have been started could influence their assessment.
263
The Posttraumatic Diagnostic Scale27 (PDS) is one of the most widely used instruments to assess PTSD,28 but to our knowledge no findings on its psychometric properties among patients with SUDs have been published so far. The aims of our study, therefore, were to (1) assess the psychometric properties of the PDS in a sample of alcohol-dependent patients undergoing detoxification treatment and (2) examine the influence of different cutoff scores on the sensitivity and specificity of the PDS in order to make recommendations for its use in alcohol-dependent patients.
METHODS Study Sample In the current study, we examined patients who were consecutively admitted to a specialized detoxification unit at the University Medical Center Hamburg-Eppendorf (Hamburg, Germany). Inclusion criteria were a DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) diagnosis of alcohol dependence, an age between 18 and 65 years, and sufficient language abilities to complete the self-report measures. Exclusion criteria were a substance use disorder other than alcohol dependence, psychotic symptoms, and severe cognitive impairments. In the current analysis, a total of 105 individuals reporting at least 1 traumatic event according to PTSD criterion A29 were included. The interviews were conducted 10–14 days after admission by trained raters. At this point in treatment, all participants had remained abstinent from alcohol and none of them still displayed symptoms of withdrawal. All participants provided written consent after having been informed on the procedures and purpose of the study. The study was approved by the local ethics committee, the Chamber of Physicians of the State of Hamburg.
Assessment Instruments The Posttraumatic Diagnostic Scale27 (PDS) consists of 49 items and has 4 different parts. The first 2 parts of the instrument address the criterion A of PTSD using a checklist of 12 potentially traumatic events. The patient is then asked to name the worst event (index trauma) and is asked more specific yes/no questions to check if criterion A2 is fulfilled. The third part consists of 17 items assessing criteria B to D on the subscales reexperiencing (5 items), avoidance (7 items), and arousal (5 items). These items can be used to diagnose PTSD according to DSM-IV. On a 4-point Likert-scale (0 D “not at all or only one time” to 3 D “five or more times a week/ almost always”) the frequency of symptoms related to the index trauma is rated for the last month. Two additional items assess the duration of symptoms (criterion E) and a possible delayed onset. Part 4 gives a list of 8 life areas that can be impaired by PTSD symptoms (criterion F). For criteria B to D to be fulfilled, at least 1 reexperiencing symptom, 3 symptoms of avoidance, and 2 symptoms of arousal must be rated “1” or higher in the PDS. If the duration of the symptoms is at least 1 month (criterion E) and impairment is reported in at least 1 area of life (criterion F), a diagnosis of PTSD is given. Apart from the diagnosis of PTSD, the PDS provides a symptom severity score that can be obtained by summing up the 17 items assessing criteria B to D. Foa27 suggested a score of 11 to be indicative of moderate and a score of 21 to be indicative of severe PTSD. In the current study,
Downloaded by [Michigan State University] at 03:53 11 January 2015
264
SUBSTANCE ABUSE
the German version of the PDS was used.30 Studies among non– substance-abusing populations reported strong psychometric properties of the PDS. In a study among 248 patients of PTSD treatment centers and individuals from high-risk populations, Foa et al.31 found good internal consistencies of the total score (.92) and all subscales (.78–.84), as well as an agreement with the Structured Clinical Interview for DSM-IV (SCID32) of 82% (.65). The sensitivity of the scale was .89, and the specificity was .75. Correlations with anxiety and depression measures were high (Beck Depression Inventory: r D .79; State-Trait Anxiety Inventory: r D .74). Similar psychometric properties have been reported for the German version.33 The PTSD module of the Structured Clinical Interview for DSM-IV (SCID32) was used to establish diagnoses of PTSD and measure the convergent validity of the PDS in this study. The SCID assesses the 17 DSM-IV-symptoms of posttraumatic stress disorder as well as their severity. Before the interview, a trauma checklist is provided including 8 different categories of potentially traumatic events. The State-Trait Anxiety Inventory (STAI34) and the Beck Depression Inventory (BDI35) were used to assess the discriminant validity of the PDS. The STAI is an internationally used 40-item self-report measure for anxiety that yields 2 separate scores for “state” as well as “trait anxiety.” According to Spielberger and Vagg,34 the STAI shows internal consistencies between .86 and .95, and its test-retest reliability ranges from r D .75 to r D .86. The BDI is a 21-item self-report measure assessing the severity of depressive symptoms. The BDI shows an average reliability of .88 in clinical samples and a construct validity ranging from .58 to .79.36 Keller et al.37 found a good internal consistency of .90 for the BDI in a sample of alcohol-dependent inpatients. The European Addiction Severity Index (EuropASI38) was used to assess substance-related problems. The EuropASI has been developed on the basis of the fifth version of the American Addiction Severity Index.39 Ten composite scores give an insight into important areas such as drug and alcohol use, social relationships, medical and psychiatric conditions, as well as the legal, employment, and economical situation of the participants. Composite scores range between 0 (“no treatment required”) and 1 (“treatment extremely required”). Chronicity of alcohol use was estimated from the participants’ reports of years of lifetime problematic alcohol use, which was defined as using at least 3 times per week or bingeing on at least 2 consecutive days per week. Scheurich et al40 found good psychometric properties for the German version of the EuropASI.
Data Analysis Chi-square tests and independent-sample t tests were used to assess group differences in sociodemographic and clinical variables. The internal reliability of the PDS was assessed by calculating Cronbach’s alpha for the subscales and the total severity score of the PDS. The agreement with the SCID was tested using the chance corrected classification agreement kappa. An independentsample t test was used to compare the PDS severity scores of participants with and without PTSD. To determine the discriminant validity of the PDS, Spearman correlations with relevant psychiatric constructs (depression and anxiety) were calculated. Finally, a receiver operating characteristic (ROC) analysis was used to compare different cutoff scores of the PDS and to determine the
resulting sensitivity and specificity. All data were analyzed using SPSS 17 for Windows (SPSS Inc., Chicago, IL, USA).
RESULTS Participant Characteristics Of the 105 participants included in the analysis, 30% were female. Participants’ average age was M D 41.3 years (SD D 9.6; range D 19–64). Almost half of the participants (45%) were employed, 63% had completed at least 10 years of school, and only 4% had left school without any qualification. Seventeen percent were married, and 29% lived in a stable relationship. The average age at onset of problematic alcohol use was M D 24.0 years (SD D 10.9; range D 5–53). The average duration of chronic alcohol abuse according to the EuropASI was M D 13.0 years (SD D 9.9; range D 0–37), and the participants reported an average of M D 2.6 previous detoxifications (SD D 3.3; range D 0–18). The results of the EuropASI composite scores can be seen in Table 1. Three quarters of the participants reported having experienced more than 1 category of traumatic events, with an average of M D 3.1 categories. The most frequent index traumatic events were nonsexual assault by known assailants (19%), nonsexual assault by unknown assailants (18%), and accidents (18%). According to the SCID, n D 21 participants (20%; n D 9 females and n D 12 males) were diagnosed with posttraumatic stress disorder (PTSD). Patients diagnosed with PTSD most frequently reported sexual assault by a known assailant as their worst experience (33%), and they also reported having experienced a higher total number of different traumatic events (M D 4.0).
Reliability and Validity of the PDS The internal reliability (Cronbach’s alpha) for the PDS total scale was .93 for the whole sample and .90 for the patients with a SCID diagnosis of PTSD. The internal reliabilities of the subscales were .91 for the reexperiencing scale, .82 for the avoidance scale, and .85 for the hyperarousal scale (for the patients with PTSD .87, .74, and .71, respectively). The subscales and the total PDS were TABLE 1 Means and Standard Deviations of the European Addiction Severity Index (EuropASI) Composite Scores ASI composite score
Mean
Standard deviation
Drug Legal Family Social relationships Employment Medical condition Psychiatric status Economical situation Alcohol
.02 .06 .18 .11 .25 .34 .30 .63 .53
.05 .13 .22 .16 .31 .32 .21 .38 .14
Note. N D 105. Means ranging between 0 ( D “no treatment required“) and 1 ( D “treatment extremely required”).
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL.
highly correlated among each other in the whole sample (see Table 2). As a first step to assess the validity of the PDS, we compared the PDS severity scores between patients diagnosed with PTSD in the SCID interview and patients without PTSD. The patients diagnosed with PTSD (n D 21) reached a significantly higher symptom severity as assessed by the PDS total score than participants without PTSD (n D 84; M D 22.19 vs. 5.39; t(23.9) D ¡6.03; P < .001). As for the total score, significant differences were found for all subscales of the PDS (subscale reexperiencing: M D 6.24 vs. 1.32; t(23.22) D ¡4.61; P < .001; subscale avoidance: M D 8.00 vs. 1.92; t(23.63) D ¡5.08; P < .001; subscale arousal: M D 8.05 vs. 2.16; t(103) D ¡7.16; P < .001). Convergent validity was obtained by comparing the diagnoses obtained by the PDS with those of the SCID interview. Of all participants, n D 87 (83%) were correctly classified by the PDS (.46). The sensitivity was .57, the specificity .89. Of the participants without PTSD, n D 75 out of n D 84 were correctly classified (89%), whereas only n D 12 out of n D 21 participants with PTSD were correctly classified (57%). Discriminant validity was assessed through Spearman correlations between the PDS and measures of depression and anxiety. Correlations with the BDI were small to moderate for the PDS total scale (r D .30; P D .01), the avoidance scale (r D .23; P D .05), and the hyperarousal scale (r D .31; P D .01), and not significant for the reexperiencing subscale (see Table 2). Correlations of the PDS with the 2 scales of the STAI were not significant, except for a small correlation of trait anxiety with the hyperarousal scale of the PDS (r D .24; P D .04).
265
TABLE 3 Effectiveness Indices for Selected Cutoff Scores of the Posttraumatic Stress Diagnostic Scale (PDS) Symptom Severity Score Cutoff score 7 8 9 10 11 12 18 20 21 22 23 24 25
Sensitivity
Specificity
Diagnostic accuracy
Kappa
.905 .905 .857 .857 .810 .762 .667 .619 .619 .619 .619 .571 .429
.690 .726 .786 .798 .810 .821 .893 .917 .917 .929 .929 .964 .976
73% 84% 80% 81% 81% 81% 85% 86% 86% 87% 87% 89% 87%
.42 .51 .51 .52 .51 .50 .54 .56 .55 .57 .57 .60 .49
Note. Bold values indicate the recommended cutoff value for use as a screening tool8 and for best agreement with the SCID.24
is 8, achieving maximal specificity (.73) while meeting a sensitivity of over .90, which has been proposed as an a priori goal for sensitivity.17 Using this cutoff value, 84% of the sample were correctly classified (.51). The optimal agreement with the SCID was obtained using a cutoff value of 24 (sensitivity D .57; specificity D .96).
DISCUSSION Determining Diagnostic Values The PDS not only indicates if each of the DSM-IV criteria for PTSD is met, but also provides a dimensional symptom severity score. The proposed cutoff values are 11 for mild to moderate symptoms, 21 for moderate to severe symptoms, and 36 for severe symptoms.27 In a ROC analysis, we compared different cutoff scores with regard to their sensitivity, specificity, and diagnostic accuracy (see Table 3). In order to use the PDS as a screening tool, our analysis suggests that the most suitable cutoff score TABLE 2 Intercorrelations Between PDS Subscales and Correlations With Measures of Anxiety and Depression PDS Total PDS Total Reexperiencing Avoidance Arousal STAI State Trait BDI
Reexperiencing
Avoidance
— .86** .91** .93**
— .82** .70**
— .76**
.14 (ns) .14 (ns) .30**
.11 (ns) .08 (ns) .20 (ns)
.12 (ns) .11 (ns) .23*
Arousal
.19 (ns) .24** .31**
Note. PDS D Posttraumatic Diagnostic Scale; STAI D State-Trait Anxiety Inventory; BDI D Beck Depression Inventory. *P < .05; **P < .01; ns D nonsignificant.
Summary and Discussion of the Results The results of our study suggest that the Posttraumatic Diagnostic Scale is a reliable instrument in patients with alcohol dependence. The internal consistencies found in this study were high and correspond well with published results for the PDS in community samples.31,33,41 Moreover, the diagnostic agreement between the PDS and the SCID was comparable to the findings of Foa et al.31 Although the specificity of the PDS in our sample exceeded the specificity reported in both of these studies, the sensitivity fell into a moderate range. This is in line with the findings of Rash et al.17 and Coffey et al.,15 who suggested that self-report measures of PTSD may have differing psychometric properties in substance abuse samples. A possible explanation for the low sensitivity of the PDS in our study is that for a diagnosis of PTSD, enough symptoms in all clusters according to DSM-IV have to be present. Substance abuse and detoxification might dampen some of the PTSD symptoms or influence their perception,21 leading to a lower sensitivity in this group of patients. As all patients included in our study had only recently undergone detoxification treatment, the use of alcohol in the weeks before admission may have dampened some of their symptoms, leading to a lower sensitivity of the PDS. Our results cannot be explained by Streiner’s observation42 that a lower base rate of a disease in a population usually increases the amount of false positives in a sample, as the base rate in this study was lower than those reported by Foa et al.31 and Griesel et al.33 Moreover, as Brenner and Gefeller43 argue, base rate is only one of the sample characteristics influencing sensitivity and specificity of a test.
Downloaded by [Michigan State University] at 03:53 11 January 2015
266
SUBSTANCE ABUSE
Further indicators of differing psychometric properties of the PDS in substance abuse samples are the low correlations of the PDS with measures of anxiety and the small to moderate correlation with depression in our study. In community samples, these correlations are expected to be moderate to high, as anxiety and depression are overlapping constructs but not identical with PTSD.44 In contrast, studies among patients with substance abuse reported lower correlations between measures of PTSD and depression.17,45 Interestingly, the finding by Foa et al,31 that the subscales avoidance and arousal tend to correlate higher with depression than the reexperiencing scale, is supported by our findings. Given the low sensitivity, using the PDS with conventional DSM-IV scoring criteria to screen for PTSD is not recommended. The results of our ROC analysis suggest that a cutoff value of 8 can be recommended to enhance the sensitivity of the PDS in substance abuse settings when it is used as a screening tool. This score is close to the score of 11 suggested by Foa27 as a cutoff for moderate PTSD symptom severity. When a cutoff score of 8 was used, the PDS identified over 90% of the PTSD cases in our sample at the cost of more false-positive cases (specificity .726). As shorter measures appear to be similarly accurate,12 the value of using the PDS as a screening tool in clinical settings could be questioned. On the other hand, the PDS has the advantage to allow both a dimensional assessment and a clinical diagnosis and is therefore widely used in clinical as well as in research settings. The highest diagnostic agreement between the PDS and the SCID could be reached using the cutoff score of 24. This score is close to Foa’s27 suggestion for a cutoff score between moderate and severe symptomatology. Used with the cutoff score of 24, the PDS correctly classified almost all cases of PTSD (specificity .964) at the expense of false-negative cases (sensitivity .571). In clinical settings, the cutoff score should be chosen depending on the consequences of possible false-positive or false-negative ratings and the primary intention (ie, broader screening vs exact diagnosis). When the arguments mentioned above are weighed against each other, it seems advisable to use the PDS in patients with substance abuse as a more specific measure to validate the diagnosis of PTSD after using a brief screener.
Another potential limitation is related to the timing of assessment in our study, where we examined patients 2 weeks after termination of active use. Although PTSD symptoms can decrease or increase during detoxification,25 Coffey et al.16 stated that major changes in symptoms should be completed within 2 weeks after termination of active use. Nevertheless, it remains difficult to determine the exact effects of withdrawal or comorbid psychopathology on self-rating instruments, and the findings cannot be generalized to patients who have been abstinent for a longer period of time or actively using patients. Symptoms of PTSD should therefore be assessed repeatedly in the course of treatment to enhance the diagnostic validity. Moreover, the restriction of our sample to patients with alcohol dependence limits the generalizability of our findings to patients with other types of substance abuse. From a clinical perspective, however, alcohol-dependent patients entering detoxification treatment represent an important subgroup, which is why we focused on patients in this setting. For methodological reasons, patients reporting no traumatic event had to be excluded from the study. Although it could be argued that this further decreases the generalizability of our findings, this doesn’t seem to be the case, as the prevalence of traumatic events is extremely high in substance abuse samples (e.g., Rash et al.17: 89%–94%). Due to the short duration of stay of patients in our hospital, it was not possible to determine the test-retest reliability of the PDS. This question should be addressed by further studies together with the psychometric properties of the instrument in alcohol-dependent patients after a longer period of abstinence and in patients with other types of substance abuse.
AUTHOR CONTRIBUTIONS IS designed the study. All analyses were performed by LW and SF. The first draft of the manuscript was written by LW and IS. All authors commented on and approved the final manuscript.
REFERENCES Limitations and Future Directions For the interpretation of our findings, some important limitations have to be taken into account. First of all, as the PDS has been tailored to assess PTSD according to DSM-IV criteria, it is an important question how our findings should be interpreted in the context of the new DSM-5 diagnosis of PTSD, where some symptoms of the DSM-IV cluster C have been combined with additional symptoms to form a new cluster D (“negative alterations in cognitions and mood”), and the former cluster D has been transferred into a new cluster E (“alterations in arousal and reactivity”). In our study, the PDS scores were significantly correlated with depression as measured by the BDI. If symptoms of depression are considered a proxy of the new DSM-5 cluster “negative alterations in cognitions and mood,” the scale might still be of use for identifying patients when the new diagnostic criteria are applied. A recent paper on the relationships between the new symptom clusters in DSM-5 and the DSM-IV diagnosis of PTSD also highlighted the close relationships between both constructs.46
[1] Jacobsen LK, Southwick SM, Kosten TR. Substance use disorders in patients with posttraumatic stress disorder: a review of the literature. Am J Psychiatry. 2001;158:1184–1190. [2] Sch€afer I, Najavits LM. Clinical challenges in the treatment of patients with posttraumatic stress disorder and substance abuse. Curr Opin Psychiatry. 2007;20:614–618. [3] Johnson SD, Striley C, Cottler LB. The association of substance use disorders with trauma exposure and PTSD among African American drug users. Addict Behav. 2006;31:2063–2073. [4] Read JP, Brown PJ, Kahler CW. Substance use and posttraumatic stress disorders: symptom interplay and effects on outcome. Addict Behav. 2004;29:1665–1672. [5] Dragan M, Lis-Turlejska M. Prevalence of posttraumatic stress disorder in alcohol dependent patients in Poland. Addict Behav. 2007;32:902–911. [6] Mills KL, Lynskey M, Teesson M, Ross J, Darke S. Post-traumatic stress disorder among people with heroin dependence in the Australian treatment outcome study (ATOS): prevalence and correlates. Drug Alcohol Depend. 2005;77:243–249.
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL. [7] Clark HW, Masson CL, Delucchi KL, Hall SM, Sees KL. Violent traumatic events and drug abuse severity. J Subst Abuse Treat. 2001;20:121–127. [8] Brown PJ, Read JP, Kahler CW. Comorbid posttraumatic stress disorder and substance use disorders: treatment outcomes and the role of coping. In: Ouimette P, Read JP, eds. Trauma and Substance Abuse. Causes, Consequences and Treatment of Comorbid Disorders. Washington DC: American Psychological Association; 2003:171–188. [9] Hien DA, Nunes E, Levin FR, Fraser D. Posttraumatic stress disorder and short-term outcome in early methadone treatment. J Subst Abuse Treat. 2000;19:31–37. [10] Najavits LM, Harned MS, Gallop RJ, et al. Six-month treatment outcomes of cocaine-dependent patients with and without PTSD in a multisite national trial. J Stud Alcohol Drugs. 2007;68:353–361. [11] Henslee AM, Coffey SF. Exposure therapy for posttraumatic stress disorder in a residential substance use treatment facility. Prof Psychol Res Pr. 2010;41:34–40. [12] Kimerling R, Trafton JA, Nguyen B. Validation of a brief screen for post-traumatic stress disorder with substance use disorder patients. Addict Behav. 2006;31:2074–2079. [13] Prins A, Ouimette PC, Kimerling R, Cameron RP, Hugelshofer DS, Shaw-Hegwer J. The primary care PTSD screen: development and operating characteristics. Primary Care Psychiatry. 2004;9:9–14. [14] Foa EB, Riggs DS, Dancu CV, Rothbaum BO. Reliability and validity of a brief instrument for assessing posttraumatic stress disorder. J Trauma Stress. 1993;6:459–473. [15] Coffey SF, Dansky BS, Falsetti SA, Saladin ME, Brady KT. Screening for PTSD in a substance abuse sample: psychometric properties of a modified version of the PTSD Symptom Scale Self-Report. Posttraumatic stress disorder. J Trauma Stress. 1998;11:393–399. [16] Coffey SF, Gudmundsdottir B, Beck JG, Palyo SA, Miller L. Screening for PTSD in motor vehicle accident survivors using the PSS-SR and IES. J Trauma Stress. 2006;19:119–128. [17] Rash CJ, Coffey SF, Baschnagel JS, Drobes DJ, Saladin ME. Psychometric properties of the IES-R in traumatized substance dependent individuals with and without PTSD. Addict Behav. 2008;33:1039– 1047. [18] Weiss D, Marmar C. The Impact of Event Scale—Revised. In: Wilson JP, Keane TM, eds. Assessing Psychological Trauma and PTSD: A Handbook for Practitioners. New York: Guilford Press; 1997:399–411. [19] Blake DD, Weathers FW, Nagy LM, et al. The development of the Clinical-Administered PTSD Scale. J Trauma Stress. 1995;8:75–90. [20] Creamer M, Bell R, Failla S. Psychometric properties of the Impact of Event Scale—Revised. Behav Res Ther. 2003;41:1489–1496. [21] Harrington T, Newman E. The psychometric utility of two self-report measures of PTSD among women substance users. Addict Behav. 2007;32:2788–2798. [22] Weathers FW, Litz BT, Huska JA, Keane TM, eds. PTSD Checklist—Civilian Version. Boston: Boston National Center for PTSD, Behavioral Science Division; 1994. [23] Hammarberg M. Penn Inventory for posttraumatic stress disorder: psychometric properties. Psychol Assess. 1992;4:204–216. [24] Ruggiero KJ, Rheingold AA, Resnick HS, Kilpatrick DG, Galea S. Comparison of two widely used PTSD-screening instruments: implications for public mental health planning. J Trauma Stress. 2006;19:699–707. [25] Saladin ME, Brady KT, Dansky BS, Kilpatrick DG. Understanding comorbidity between PTSD and substance use disorders: two preliminary investigations. Addict Behav. 1995;20:643–655.
267
[26] Davidson KM. Diagnosis of depression in alcohol dependence: changes in prevalence with drinking status. Br J Psychiatry. 1995;166:199–204. [27] Foa EB, ed. Posttraumatic Stress Diagnostic Scale: Manual. Minneapolis, MN: National Computer Systems; 1995. [28] Elhai JD, Gray MJ, Kashdan TB, Franklin CL. Which instruments are most commonly used to assess traumatic event exposure and posttraumatic effects? A survey of traumatic stress professionals. J Trauma Stress. 2005;18:541–545. [29] American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington DC: American Psychiatric Association; 1994. € [30] Ehlers A, Steil R, Winter H, Foa EB, eds. Deutsche Ubersetzung der Posttraumatic Diagnostic Scale (PDS). Oxford, UK: Oxford University, Warneford Hospital, Department of Psychiatry; 1996. [31] Foa EB, Cashman L, Jaycox L, Perry K. The validation of a selfreport measure of posttraumatic stress disorder: the Posttraumatic Diagnostic Scale. Psychol Assess. 1997;9:445–451. [32] First RL, Spitzer M, Gibbon J, eds. Structured Clinical Interview for DSM-IV Axis I Disorders—Patient Edition (SCID-I/P, Version 2.0). New York: Biometrics Research Department, New York State Psychiatric Institute; 1996. [33] Griesel D, Wessa M, Flor H. Psychometric qualities of the German version of the Posttraumatic Diagnostic Scale (PTDS). Psychol Assess. 2006;18:262–268. [34] Spielberger CD, Vagg PR. Psychometric properties of the STAI: a reply to Ramanaiah, Franzen, and Schill. J Pers Assess. 1984;48:95–97. [35] Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. [36] Richter P, Werner J, Heerlein A, Kraus A, Sauer H. On the validity of the Beck Depression Inventory: a review. Psychopathology. 1998;31:160–168. [37] Keller F, Hoffmann M, Weithmann G. Psychometrische Eigenschaften des Beck-Depressionsinventars (BDI) bei Alkoholabh€angigen. Z Klin Psychol Psychother. 2002;31:169–177. [38] Kokkevi A, Hartgers C. EuropASI: European adaptation of a multidimensional assessment instrument for drug and alcohol dependence. Eur Addict Res. 1995;1:208–210. [39] McLellan AT, Kushner H, Metzger D, et al. The Fifth Edition of the Addiction Severity Index. J Subst Abuse Treat. 1992;9:199–213. [40] Scheurich A, Muller MJ, Wetzel H, et al. Reliability and validity of the German version of the European Addiction Severity Index (EuropASI). J Stud Alcohol. 2000;61:916–919. [41] Powell S, Rosner R. The Bosnian version of the international selfreport measure of posttraumatic stress disorder, the Posttraumatic Stress Diagnostic Scale, is reliable and valid in a variety of different adult samples affected by war. BMC Psychiatry. 2005;5. doi: 10.1186/1471-244X-5-11. [42] Streiner DL. Diagnosing tests: using and misusing diagnostic and screening tests. J Pers Assess. 2003;81:209–219. [43] Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16:981–991. [44] Franklin CL, Zimmerman M. Posttraumatic stress disorder and major depressive disorder: investigating the role of overlapping symptoms in diagnostic comorbidity. J Nerv Ment Dis. 2001;189:548–551. [45] Beck JG, Grant DM, Read JR, et al. The impact of Event Scale— Revised: psychometric properties in a sample of motor vehicle accident survivors. J Anxiety Disord. 2008;22:187–198. [46] Koffel E, Polusny MA, Arbisi PA, Erbes CR. A prelimiary investigation of the new and revised symptoms of posttraumatic stress disorder in DSM-5. Depress Anxiety. 2012;29:731–738.
Substance Abuse Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wsub20
Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients a
ab
ab
a
Laura E. Winters MS , Anne Karow MD , Jens Reimer MD , Susanne Fricke PhD , Olaf a
ab
Kuhnigk MD & Ingo Schäfer MD, MPH a
Department of Psychiatry and Psychotherapy, University Medical Center, HamburgEppendorf, Hamburg, Germany b
Center for Interdisciplinary Addiction Research, University of Hamburg, Hamburg, Germany Accepted author version posted online: 17 Mar 2014.Published online: 08 Aug 2014.
Click for updates To cite this article: Laura E. Winters MS, Anne Karow MD, Jens Reimer MD, Susanne Fricke PhD, Olaf Kuhnigk MD & Ingo Schäfer MD, MPH (2014) Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients, Substance Abuse, 35:3, 262-267, DOI: 10.1080/08897077.2014.891555 To link to this article: http://dx.doi.org/10.1080/08897077.2014.891555
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
SUBSTANCE ABUSE, 35: 262–267, 2014 Copyright Ó Taylor & Francis Group, LLC ISSN: 0889-7077 print / 1547-0164 online DOI: 10.1080/08897077.2014.891555
Psychometric Properties of the Posttraumatic Diagnostic Scale (PDS) in Alcohol-Dependent Patients Laura E. Winters, MS,1 Anne Karow, MD,1,2 Jens Reimer, MD,1,2 Susanne Fricke, PhD,1 Olaf Kuhnigk, MD,1 and Ingo Sch€afer, MD, MPH1,2 Downloaded by [Michigan State University] at 03:53 11 January 2015
ABSTRACT. Background: A high prevalence of comorbid posttraumatic stress disorder (PTSD) is found in patients with substance use disorders (SUDs). In the few existing studies, mixed results regarding the psychometric properties of common screening instruments for PTSD have been reported for patients with SUDs. No results are available for the Posttraumatic Diagnostic Scale (PDS), an established self-report measure for PTSD.Methods: The authors assessed 105 patients with alcohol dependence according to DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) (70% male) 2 weeks after their admission to an inpatient detoxification unit. Participants were administered the PDS, the PTSD module of the Structured Clinical Interview for DSM-IV (SCID), as well as measures of depression and anxiety. Patients with other substance use disorders were excluded as were patients reporting no traumatic event. Results: Internal consistencies were good to very good for the total scale (.93) and the subscales of the PDS (.82–.91). In our sample, the PDS had a high specificity (.89) but only moderate sensitivity (.57). Diagnostic agreement with the SCID was 83% (.46). The results of a receiver operating characteristic (ROC) analysis suggested that a PDS score of 8 was the optimal cutoff to screen for PTSD. The highest diagnostic agreement between PDS and SCID (89%; .60) was achieved using a cutoff score of 24. Conclusions: These findings confirm previous results suggesting that the psychometric properties of self-report measures of PTSD in patients with SUDs might differ from those in the general population. When the PDS is used in recently detoxified patients with alcohol dependence, it seems advisable to modify the cutoff score of this instrument to improve its sensitivity and diagnostic accuracy.
Keywords: Alcohol dependence, Posttraumatic Diagnostic Scale, PTSD, sensitivity, specificity
INTRODUCTION Over the past decade, the importance of co-occurring posttraumatic stress disorder (PTSD) and substance use disorder (SUD) has become increasingly apparent. Of all patients treated for SUDs, 26%–52% have a lifetime diagnosis of PTSD, and 15%– 41% currently meet PTSD criteria.1,2 SUD patients with PTSD have a more severe clinical profile compared with patients without 1
Department of Psychiatry and Psychotherapy, University Medical Center, Hamburg-Eppendorf, Hamburg, Germany 2 Center for Interdisciplinary Addiction Research, University of Hamburg, Hamburg, Germany Correspondence should be addressed to Ingo Sch€afer, MD, MPH, Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany. E-mail: i. [email protected]
PTSD. They have an earlier onset of substance abuse and more years of problematic use,3,4 they report more polydrug use,5,6 and they have greater severity of current substance use.6,7 Moreover, PTSD interferes with patients’ ability to benefit from SUD treatment. Studies in different samples of SUD patients suggest that those with PTSD have a poorer adherence to treatment and a shorter duration of abstinence,8,9 and they have consistently worse outcomes across a variety of measures.10 Despite the high prevalence of PTSD in patients with SUDs, and the obvious need for specific treatment,11 few diagnostic instruments have been evaluated for their utility in diagnosing PTSD in SUD populations. The few existing studies yielded inconsistent results regarding the psychometric properties of wellestablished measures of PTSD in patients with SUDs. Kimerling et al.12 found that the Primary Care-PTSD Scale (PC-PTSD13), a 4-item screener for PTSD, had a higher sensitivity and a specificity that was only slightly lower in patients with SUDs (.91 and
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL.
.80, respectively) as compared with a primary care sample (.78 and .87, respectively). Similar results were reported for the modified version of the PTSD Symptom Scale (PSS-SR14), a predecessor version of the instrument that is the focus of this paper. Coffey and colleagues15 found a comparable sensitivity of the PSS-SR and a slightly lower specificity in a substance abuse sample (.89 and .65, respectively) as compared with a community sample (.91 and .72, respectively16). Several other studies reported weaker psychometric properties of established measures of PTSD in patients with SUDs. Rash et al.17 compared the Impact of Events Scale—Revised (IES-R18) in patients with either alcohol dependence or drug dependence to a clinical interview (Clinician Administered PTSD Scale19). They found that the diagnostic agreement between both instruments was only moderate (.50), and that the sensitivity and specificity of the IES-R were markedly lower in their sample (.73 and .72, respectively) than had been reported for a community sample (.91 and .82, respectively20). In contrast to the proposed cutscore of 33,20 Rash et al.,17 therefore, suggested a lower cutoff score of 22 for screening purposes in patients with SUDs, resulting in a sensitivity of .92 and a specificity of .57 in their sample. Harrington and Newman21 reported similar results for the PTSD Checklist—Civilian Version (PCL-C22), and the Penn Inventory for Posttraumatic Stress Disorder.23 Although a cutoff score of 44 on the PCL-C resulted in a sensitivity of .90 and a specificity of .95 in a sample of college students,24 both parameters were markedly lower in their sample of female patients with SUDs (.76 and .79, respectively). A lower cutoff score of 38 resulted in a sensitivity of .82 and a specificity of .66 in this sample. Similarly, the Penn Inventory for Posttraumatic Stress Disorder23 had a high sensitivity and specificity (both .90) in a population consisting of veterans and civilians,23 but a much lower sensitivity and specificity in the SUD sample (.86 and .53, respectively). In contrast to the cutoff score of 35 applied by Hammarberg,23 Harrington and Newman,21 therefore, proposed a lower cutoff score of 25, resulting in a sensitivity of .82 and a specificity of .64 in their patients. The findings of these studies underline the need to evaluate the psychometric properties of established instruments for PTSD when used in patients with SUDs. The weaker diagnostic accuracy of these instruments seems to be due to both a lower sensitivity and a lower specificity. Harrington and Newman21 suggested that patients with SUDs might minimize their PTSD symptoms, making lower cutoff scores necessary to achieve an acceptable sensitivity. Another explanation could be that the degree of symptoms in the 3 symptom clusters of PTSD (i.e., intrusions, avoidance, and hyperarousal) might differ between patients with and without SUDs, and even between SUD patients with different types of substance abuse.25 A question of high clinical relevance, finally, is how accurately a diagnosis of PTSD can be made when self-report questionnaires are used among patients in detoxification treatment, given the high overlap of some PTSD symptoms with symptoms of acute withdrawal (e.g., feeling irritable or having trouble to fall asleep) and the overlap with comorbid conditions, namely, depression. Depressive symptoms are present in many patients when entering detoxification treatment and seem to remit in most of them in the course of the treatment.26 Even if PTSD symptoms remain stable over time, the changes in comorbid symptoms or the medication that might have been started could influence their assessment.
263
The Posttraumatic Diagnostic Scale27 (PDS) is one of the most widely used instruments to assess PTSD,28 but to our knowledge no findings on its psychometric properties among patients with SUDs have been published so far. The aims of our study, therefore, were to (1) assess the psychometric properties of the PDS in a sample of alcohol-dependent patients undergoing detoxification treatment and (2) examine the influence of different cutoff scores on the sensitivity and specificity of the PDS in order to make recommendations for its use in alcohol-dependent patients.
METHODS Study Sample In the current study, we examined patients who were consecutively admitted to a specialized detoxification unit at the University Medical Center Hamburg-Eppendorf (Hamburg, Germany). Inclusion criteria were a DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) diagnosis of alcohol dependence, an age between 18 and 65 years, and sufficient language abilities to complete the self-report measures. Exclusion criteria were a substance use disorder other than alcohol dependence, psychotic symptoms, and severe cognitive impairments. In the current analysis, a total of 105 individuals reporting at least 1 traumatic event according to PTSD criterion A29 were included. The interviews were conducted 10–14 days after admission by trained raters. At this point in treatment, all participants had remained abstinent from alcohol and none of them still displayed symptoms of withdrawal. All participants provided written consent after having been informed on the procedures and purpose of the study. The study was approved by the local ethics committee, the Chamber of Physicians of the State of Hamburg.
Assessment Instruments The Posttraumatic Diagnostic Scale27 (PDS) consists of 49 items and has 4 different parts. The first 2 parts of the instrument address the criterion A of PTSD using a checklist of 12 potentially traumatic events. The patient is then asked to name the worst event (index trauma) and is asked more specific yes/no questions to check if criterion A2 is fulfilled. The third part consists of 17 items assessing criteria B to D on the subscales reexperiencing (5 items), avoidance (7 items), and arousal (5 items). These items can be used to diagnose PTSD according to DSM-IV. On a 4-point Likert-scale (0 D “not at all or only one time” to 3 D “five or more times a week/ almost always”) the frequency of symptoms related to the index trauma is rated for the last month. Two additional items assess the duration of symptoms (criterion E) and a possible delayed onset. Part 4 gives a list of 8 life areas that can be impaired by PTSD symptoms (criterion F). For criteria B to D to be fulfilled, at least 1 reexperiencing symptom, 3 symptoms of avoidance, and 2 symptoms of arousal must be rated “1” or higher in the PDS. If the duration of the symptoms is at least 1 month (criterion E) and impairment is reported in at least 1 area of life (criterion F), a diagnosis of PTSD is given. Apart from the diagnosis of PTSD, the PDS provides a symptom severity score that can be obtained by summing up the 17 items assessing criteria B to D. Foa27 suggested a score of 11 to be indicative of moderate and a score of 21 to be indicative of severe PTSD. In the current study,
Downloaded by [Michigan State University] at 03:53 11 January 2015
264
SUBSTANCE ABUSE
the German version of the PDS was used.30 Studies among non– substance-abusing populations reported strong psychometric properties of the PDS. In a study among 248 patients of PTSD treatment centers and individuals from high-risk populations, Foa et al.31 found good internal consistencies of the total score (.92) and all subscales (.78–.84), as well as an agreement with the Structured Clinical Interview for DSM-IV (SCID32) of 82% (.65). The sensitivity of the scale was .89, and the specificity was .75. Correlations with anxiety and depression measures were high (Beck Depression Inventory: r D .79; State-Trait Anxiety Inventory: r D .74). Similar psychometric properties have been reported for the German version.33 The PTSD module of the Structured Clinical Interview for DSM-IV (SCID32) was used to establish diagnoses of PTSD and measure the convergent validity of the PDS in this study. The SCID assesses the 17 DSM-IV-symptoms of posttraumatic stress disorder as well as their severity. Before the interview, a trauma checklist is provided including 8 different categories of potentially traumatic events. The State-Trait Anxiety Inventory (STAI34) and the Beck Depression Inventory (BDI35) were used to assess the discriminant validity of the PDS. The STAI is an internationally used 40-item self-report measure for anxiety that yields 2 separate scores for “state” as well as “trait anxiety.” According to Spielberger and Vagg,34 the STAI shows internal consistencies between .86 and .95, and its test-retest reliability ranges from r D .75 to r D .86. The BDI is a 21-item self-report measure assessing the severity of depressive symptoms. The BDI shows an average reliability of .88 in clinical samples and a construct validity ranging from .58 to .79.36 Keller et al.37 found a good internal consistency of .90 for the BDI in a sample of alcohol-dependent inpatients. The European Addiction Severity Index (EuropASI38) was used to assess substance-related problems. The EuropASI has been developed on the basis of the fifth version of the American Addiction Severity Index.39 Ten composite scores give an insight into important areas such as drug and alcohol use, social relationships, medical and psychiatric conditions, as well as the legal, employment, and economical situation of the participants. Composite scores range between 0 (“no treatment required”) and 1 (“treatment extremely required”). Chronicity of alcohol use was estimated from the participants’ reports of years of lifetime problematic alcohol use, which was defined as using at least 3 times per week or bingeing on at least 2 consecutive days per week. Scheurich et al40 found good psychometric properties for the German version of the EuropASI.
Data Analysis Chi-square tests and independent-sample t tests were used to assess group differences in sociodemographic and clinical variables. The internal reliability of the PDS was assessed by calculating Cronbach’s alpha for the subscales and the total severity score of the PDS. The agreement with the SCID was tested using the chance corrected classification agreement kappa. An independentsample t test was used to compare the PDS severity scores of participants with and without PTSD. To determine the discriminant validity of the PDS, Spearman correlations with relevant psychiatric constructs (depression and anxiety) were calculated. Finally, a receiver operating characteristic (ROC) analysis was used to compare different cutoff scores of the PDS and to determine the
resulting sensitivity and specificity. All data were analyzed using SPSS 17 for Windows (SPSS Inc., Chicago, IL, USA).
RESULTS Participant Characteristics Of the 105 participants included in the analysis, 30% were female. Participants’ average age was M D 41.3 years (SD D 9.6; range D 19–64). Almost half of the participants (45%) were employed, 63% had completed at least 10 years of school, and only 4% had left school without any qualification. Seventeen percent were married, and 29% lived in a stable relationship. The average age at onset of problematic alcohol use was M D 24.0 years (SD D 10.9; range D 5–53). The average duration of chronic alcohol abuse according to the EuropASI was M D 13.0 years (SD D 9.9; range D 0–37), and the participants reported an average of M D 2.6 previous detoxifications (SD D 3.3; range D 0–18). The results of the EuropASI composite scores can be seen in Table 1. Three quarters of the participants reported having experienced more than 1 category of traumatic events, with an average of M D 3.1 categories. The most frequent index traumatic events were nonsexual assault by known assailants (19%), nonsexual assault by unknown assailants (18%), and accidents (18%). According to the SCID, n D 21 participants (20%; n D 9 females and n D 12 males) were diagnosed with posttraumatic stress disorder (PTSD). Patients diagnosed with PTSD most frequently reported sexual assault by a known assailant as their worst experience (33%), and they also reported having experienced a higher total number of different traumatic events (M D 4.0).
Reliability and Validity of the PDS The internal reliability (Cronbach’s alpha) for the PDS total scale was .93 for the whole sample and .90 for the patients with a SCID diagnosis of PTSD. The internal reliabilities of the subscales were .91 for the reexperiencing scale, .82 for the avoidance scale, and .85 for the hyperarousal scale (for the patients with PTSD .87, .74, and .71, respectively). The subscales and the total PDS were TABLE 1 Means and Standard Deviations of the European Addiction Severity Index (EuropASI) Composite Scores ASI composite score
Mean
Standard deviation
Drug Legal Family Social relationships Employment Medical condition Psychiatric status Economical situation Alcohol
.02 .06 .18 .11 .25 .34 .30 .63 .53
.05 .13 .22 .16 .31 .32 .21 .38 .14
Note. N D 105. Means ranging between 0 ( D “no treatment required“) and 1 ( D “treatment extremely required”).
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL.
highly correlated among each other in the whole sample (see Table 2). As a first step to assess the validity of the PDS, we compared the PDS severity scores between patients diagnosed with PTSD in the SCID interview and patients without PTSD. The patients diagnosed with PTSD (n D 21) reached a significantly higher symptom severity as assessed by the PDS total score than participants without PTSD (n D 84; M D 22.19 vs. 5.39; t(23.9) D ¡6.03; P < .001). As for the total score, significant differences were found for all subscales of the PDS (subscale reexperiencing: M D 6.24 vs. 1.32; t(23.22) D ¡4.61; P < .001; subscale avoidance: M D 8.00 vs. 1.92; t(23.63) D ¡5.08; P < .001; subscale arousal: M D 8.05 vs. 2.16; t(103) D ¡7.16; P < .001). Convergent validity was obtained by comparing the diagnoses obtained by the PDS with those of the SCID interview. Of all participants, n D 87 (83%) were correctly classified by the PDS (.46). The sensitivity was .57, the specificity .89. Of the participants without PTSD, n D 75 out of n D 84 were correctly classified (89%), whereas only n D 12 out of n D 21 participants with PTSD were correctly classified (57%). Discriminant validity was assessed through Spearman correlations between the PDS and measures of depression and anxiety. Correlations with the BDI were small to moderate for the PDS total scale (r D .30; P D .01), the avoidance scale (r D .23; P D .05), and the hyperarousal scale (r D .31; P D .01), and not significant for the reexperiencing subscale (see Table 2). Correlations of the PDS with the 2 scales of the STAI were not significant, except for a small correlation of trait anxiety with the hyperarousal scale of the PDS (r D .24; P D .04).
265
TABLE 3 Effectiveness Indices for Selected Cutoff Scores of the Posttraumatic Stress Diagnostic Scale (PDS) Symptom Severity Score Cutoff score 7 8 9 10 11 12 18 20 21 22 23 24 25
Sensitivity
Specificity
Diagnostic accuracy
Kappa
.905 .905 .857 .857 .810 .762 .667 .619 .619 .619 .619 .571 .429
.690 .726 .786 .798 .810 .821 .893 .917 .917 .929 .929 .964 .976
73% 84% 80% 81% 81% 81% 85% 86% 86% 87% 87% 89% 87%
.42 .51 .51 .52 .51 .50 .54 .56 .55 .57 .57 .60 .49
Note. Bold values indicate the recommended cutoff value for use as a screening tool8 and for best agreement with the SCID.24
is 8, achieving maximal specificity (.73) while meeting a sensitivity of over .90, which has been proposed as an a priori goal for sensitivity.17 Using this cutoff value, 84% of the sample were correctly classified (.51). The optimal agreement with the SCID was obtained using a cutoff value of 24 (sensitivity D .57; specificity D .96).
DISCUSSION Determining Diagnostic Values The PDS not only indicates if each of the DSM-IV criteria for PTSD is met, but also provides a dimensional symptom severity score. The proposed cutoff values are 11 for mild to moderate symptoms, 21 for moderate to severe symptoms, and 36 for severe symptoms.27 In a ROC analysis, we compared different cutoff scores with regard to their sensitivity, specificity, and diagnostic accuracy (see Table 3). In order to use the PDS as a screening tool, our analysis suggests that the most suitable cutoff score TABLE 2 Intercorrelations Between PDS Subscales and Correlations With Measures of Anxiety and Depression PDS Total PDS Total Reexperiencing Avoidance Arousal STAI State Trait BDI
Reexperiencing
Avoidance
— .86** .91** .93**
— .82** .70**
— .76**
.14 (ns) .14 (ns) .30**
.11 (ns) .08 (ns) .20 (ns)
.12 (ns) .11 (ns) .23*
Arousal
.19 (ns) .24** .31**
Note. PDS D Posttraumatic Diagnostic Scale; STAI D State-Trait Anxiety Inventory; BDI D Beck Depression Inventory. *P < .05; **P < .01; ns D nonsignificant.
Summary and Discussion of the Results The results of our study suggest that the Posttraumatic Diagnostic Scale is a reliable instrument in patients with alcohol dependence. The internal consistencies found in this study were high and correspond well with published results for the PDS in community samples.31,33,41 Moreover, the diagnostic agreement between the PDS and the SCID was comparable to the findings of Foa et al.31 Although the specificity of the PDS in our sample exceeded the specificity reported in both of these studies, the sensitivity fell into a moderate range. This is in line with the findings of Rash et al.17 and Coffey et al.,15 who suggested that self-report measures of PTSD may have differing psychometric properties in substance abuse samples. A possible explanation for the low sensitivity of the PDS in our study is that for a diagnosis of PTSD, enough symptoms in all clusters according to DSM-IV have to be present. Substance abuse and detoxification might dampen some of the PTSD symptoms or influence their perception,21 leading to a lower sensitivity in this group of patients. As all patients included in our study had only recently undergone detoxification treatment, the use of alcohol in the weeks before admission may have dampened some of their symptoms, leading to a lower sensitivity of the PDS. Our results cannot be explained by Streiner’s observation42 that a lower base rate of a disease in a population usually increases the amount of false positives in a sample, as the base rate in this study was lower than those reported by Foa et al.31 and Griesel et al.33 Moreover, as Brenner and Gefeller43 argue, base rate is only one of the sample characteristics influencing sensitivity and specificity of a test.
Downloaded by [Michigan State University] at 03:53 11 January 2015
266
SUBSTANCE ABUSE
Further indicators of differing psychometric properties of the PDS in substance abuse samples are the low correlations of the PDS with measures of anxiety and the small to moderate correlation with depression in our study. In community samples, these correlations are expected to be moderate to high, as anxiety and depression are overlapping constructs but not identical with PTSD.44 In contrast, studies among patients with substance abuse reported lower correlations between measures of PTSD and depression.17,45 Interestingly, the finding by Foa et al,31 that the subscales avoidance and arousal tend to correlate higher with depression than the reexperiencing scale, is supported by our findings. Given the low sensitivity, using the PDS with conventional DSM-IV scoring criteria to screen for PTSD is not recommended. The results of our ROC analysis suggest that a cutoff value of 8 can be recommended to enhance the sensitivity of the PDS in substance abuse settings when it is used as a screening tool. This score is close to the score of 11 suggested by Foa27 as a cutoff for moderate PTSD symptom severity. When a cutoff score of 8 was used, the PDS identified over 90% of the PTSD cases in our sample at the cost of more false-positive cases (specificity .726). As shorter measures appear to be similarly accurate,12 the value of using the PDS as a screening tool in clinical settings could be questioned. On the other hand, the PDS has the advantage to allow both a dimensional assessment and a clinical diagnosis and is therefore widely used in clinical as well as in research settings. The highest diagnostic agreement between the PDS and the SCID could be reached using the cutoff score of 24. This score is close to Foa’s27 suggestion for a cutoff score between moderate and severe symptomatology. Used with the cutoff score of 24, the PDS correctly classified almost all cases of PTSD (specificity .964) at the expense of false-negative cases (sensitivity .571). In clinical settings, the cutoff score should be chosen depending on the consequences of possible false-positive or false-negative ratings and the primary intention (ie, broader screening vs exact diagnosis). When the arguments mentioned above are weighed against each other, it seems advisable to use the PDS in patients with substance abuse as a more specific measure to validate the diagnosis of PTSD after using a brief screener.
Another potential limitation is related to the timing of assessment in our study, where we examined patients 2 weeks after termination of active use. Although PTSD symptoms can decrease or increase during detoxification,25 Coffey et al.16 stated that major changes in symptoms should be completed within 2 weeks after termination of active use. Nevertheless, it remains difficult to determine the exact effects of withdrawal or comorbid psychopathology on self-rating instruments, and the findings cannot be generalized to patients who have been abstinent for a longer period of time or actively using patients. Symptoms of PTSD should therefore be assessed repeatedly in the course of treatment to enhance the diagnostic validity. Moreover, the restriction of our sample to patients with alcohol dependence limits the generalizability of our findings to patients with other types of substance abuse. From a clinical perspective, however, alcohol-dependent patients entering detoxification treatment represent an important subgroup, which is why we focused on patients in this setting. For methodological reasons, patients reporting no traumatic event had to be excluded from the study. Although it could be argued that this further decreases the generalizability of our findings, this doesn’t seem to be the case, as the prevalence of traumatic events is extremely high in substance abuse samples (e.g., Rash et al.17: 89%–94%). Due to the short duration of stay of patients in our hospital, it was not possible to determine the test-retest reliability of the PDS. This question should be addressed by further studies together with the psychometric properties of the instrument in alcohol-dependent patients after a longer period of abstinence and in patients with other types of substance abuse.
AUTHOR CONTRIBUTIONS IS designed the study. All analyses were performed by LW and SF. The first draft of the manuscript was written by LW and IS. All authors commented on and approved the final manuscript.
REFERENCES Limitations and Future Directions For the interpretation of our findings, some important limitations have to be taken into account. First of all, as the PDS has been tailored to assess PTSD according to DSM-IV criteria, it is an important question how our findings should be interpreted in the context of the new DSM-5 diagnosis of PTSD, where some symptoms of the DSM-IV cluster C have been combined with additional symptoms to form a new cluster D (“negative alterations in cognitions and mood”), and the former cluster D has been transferred into a new cluster E (“alterations in arousal and reactivity”). In our study, the PDS scores were significantly correlated with depression as measured by the BDI. If symptoms of depression are considered a proxy of the new DSM-5 cluster “negative alterations in cognitions and mood,” the scale might still be of use for identifying patients when the new diagnostic criteria are applied. A recent paper on the relationships between the new symptom clusters in DSM-5 and the DSM-IV diagnosis of PTSD also highlighted the close relationships between both constructs.46
[1] Jacobsen LK, Southwick SM, Kosten TR. Substance use disorders in patients with posttraumatic stress disorder: a review of the literature. Am J Psychiatry. 2001;158:1184–1190. [2] Sch€afer I, Najavits LM. Clinical challenges in the treatment of patients with posttraumatic stress disorder and substance abuse. Curr Opin Psychiatry. 2007;20:614–618. [3] Johnson SD, Striley C, Cottler LB. The association of substance use disorders with trauma exposure and PTSD among African American drug users. Addict Behav. 2006;31:2063–2073. [4] Read JP, Brown PJ, Kahler CW. Substance use and posttraumatic stress disorders: symptom interplay and effects on outcome. Addict Behav. 2004;29:1665–1672. [5] Dragan M, Lis-Turlejska M. Prevalence of posttraumatic stress disorder in alcohol dependent patients in Poland. Addict Behav. 2007;32:902–911. [6] Mills KL, Lynskey M, Teesson M, Ross J, Darke S. Post-traumatic stress disorder among people with heroin dependence in the Australian treatment outcome study (ATOS): prevalence and correlates. Drug Alcohol Depend. 2005;77:243–249.
Downloaded by [Michigan State University] at 03:53 11 January 2015
WINTERS ET AL. [7] Clark HW, Masson CL, Delucchi KL, Hall SM, Sees KL. Violent traumatic events and drug abuse severity. J Subst Abuse Treat. 2001;20:121–127. [8] Brown PJ, Read JP, Kahler CW. Comorbid posttraumatic stress disorder and substance use disorders: treatment outcomes and the role of coping. In: Ouimette P, Read JP, eds. Trauma and Substance Abuse. Causes, Consequences and Treatment of Comorbid Disorders. Washington DC: American Psychological Association; 2003:171–188. [9] Hien DA, Nunes E, Levin FR, Fraser D. Posttraumatic stress disorder and short-term outcome in early methadone treatment. J Subst Abuse Treat. 2000;19:31–37. [10] Najavits LM, Harned MS, Gallop RJ, et al. Six-month treatment outcomes of cocaine-dependent patients with and without PTSD in a multisite national trial. J Stud Alcohol Drugs. 2007;68:353–361. [11] Henslee AM, Coffey SF. Exposure therapy for posttraumatic stress disorder in a residential substance use treatment facility. Prof Psychol Res Pr. 2010;41:34–40. [12] Kimerling R, Trafton JA, Nguyen B. Validation of a brief screen for post-traumatic stress disorder with substance use disorder patients. Addict Behav. 2006;31:2074–2079. [13] Prins A, Ouimette PC, Kimerling R, Cameron RP, Hugelshofer DS, Shaw-Hegwer J. The primary care PTSD screen: development and operating characteristics. Primary Care Psychiatry. 2004;9:9–14. [14] Foa EB, Riggs DS, Dancu CV, Rothbaum BO. Reliability and validity of a brief instrument for assessing posttraumatic stress disorder. J Trauma Stress. 1993;6:459–473. [15] Coffey SF, Dansky BS, Falsetti SA, Saladin ME, Brady KT. Screening for PTSD in a substance abuse sample: psychometric properties of a modified version of the PTSD Symptom Scale Self-Report. Posttraumatic stress disorder. J Trauma Stress. 1998;11:393–399. [16] Coffey SF, Gudmundsdottir B, Beck JG, Palyo SA, Miller L. Screening for PTSD in motor vehicle accident survivors using the PSS-SR and IES. J Trauma Stress. 2006;19:119–128. [17] Rash CJ, Coffey SF, Baschnagel JS, Drobes DJ, Saladin ME. Psychometric properties of the IES-R in traumatized substance dependent individuals with and without PTSD. Addict Behav. 2008;33:1039– 1047. [18] Weiss D, Marmar C. The Impact of Event Scale—Revised. In: Wilson JP, Keane TM, eds. Assessing Psychological Trauma and PTSD: A Handbook for Practitioners. New York: Guilford Press; 1997:399–411. [19] Blake DD, Weathers FW, Nagy LM, et al. The development of the Clinical-Administered PTSD Scale. J Trauma Stress. 1995;8:75–90. [20] Creamer M, Bell R, Failla S. Psychometric properties of the Impact of Event Scale—Revised. Behav Res Ther. 2003;41:1489–1496. [21] Harrington T, Newman E. The psychometric utility of two self-report measures of PTSD among women substance users. Addict Behav. 2007;32:2788–2798. [22] Weathers FW, Litz BT, Huska JA, Keane TM, eds. PTSD Checklist—Civilian Version. Boston: Boston National Center for PTSD, Behavioral Science Division; 1994. [23] Hammarberg M. Penn Inventory for posttraumatic stress disorder: psychometric properties. Psychol Assess. 1992;4:204–216. [24] Ruggiero KJ, Rheingold AA, Resnick HS, Kilpatrick DG, Galea S. Comparison of two widely used PTSD-screening instruments: implications for public mental health planning. J Trauma Stress. 2006;19:699–707. [25] Saladin ME, Brady KT, Dansky BS, Kilpatrick DG. Understanding comorbidity between PTSD and substance use disorders: two preliminary investigations. Addict Behav. 1995;20:643–655.
267
[26] Davidson KM. Diagnosis of depression in alcohol dependence: changes in prevalence with drinking status. Br J Psychiatry. 1995;166:199–204. [27] Foa EB, ed. Posttraumatic Stress Diagnostic Scale: Manual. Minneapolis, MN: National Computer Systems; 1995. [28] Elhai JD, Gray MJ, Kashdan TB, Franklin CL. Which instruments are most commonly used to assess traumatic event exposure and posttraumatic effects? A survey of traumatic stress professionals. J Trauma Stress. 2005;18:541–545. [29] American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington DC: American Psychiatric Association; 1994. € [30] Ehlers A, Steil R, Winter H, Foa EB, eds. Deutsche Ubersetzung der Posttraumatic Diagnostic Scale (PDS). Oxford, UK: Oxford University, Warneford Hospital, Department of Psychiatry; 1996. [31] Foa EB, Cashman L, Jaycox L, Perry K. The validation of a selfreport measure of posttraumatic stress disorder: the Posttraumatic Diagnostic Scale. Psychol Assess. 1997;9:445–451. [32] First RL, Spitzer M, Gibbon J, eds. Structured Clinical Interview for DSM-IV Axis I Disorders—Patient Edition (SCID-I/P, Version 2.0). New York: Biometrics Research Department, New York State Psychiatric Institute; 1996. [33] Griesel D, Wessa M, Flor H. Psychometric qualities of the German version of the Posttraumatic Diagnostic Scale (PTDS). Psychol Assess. 2006;18:262–268. [34] Spielberger CD, Vagg PR. Psychometric properties of the STAI: a reply to Ramanaiah, Franzen, and Schill. J Pers Assess. 1984;48:95–97. [35] Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. [36] Richter P, Werner J, Heerlein A, Kraus A, Sauer H. On the validity of the Beck Depression Inventory: a review. Psychopathology. 1998;31:160–168. [37] Keller F, Hoffmann M, Weithmann G. Psychometrische Eigenschaften des Beck-Depressionsinventars (BDI) bei Alkoholabh€angigen. Z Klin Psychol Psychother. 2002;31:169–177. [38] Kokkevi A, Hartgers C. EuropASI: European adaptation of a multidimensional assessment instrument for drug and alcohol dependence. Eur Addict Res. 1995;1:208–210. [39] McLellan AT, Kushner H, Metzger D, et al. The Fifth Edition of the Addiction Severity Index. J Subst Abuse Treat. 1992;9:199–213. [40] Scheurich A, Muller MJ, Wetzel H, et al. Reliability and validity of the German version of the European Addiction Severity Index (EuropASI). J Stud Alcohol. 2000;61:916–919. [41] Powell S, Rosner R. The Bosnian version of the international selfreport measure of posttraumatic stress disorder, the Posttraumatic Stress Diagnostic Scale, is reliable and valid in a variety of different adult samples affected by war. BMC Psychiatry. 2005;5. doi: 10.1186/1471-244X-5-11. [42] Streiner DL. Diagnosing tests: using and misusing diagnostic and screening tests. J Pers Assess. 2003;81:209–219. [43] Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16:981–991. [44] Franklin CL, Zimmerman M. Posttraumatic stress disorder and major depressive disorder: investigating the role of overlapping symptoms in diagnostic comorbidity. J Nerv Ment Dis. 2001;189:548–551. [45] Beck JG, Grant DM, Read JR, et al. The impact of Event Scale— Revised: psychometric properties in a sample of motor vehicle accident survivors. J Anxiety Disord. 2008;22:187–198. [46] Koffel E, Polusny MA, Arbisi PA, Erbes CR. A prelimiary investigation of the new and revised symptoms of posttraumatic stress disorder in DSM-5. Depress Anxiety. 2012;29:731–738.
