This article was downloaded by: [New York University] On: 19 May 2015, At: 07:39 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Archives of Suicide Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/usui20
The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings a
b
Edwin D. Boudreaux PhD , Michelle L. Jaques BA , Kaitlyn M. Brady c
b
d
MS , Adam Matson BS & Michael H. Allen MD a
Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, Massachusetts, USA b
Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA c
Click for updates
Worcester Polytechnic Institute, Worcester, Massachusetts, USA
d
Colorado Depression Center, University of Colorado School of Medicine and the VISN 19 Mental Illness Research Education and Clinical Center, Denver, Colorado, USA Accepted author version posted online: 31 Mar 2015.
To cite this article: Edwin D. Boudreaux PhD, Michelle L. Jaques BA, Kaitlyn M. Brady MS, Adam Matson BS & Michael H. Allen MD (2015) The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings, Archives of Suicide Research, 19:2, 151-160, DOI: 10.1080/13811118.2015.1034604 To link to this article: http://dx.doi.org/10.1080/13811118.2015.1034604
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [New York University] at 07:39 19 May 2015
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Archives of Suicide Research, 19:151–160, 2015 Copyright # International Academy for Suicide Research ISSN: 1381-1118 print=1543-6136 online DOI: 10.1080/13811118.2015.1034604
Downloaded by [New York University] at 07:39 19 May 2015
The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings Edwin D. Boudreaux, Michelle L. Jaques, Kaitlyn M. Brady, Adam Matson, and Michael H. Allen This study evaluated the concurrent validity of a brief suicide risk screener for adults in the emergency department (ED). Two versions of the verbally administered Patient Safety Screener (2-item, 3-item) were compared to a reference standard, the Beck Scale for Suicide Ideation (BSSI). Analyses included measures of agreement (Kappa). Agreement between the Patient Safety Screener-2 and -3 and the BSSI (n ¼ 951) was almost perfect for overall positive screening (K ¼ 0.94–0.95) and past suicide attempts (K ¼ 0.97–0.98). Agreement on ideation ranged from fair (K ¼ 0.34) for the 2-item version to good (K ¼ 0.61) for the 3-item version. The Patient Safety Screener’s concurrent validity with the BSSI ranged from fair to almost perfect and warrants additional study. Keywords
emergency medicine, emergency psychiatry, suicide screening and assessment
INTRODUCTION
et al., 2006; Claassen and Larkin, 2005; Ilgen, Walton, Cunningham et al., 2009). Similarly, while diagnostic coding data suggest only 0.4% of ED visits are directly related to intentional self-injury (Ting, Sullivan, Boudreaux et al., 2012), studies administering standardized screeners have found 12–13% (Allen, Abar, McCormick et al., 2013; Boudreaux, Cagande, Kilgannon et al., 2006) of ED patients have had at least one suicide attempt in their lifetime. The discrepancy between low detection of suicide risk within treatment as usual samples and the high prevalence of actual suicide risk within research samples is likely due to the fact that most ED clinicians only
Structured chart reviews of emergency department (ED) medical records and diagnostic coding data reveal that less than 3% of emergency department (ED) patients are identified by ED clinicians during usual care as having suicidal ideation or behavior (Caterino, Sullivan, Betz et al., 2013; Ting, Sullivan, Boudreaux et al., 2012). However, standardized screening studies using verbally administered research interviews or paper-and-pencil instruments have found that the point prevalence of active suicidal ideation among ED can reach as high as 11% (Boudreaux, Cagande, Kilgannon
151
Downloaded by [New York University] at 07:39 19 May 2015
Validation of the Patient Safety Screener
screen patients presenting with a psychiatric chief complaint (Betz, Sullivan, Manton et al., 2013). This is referred to as clinically indicated screening. Clinically indicated screening misses many people with active suicidal ideation or a past suicide attempt (Allen, Abar, McCormick et al., 2013, Claassen and Larkin, 2005). By contrast, universal screening may help to identify suicide risk among patients presenting for nonpsychiatric symptoms, sometimes called occult or incidental detection. However, two elementary problems stymie implementation of universal screening in clinical practice in the ED: there are simply no good-fit, well-validated screeners for use with general adult patients, and the impact of screening on improving suicide related outcomes has not been studied (D’Onofrio, Jauch, Jagoda et al., 2010; Randall, Colman, & Rowe, 2011; Ronquillo, Minassian, Vilke et al. 2012; U.S. Department of Health and Human Services [HHS], 2012). A solution for primary suicide risk detection in the ED will need to be simple, efficient, and calibrated to easily separate individuals who require further risk assessment from those who do not. Traditional suicide risk measures, such as the Beck Scale for Suicide Ideation (BSSI) (Beck & Steer, 1991) and the Columbia Suicide Rating Scale (CSSRS) (Posner, Brown, Stanley et al., 2011), are too lengthy and complex to be administered universally by ED staff. In contrast, the Patient Safety Screener-5 (PSS-5) (Allen, Abar, McCormick et al., 2013) was developed as a 5-item primary detection screener for adults in the ED setting. In a sample of 1,068 patients presenting with non-psychiatric medical complaints recruited from six EDs, 2.3% screened positive on active suicidal ideation within the past two weeks, 12.8% reported having a lifetime history of a suicide attempt, and 1.1% reported both active ideation and a past attempt. Unfortunately, no psychometric analyses have been reported
152
on the PSS-5, and it was never validated against a reference standard instrument or post-visit suicidal behavior. Consequently, important test characteristics, such as the PSS-50 s sensitivity, specificity, and false negative rates, have not yet been established. While the PSS-5 was designed to be brief, even five items may suffer from low rates of adoption in busy clinical settings. In general, the efficiency principle holds true: the shorter the screener, the more likely it is to be accepted by clinicians (Glasgow, Kaplan, Ockene et al., 2012). Efficiency is particularly important when considering the ever-expanding list of behaviors and conditions for which clinicians are expected to screen. Screening for suicide risk is not likely to occur in a vacuum, but, instead, as part of a larger battery of preventive and behavioral health screenings. Consequently, the field needs an ultra-brief primary suicide risk screener that can be applied universally by ED staff. It should efficiently identify individuals who need further risk assessment and easily integrate with the larger clinical assessment or preventive health screening battery. This study aimed to validate two shorter versions of the PSS-5 (Allen, Abar, McCormick et al., 2013) (see Figure 1) against the BSSI as a reference standard. The original PSS-5 (Allen, Abar, McCormick et al., 2013) consisted of two items from the PHQ-9 (Kroenke & Spitzer, 2002) assessing depressed mood and anhedonia and three items modeled after the CSSRS (Posner, Brown, Stanley et al. 2011) assessing passive suicidal ideation, active suicidal ideation, and lifetime suicide attempt. In addition, any patient reporting a lifetime attempt is asked to indicate the timing of the most recent attempt (Today, Within 30 days, Within 1–6 months, Greater than 6 months). Allen, Abar, McCormick et al. (2013) found that the items assessing anhedonia and passive ideation did not identify significantly new
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
FIGURE 1. Patient Safety Screener.
cases when compared to the combination of the other three items (depression, active ideation, lifetime attempts). Consequently, to create the PSS-3, the anhedonia and passive ideation items were removed. The depression item was retained in the PSS-3 and is conceptualized as a segue that may help transition from the screener introduction to the potentially sensitive suicide screening items. To create the PSS-2, the depression item was also removed, leaving only active ideation and lifetime attempt. METHODS Participants
At each of three different EDs belonging to the University of Massachusetts Memorial Healthcare System, 15 days were randomly selected between June and September of 2011. Shifts ran between 9:00 a.m. and 9:00 p.m., and all days of the week were represented. All patients 18
years old were logged and considered for approach, regardless of the reason for their visit. Individuals were excluded if they had a medical, emotional, cognitive, or behavioral condition that prevented them from engaging in conversation through the duration of their visit, such as persistent altered mental status, intractable pain or vomiting, or intubation. The age, race, gender, and insurance status of all patients logged were recorded. The average age of the sample was 44.6 (SD ¼ 18), with 52% female, 12% Latino, 88% white, and 8% African American. After providing a complete description of the study to the participants, written informed consent was obtained. Institutional Review Board approval was also obtained by the University of Massachusetts Medical School IRB. Materials
The Patient Safety Screener (PSS). For both versions of the PSS (described in the
ARCHIVES OF SUICIDE RESEARCH
153
Validation of the Patient Safety Screener
Downloaded by [New York University] at 07:39 19 May 2015
Introduction), a positive screen was identified as a positive response for active ideation within the past two weeks OR a lifetime history of a suicide attempt (PSS Overall Positive). The Beck Scale for Suicide Ideation (BSSI). The BSSI is a well-validated, widely used measure of suicide risk. It is comprised of 21 items that may be used to identify the presence and severity of suicidal ideation and past attempts. Each item requires the individual to read three statements and indicate which best describes him=her. The first five items are used to screen for suicidal ideation and includes three items measuring passive ideation (wish to live, wish to die, reasons for living) and two items measuring active ideation (desire to kill oneself, whether one would save one’s life in a life-threatening situation). The remaining 14 ideation items are only asked if the individual screens positive on one of the two active ideation items. In addition, the BSSI includes an item assessing past suicide attempts, which is asked of everyone, and, if positive, a subsequent item inquiring at what level the respondent wished to die during the last attempt. A positive screen for the BSSI was defined as a positive response on one or both of the two items assessing active ideation OR a lifetime history of a past attempt (BSSI Overall Positive). Procedure
Trained research staff evaluated each consecutively presenting patient’s ED visit record, approached those without obvious exclusion criteria, and performed an eligibility interview. Research staff consulted with ED clinicians regarding eligibility, as needed. Those who were eligible were offered the opportunity to participate. Because maximizing response rates was critical and the ED is a time sensitive setting, the assessment was kept brief, and only the data most critical to answering
154
the research question were collected. Those who consented were randomized to one of four groups based on the version of the PSS administered (PSS-2 vs. PSS-3) and the order of the presentation of the instruments (PSS before the BSSI vs. PSS after the BSSI). This design allowed the study to validate both versions while controlling for potential ordering effects. If patients scored positive for active suicidal ideation on either the PSS or the BSSI, the patient’s treating physician or nurse was informed by the research assistant, and the patient was managed based on usual and customary care at the site. The study was approved by the University of Massachusetts Medical School Institutional Review Board. Data Analyses
Descriptive statistics were computed, including the rates on the PSS Overall Positive and the BSSI Overall Positive, as well as the subdomains of active ideation and lifetime attempt. Kappa, a measure of agreement beyond chance, was computed between each of the two PSS versions and the BSSI. To further validate the PSS, the total BSSI scores were compared between those who screened positive and those who screened negative using a Wilcoxon Rank Sum test. Ordering effects were assessed by examining whether the proportion of positive screens was related to the order of presentation (i.e., PSS first or second). Finally, site effects were assessed by examining whether the proportion of positive screens differed based on enrollment site. All statistics were computed at individual 95% confidence levels. SPSS 20 (Chicago IL) was used. RESULTS Participant Enrollment
There were 2,258 adult individuals who presented during the research shifts and
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
were considered for participation; 1,307 were either excluded or declined participation, leaving 951 (42%) who were enrolled into the study. Of those enrolled, 438 were enrolled at University, 411 at Memorial, and 102 at Marlborough. Four-hundred-fifty-nine (n ¼ 459) were randomized to the PSS-2, while 492 were randomized to the PSS-3. Compared to patients who were not enrolled, participants were more likely to be younger (not enrolled: 47 years old vs. enrolled 45 years old, p < 0.05) and female (not enrolled: 51% vs. enrolled: 58%, p < 0.05). There were no differences observed between those enrolled and those not enrolled by race or insurance (both p > 0.05). Prevalence Rates
Table 1 reports the descriptive statistics for both versions of the PSS and the BSSI. Overall, 145 (15.2%) screened positive on at least one of the screeners. Two Chi-square analyses, one computed on each PSS version, demonstrated that the proportion of patients scoring positive
on the PSS did not differ based on the order of presentation (i.e., whether it came before or after the BSSI) (both p > 0.10). Consequently, for all analyses, all participants within each of the two PSS groups were combined regardless of the order of completion. Of the 459 participants who completed the PSS-2, 60 (13.1%) screened positive (Table 2). Of the 492 who took the PSS-3, 82 (16.7%) screened positive. The difference in proportions of positive screens between the two PSS versions was not statistically different, X2 (1, n ¼ 951) ¼ 2.4, p > 0.05. Additionally, sites did not differ significantly from one another on the proportion of positive screens on either version of the PSS or the BSSI (all p > 0.05). Of the 134 who reported a lifetime attempt on either the PSS-2 or PSS-3, 11 (8.2%) reported the most recent attempt occurred within 6 months of the ED visit. Agreement between the Screeners and BSSI
Table 3 reports the Kappa statistics, with 95% Confidence Intervals. Both the
TABLE 1. Descriptive Statistics for the Patient Safety Screener and Beck Scale for Suicide Ideation N (%) 2-item PSS (n ¼ 459)
3-item PSS (n ¼ 492)
PSS-2 Overall positive PSS-2 Active ideation PSS-2 Lifetime attempt BSSI Overall positive BSSI Ideation BSSI Lifetime attempt PSS Overall positive PSS Active ideation PSS Lifetime attempt BSSI Overall positive BSSI Active ideation BSSI Lifetime attempt
60 8 59 60 10 57 82 20 75 76 19 73
(13.1) (1.7) (12.9) (13.1) (2.2) (12.4) (16.7) (4.1) (15.2) (15.4) (3.9) (14.8)
95% C.I. 0.1–0.162 0.006–0.029 0.098–0.159 0.1–0.162 0.009–0.035 0.094–0.154 0.134–0.2 0.023–0.058 0.121–0.184 0.123–0.186 0.022–0.056 0.117–0.18
Note. Of the 134 who reported a lifetime attempt on either the PSS-2 or PSS-3, 3 (2.2%) reported the attempt was within the past 24 hours, 1 (0.7%) within the last month, 7 (5.2%) between 1 and 6 months ago, and 123 (91.7%) longer than 6 months ago.
ARCHIVES OF SUICIDE RESEARCH
155
Validation of the Patient Safety Screener
TABLE 2. 2 2 Tables for 2- and 3-Item Patient Safety Screener and the Beck Scale for Suicide Ideation PSS-2 BSSI
PSS-2 Negative
Downloaded by [New York University] at 07:39 19 May 2015
PSS-3
Positive
Total
3 57 60
399 60 459
4 6 10
451 8 459
0 57 57
400
Overall positive Negative 396 Positive 3 Total 399 Active ideation Negative 447 Positive 2 Total 449 Lifetime attempt Negative 400 Positive 2 Total 402
BSSI
PSS-3
459
PSS-2 and PSS-3 exhibited strong overall agreement with the BSSI, each with a Kappa of 0.95 (Landis & Koch, 1997). Similarly, agreement for the subdomain of lifetime suicide attempt was strong, with Kappas of 0.98 and 0.97, respectively. However, agreement for the subdomain of active ideation was weaker, with Kappas of 0.34 and 0.61, respectively. To determine if the magnitude of the agreement with the BSSI differed between
Negative Overall positive Negative 410 Positive 6 Total 416 Active ideation Negative 466 Positive 7 Total 473 Lifetime attempt Negative 416 Positive 3 Total 419
Positive
Total
0 76 76
410 82 492
6 13 19
472 20 492
1 72 73
417 75 492
the PSS-2 and PSS-3, the difference and the associated 95% confidence intervals between the Kappas for the PSS Overall Positive and the two subdomains (active ideation, lifetime attempt) were examined. The 95% confidence intervals for the difference in Kappas between the PSS-2 and the PSS-3 (Kappa (PSS-2, BSSI) – Kappa (PSS-3, BSSI) ¼ Kappa difference) crossed zero for both Overall Positive and lifetime attempt, suggesting they did not differ
TABLE 3. Agreement Between the 2-Item and 3-Item Patient Safety Screener with the Beck Scale for Suicide Ideation Screener assessment 2-Item Patient Safety Screener Overall positive Active ideation Lifetime attempt 3-Item Patient Safety Screener Overall positive Active ideation Lifetime attempt
156
Kappa
S.E.
C.I. lower
C.I. upper
0.94 0.34 0.98
0.02 0.10 0.02
0.90 0.14 0.95
1.00 0.54 1.00
0.95 0.61 0.97
0.02 0.08 0.02
0.91 0.45 0.94
0.99 0.77 1.00
VOLUME 19 NUMBER 2 2015
E. Boudreaux et al.
Downloaded by [New York University] at 07:39 19 May 2015
greater than chance: difference in Kappa for overall positive ¼ 0.01 (95% CI, 0.08, 0.05); lifetime attempt ¼ 0.01 (95% CI, 0.03, 0.06). However, the PSS-3 showed significantly stronger agreement with the BSSI on active ideation than the PSS-2: Kappa difference for active ideation ¼ 0.27 (95% CI, 0.52, 0.12). Finally, BSSI mean scores were markedly higher for those who screened positive (overall) on the PSS than those who screened negative (PSS-2, z ¼ 17.46, p < 0.0001 and PSS-3, z ¼ 19.29, p < 0.0001). DISCUSSION
Developing and validating an ultra-brief universal suicide risk screening tool for use with adults in the ED is important (D’Onofrio, Jauch, Jagoda et al., 2010; Randall, Colman, & Rowe, 2010). The results of this study hold intriguing implications for this line of study. Both the 2- and 3-item versions demonstrated very strong overall agreement with the BSSI, a well-established suicide risk screening and assessment measure. However, this strong agreement seems to be buoyed primarily by the subdomain of lifetime suicide attempt. While lifetime attempt exhibited near perfect agreement between the PSS and the BSSI measures, agreement across active ideation was more variable. While the 3-item version had a Kappa of 0.61, which is at the traditional cut-off indicating substantial agreement, the Kappa for the 2-items version reached only 0.34, which indicates fair agreement. The reason the PSS did not agree more strongly with the BSSI’s active ideation screening is not a simple story and highlights the complexity of assessing active ideation. The PSS assesses active ideation with a single ‘‘yes=no’’ question that directly assesses active ideation: ‘‘Over the past two weeks, have you had thoughts of killing yourself?’’ However, the BSSI
assesses active ideation in a more graduated manner. It uses two items instead of one, and each item has three ordinal response options. One item assesses the individual’s desire to kill him or herself, and is conceptually similar to the PSS’s active ideation question. The second item, however, assesses whether an individual would take efforts to save his or her life if found in a life threatening-situation. This second item is less clearly assessing active ideation, and could be interpreted as assessing passive ideation. On balance, it would seem easier, therefore, to screen positive on the BSSI than the PSS. If the threshold of the BSSI is consistently easier to reach than the PSS, one would expect less than stellar agreement between the instruments. However, the pattern observed with this data did not support such a simple explanation (see Table 4). The difference between the percentages of positive screens between the PSS and the BSSI were very small and did not suggest greater sensitivity for the BSSI. Ultra-brief screeners should continue to include questions assessing lifetime suicide attempts. The finding that the PSS’s lifetime attempt question detected as many attempters as the BSSI supports its validity. As noted by Allen, Abar, McCormick et al. (2013), most previous studies on suicide risk screening in the ED have focused entirely on ideation and have neglected past attempts. One of the best predictors of a future suicide attempt is having a past suicide attempt (Jacobs, Baldessarini, Conwell et al., 2003; Steeg, Kapur, Webb et al., 2012). While very brief screeners are needed, relying solely upon active ideation and ignoring past attempts would likely miss a significant number of patients at risk. A past attempt alone, particularly a remote attempt, may not place one at imminent risk but it may be an indicator that further assessment is warranted, and can certainly be interpreted as placing an individual with current active ideation into
ARCHIVES OF SUICIDE RESEARCH
157
Validation of the Patient Safety Screener
TABLE 4. Focus on Active Ideation with a Breakdown of the BSSI Items PSS-2
PSS-3
Active ideation
Active ideation
BSSI
PSS-2
(þ) (þ) Not (þ) Desire avoid Both to kill death () items self only only Both Total
Downloaded by [New York University] at 07:39 19 May 2015
Negative Positive Total
1 4 5
1 0 1
2 2 4
447 2 449
451 8 459
a higher risk category requiring more thorough evaluation. This reasoning is consistent with general principles of suicide risk assessment embodied in the American Psychiatric Association’s guidelines (Jacobs, Baldessarini, Conwell et al., 2003), as well as instruments like the CSSRS (Posner, Brown, Stanley et al., 2011) which assesses both ideation and behavior as components of suicide risk. One important facet of screening for past attempts that requires additional study is the importance of the timing of the last attempt. Distant attempts may not be as clinically relevant as recent attempts. The current study’s results supported those of Allen, Abar, McCormick et al. (2013) showing that the vast majority of the positive screens for lifetime attempt were for an attempt occurring more than 6 months before the ED visit. Limitations
This study has several limitations. Because the ED provides care 24 hours a day, 24 hour coverage would have been ideal to ensure sample representativeness. However, logistically, such coverage is difficult, and t most patients who present late in the evening or early morning hours are reluctant or incapable of participating in
158
BSSI
PSS-3
(þ) (þ) Not (þ) Desire avoid Both to kill death () items self only only Both Total Negative Positive Total
0 7 7
4 4 8
2 2 4
466 7 473
472 20 492
survey research. Although the screeners are intended for universal use, and the study was carefully designed to consider every patient who presented to the ED during the study shifts, patient enrollment was limited to adults who were medically, emotionally, and cognitively healthy enough to be interviewed. Overall, the sample is likely to be healthier than the average ED patient. This limitation is mitigated somewhat by the fact that this same limitation is also likely to apply to any clinical implementation of screening. Clinical staff would be unlikely to screen very ill or cognitively impaired patients who are unable to provide valid self-report. Because of requirements of informed consent, participants knew the study focused on mental health screening prior to the administration of the survey. The consent process informed them that those who are at risk for harming themselves would have additional evaluation by their treatment team. This may have biased the findings through discouraging individuals from providing accurate information to avoid possible lengthening of their ED visit. The net effect of this would be to depress the true rate of positive screens but should not affect comparisons between instruments. Finally, the PSS was validated against the BSSI rather than suicidal behavior after
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
the ED visit. This design choice was primarily a pragmatic one. There is no universally accepted suicide risk measure, but the BSSI is one of the most widely used instruments and is commonly accepted as a reliable, valid measure. Following patients to determine suicidal behavior after the ED visit was not feasible because of the unfunded nature of the study.
Correspondence concerning this article should be addressed to Edwin D. Boudreaux, PhD, Professor, Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Emergency Medicine, LA-189, 55 Lake Avenue North, Worcester, MA 01655. E-mail: [email protected]
CONCLUSIONS
FUNDING
Despite variability in estimates of active ideation between the screeners, the overall findings support that active suicidal ideation is present in roughly 2 to 4% of general ED patients and lifetime attempts are relatively common, with estimates ranging from 12 to 15%. It appears that the 3 item PSS is substantially correlated with the BSSI for detecting suicidal ideation, and should be considered for further study.
This study was funded through internal department funds.
AUTHOR NOTE
Edwin D. Boudreaux, PhD, Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Michelle L. Jaques, BA, Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Kaitlyn M. Brady, MS, Worcester Polytechnic Institute, Worcester, Massachusetts, USA. Adam Matson, BS, Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Michael H. Allen, MD, Colorado Depression Center, University of Colorado School of Medicine and the VISN 19 Mental Illness Research Education and Clinical Center, Denver, Colorado, USA.
REFERENCES Allen, M. H., Abar, B. W., McCormick M., Barnes, D. H., Haukoos, J., Gamel, G. M., & Boudreaux, E. D. (2013). Screening for suicidal ideation and attempts among emergency department medical patients: Instrument and results from the Psychiatric Emergency Research Collaboration. Suicide and Life Threatening Behavior, 43, 1–11. Beck, A. T., & Steer, R. A. Manual for the Beck Scale for Suicide Ideation. (1991) San Antonio, TX: Pearson. Betz, M. E., Sullivan, A. F., Manton, A. P., Espinola, J. A., Miller, I., Camargo, C. A.,. . . ED-SAFE (2013). Knowledge, attitudes, and practices of emergency department providers in the care of suicidal patients. Academic Emergency Medicine, 20, S112. Boudreaux, E. D., Cagande, C., Kilgannon, H., Kumar, A., & Camargo. C. A. (2006). A prospective study of depression among adult patients in an urban emergency department. Journal of Clinical Psychiatry, 8, 66–70. Caterino, J., Sullivan, A. F., Betz, M. E., Espinola, J. A., Miller, I., Camargo, Jr., C. A. . . . ED-SAFE (2013). Evaluating current patterns of assessment for self-harm in emergency departments, a multicenter study. Academic Emergency Medicine, 20, 807–15. Claassen, C. A., & Larkin, G. L. (2005). Occult suicidality in an emergency department population. British Journal of Psychiatry, 186, 352–353. D’Onofrio, G., Jauch, E., Jagoda, A., Allen, M. H., Anglin, D., Barsan, W. G., . . . Zatzick, D. (2010). NIH roundtable on opportunities to advance research on neurologic and psychiatric emergencies. Annals of Emergency Medicine. 56, 551–564.
ARCHIVES OF SUICIDE RESEARCH
159
Downloaded by [New York University] at 07:39 19 May 2015
Validation of the Patient Safety Screener
Glasgow, R. E., Kaplan, R. M., Ockene, J. K., Fisher, E. B., & Emmons, K. M. (2012). Patient-reported measures of psychosocial issues and health behavior should be added to electronic health records. Health Affairs, 31, 497–504. Ilgen, M. A., Walton, M. A., Cunningham, R. M., Barry K. L., Chermack, S. T., De Chavez, P., & Blow, F. C. (2009). Recent suicidal ideation among patients in an inner city emergency department. Suicide and Life Threatening Behavior, 39, 508–517. Jacobs, D. G., Baldessarini, R. J., Conwell Y., Fawcett, J. A., Horton, L., Meltzer, H., . . . Simo, R. (2003). Practice guideline for the assessment and treatment of patients with suicidal behaviors. American Psychiatric Association. Washington DC: American Psychiatric Publishing. Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals. 32, 1–7. Landis, J. R., & Koch, G. G. (1997). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. Posner, K., Brown, G. K., Stanley, B., Yershova, K. V., Oquendo, M. A., Currier, G. W., . . . Mann, J. J. (2011). The Columbia-Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents
160
and adults. American Journal of Psychiatry, 168, 1266–1277. Randall, J. R., Colman, I., Rowe, B. H. (2011). A systematic review of psychometric assessment of self-harm risk in the emergency department. Journal of Affective Disorders, 34, 348–355. Ronquillo, L., Minassian, A., Vilke, G. M., & Wilson, M. P. (2012). Literature-based recommendations for suicide assessment in the emergency department: a review. The Journal of Emergency Medicine, 43, 836–842. Steeg, S., Kapur, N., Webb, R., Applegate, E., Stewart, S. L., Hawton, K., . . . Cooper, J. (2012). The development of a population-level clinical screening tool for self-harm repetition and suicide: the ReACT Self-Harm Rule. Psychological Medicine, 7, 1–12. Ting, S. A., Sullivan, A. F., Boudreaux, E. D., Miller, I., & Camargo Jr., C. A., (2012). Trends in U.S. emergency department visits for attempted suicide and self-inflicted injury, 1993–2008. General Hospital Psychiatry, 34, 557–565. U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention. (2012). National strategy for suicide prevention: goals and objectives for action. Washington, DC: HHS.
VOLUME 19 NUMBER 2 2015
Archives of Suicide Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/usui20
The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings a
b
Edwin D. Boudreaux PhD , Michelle L. Jaques BA , Kaitlyn M. Brady c
b
d
MS , Adam Matson BS & Michael H. Allen MD a
Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, Massachusetts, USA b
Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA c
Click for updates
Worcester Polytechnic Institute, Worcester, Massachusetts, USA
d
Colorado Depression Center, University of Colorado School of Medicine and the VISN 19 Mental Illness Research Education and Clinical Center, Denver, Colorado, USA Accepted author version posted online: 31 Mar 2015.
To cite this article: Edwin D. Boudreaux PhD, Michelle L. Jaques BA, Kaitlyn M. Brady MS, Adam Matson BS & Michael H. Allen MD (2015) The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings, Archives of Suicide Research, 19:2, 151-160, DOI: 10.1080/13811118.2015.1034604 To link to this article: http://dx.doi.org/10.1080/13811118.2015.1034604
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [New York University] at 07:39 19 May 2015
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Archives of Suicide Research, 19:151–160, 2015 Copyright # International Academy for Suicide Research ISSN: 1381-1118 print=1543-6136 online DOI: 10.1080/13811118.2015.1034604
Downloaded by [New York University] at 07:39 19 May 2015
The Patient Safety Screener: Validation of a Brief Suicide Risk Screener for Emergency Department Settings Edwin D. Boudreaux, Michelle L. Jaques, Kaitlyn M. Brady, Adam Matson, and Michael H. Allen This study evaluated the concurrent validity of a brief suicide risk screener for adults in the emergency department (ED). Two versions of the verbally administered Patient Safety Screener (2-item, 3-item) were compared to a reference standard, the Beck Scale for Suicide Ideation (BSSI). Analyses included measures of agreement (Kappa). Agreement between the Patient Safety Screener-2 and -3 and the BSSI (n ¼ 951) was almost perfect for overall positive screening (K ¼ 0.94–0.95) and past suicide attempts (K ¼ 0.97–0.98). Agreement on ideation ranged from fair (K ¼ 0.34) for the 2-item version to good (K ¼ 0.61) for the 3-item version. The Patient Safety Screener’s concurrent validity with the BSSI ranged from fair to almost perfect and warrants additional study. Keywords
emergency medicine, emergency psychiatry, suicide screening and assessment
INTRODUCTION
et al., 2006; Claassen and Larkin, 2005; Ilgen, Walton, Cunningham et al., 2009). Similarly, while diagnostic coding data suggest only 0.4% of ED visits are directly related to intentional self-injury (Ting, Sullivan, Boudreaux et al., 2012), studies administering standardized screeners have found 12–13% (Allen, Abar, McCormick et al., 2013; Boudreaux, Cagande, Kilgannon et al., 2006) of ED patients have had at least one suicide attempt in their lifetime. The discrepancy between low detection of suicide risk within treatment as usual samples and the high prevalence of actual suicide risk within research samples is likely due to the fact that most ED clinicians only
Structured chart reviews of emergency department (ED) medical records and diagnostic coding data reveal that less than 3% of emergency department (ED) patients are identified by ED clinicians during usual care as having suicidal ideation or behavior (Caterino, Sullivan, Betz et al., 2013; Ting, Sullivan, Boudreaux et al., 2012). However, standardized screening studies using verbally administered research interviews or paper-and-pencil instruments have found that the point prevalence of active suicidal ideation among ED can reach as high as 11% (Boudreaux, Cagande, Kilgannon
151
Downloaded by [New York University] at 07:39 19 May 2015
Validation of the Patient Safety Screener
screen patients presenting with a psychiatric chief complaint (Betz, Sullivan, Manton et al., 2013). This is referred to as clinically indicated screening. Clinically indicated screening misses many people with active suicidal ideation or a past suicide attempt (Allen, Abar, McCormick et al., 2013, Claassen and Larkin, 2005). By contrast, universal screening may help to identify suicide risk among patients presenting for nonpsychiatric symptoms, sometimes called occult or incidental detection. However, two elementary problems stymie implementation of universal screening in clinical practice in the ED: there are simply no good-fit, well-validated screeners for use with general adult patients, and the impact of screening on improving suicide related outcomes has not been studied (D’Onofrio, Jauch, Jagoda et al., 2010; Randall, Colman, & Rowe, 2011; Ronquillo, Minassian, Vilke et al. 2012; U.S. Department of Health and Human Services [HHS], 2012). A solution for primary suicide risk detection in the ED will need to be simple, efficient, and calibrated to easily separate individuals who require further risk assessment from those who do not. Traditional suicide risk measures, such as the Beck Scale for Suicide Ideation (BSSI) (Beck & Steer, 1991) and the Columbia Suicide Rating Scale (CSSRS) (Posner, Brown, Stanley et al., 2011), are too lengthy and complex to be administered universally by ED staff. In contrast, the Patient Safety Screener-5 (PSS-5) (Allen, Abar, McCormick et al., 2013) was developed as a 5-item primary detection screener for adults in the ED setting. In a sample of 1,068 patients presenting with non-psychiatric medical complaints recruited from six EDs, 2.3% screened positive on active suicidal ideation within the past two weeks, 12.8% reported having a lifetime history of a suicide attempt, and 1.1% reported both active ideation and a past attempt. Unfortunately, no psychometric analyses have been reported
152
on the PSS-5, and it was never validated against a reference standard instrument or post-visit suicidal behavior. Consequently, important test characteristics, such as the PSS-50 s sensitivity, specificity, and false negative rates, have not yet been established. While the PSS-5 was designed to be brief, even five items may suffer from low rates of adoption in busy clinical settings. In general, the efficiency principle holds true: the shorter the screener, the more likely it is to be accepted by clinicians (Glasgow, Kaplan, Ockene et al., 2012). Efficiency is particularly important when considering the ever-expanding list of behaviors and conditions for which clinicians are expected to screen. Screening for suicide risk is not likely to occur in a vacuum, but, instead, as part of a larger battery of preventive and behavioral health screenings. Consequently, the field needs an ultra-brief primary suicide risk screener that can be applied universally by ED staff. It should efficiently identify individuals who need further risk assessment and easily integrate with the larger clinical assessment or preventive health screening battery. This study aimed to validate two shorter versions of the PSS-5 (Allen, Abar, McCormick et al., 2013) (see Figure 1) against the BSSI as a reference standard. The original PSS-5 (Allen, Abar, McCormick et al., 2013) consisted of two items from the PHQ-9 (Kroenke & Spitzer, 2002) assessing depressed mood and anhedonia and three items modeled after the CSSRS (Posner, Brown, Stanley et al. 2011) assessing passive suicidal ideation, active suicidal ideation, and lifetime suicide attempt. In addition, any patient reporting a lifetime attempt is asked to indicate the timing of the most recent attempt (Today, Within 30 days, Within 1–6 months, Greater than 6 months). Allen, Abar, McCormick et al. (2013) found that the items assessing anhedonia and passive ideation did not identify significantly new
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
FIGURE 1. Patient Safety Screener.
cases when compared to the combination of the other three items (depression, active ideation, lifetime attempts). Consequently, to create the PSS-3, the anhedonia and passive ideation items were removed. The depression item was retained in the PSS-3 and is conceptualized as a segue that may help transition from the screener introduction to the potentially sensitive suicide screening items. To create the PSS-2, the depression item was also removed, leaving only active ideation and lifetime attempt. METHODS Participants
At each of three different EDs belonging to the University of Massachusetts Memorial Healthcare System, 15 days were randomly selected between June and September of 2011. Shifts ran between 9:00 a.m. and 9:00 p.m., and all days of the week were represented. All patients 18
years old were logged and considered for approach, regardless of the reason for their visit. Individuals were excluded if they had a medical, emotional, cognitive, or behavioral condition that prevented them from engaging in conversation through the duration of their visit, such as persistent altered mental status, intractable pain or vomiting, or intubation. The age, race, gender, and insurance status of all patients logged were recorded. The average age of the sample was 44.6 (SD ¼ 18), with 52% female, 12% Latino, 88% white, and 8% African American. After providing a complete description of the study to the participants, written informed consent was obtained. Institutional Review Board approval was also obtained by the University of Massachusetts Medical School IRB. Materials
The Patient Safety Screener (PSS). For both versions of the PSS (described in the
ARCHIVES OF SUICIDE RESEARCH
153
Validation of the Patient Safety Screener
Downloaded by [New York University] at 07:39 19 May 2015
Introduction), a positive screen was identified as a positive response for active ideation within the past two weeks OR a lifetime history of a suicide attempt (PSS Overall Positive). The Beck Scale for Suicide Ideation (BSSI). The BSSI is a well-validated, widely used measure of suicide risk. It is comprised of 21 items that may be used to identify the presence and severity of suicidal ideation and past attempts. Each item requires the individual to read three statements and indicate which best describes him=her. The first five items are used to screen for suicidal ideation and includes three items measuring passive ideation (wish to live, wish to die, reasons for living) and two items measuring active ideation (desire to kill oneself, whether one would save one’s life in a life-threatening situation). The remaining 14 ideation items are only asked if the individual screens positive on one of the two active ideation items. In addition, the BSSI includes an item assessing past suicide attempts, which is asked of everyone, and, if positive, a subsequent item inquiring at what level the respondent wished to die during the last attempt. A positive screen for the BSSI was defined as a positive response on one or both of the two items assessing active ideation OR a lifetime history of a past attempt (BSSI Overall Positive). Procedure
Trained research staff evaluated each consecutively presenting patient’s ED visit record, approached those without obvious exclusion criteria, and performed an eligibility interview. Research staff consulted with ED clinicians regarding eligibility, as needed. Those who were eligible were offered the opportunity to participate. Because maximizing response rates was critical and the ED is a time sensitive setting, the assessment was kept brief, and only the data most critical to answering
154
the research question were collected. Those who consented were randomized to one of four groups based on the version of the PSS administered (PSS-2 vs. PSS-3) and the order of the presentation of the instruments (PSS before the BSSI vs. PSS after the BSSI). This design allowed the study to validate both versions while controlling for potential ordering effects. If patients scored positive for active suicidal ideation on either the PSS or the BSSI, the patient’s treating physician or nurse was informed by the research assistant, and the patient was managed based on usual and customary care at the site. The study was approved by the University of Massachusetts Medical School Institutional Review Board. Data Analyses
Descriptive statistics were computed, including the rates on the PSS Overall Positive and the BSSI Overall Positive, as well as the subdomains of active ideation and lifetime attempt. Kappa, a measure of agreement beyond chance, was computed between each of the two PSS versions and the BSSI. To further validate the PSS, the total BSSI scores were compared between those who screened positive and those who screened negative using a Wilcoxon Rank Sum test. Ordering effects were assessed by examining whether the proportion of positive screens was related to the order of presentation (i.e., PSS first or second). Finally, site effects were assessed by examining whether the proportion of positive screens differed based on enrollment site. All statistics were computed at individual 95% confidence levels. SPSS 20 (Chicago IL) was used. RESULTS Participant Enrollment
There were 2,258 adult individuals who presented during the research shifts and
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
were considered for participation; 1,307 were either excluded or declined participation, leaving 951 (42%) who were enrolled into the study. Of those enrolled, 438 were enrolled at University, 411 at Memorial, and 102 at Marlborough. Four-hundred-fifty-nine (n ¼ 459) were randomized to the PSS-2, while 492 were randomized to the PSS-3. Compared to patients who were not enrolled, participants were more likely to be younger (not enrolled: 47 years old vs. enrolled 45 years old, p < 0.05) and female (not enrolled: 51% vs. enrolled: 58%, p < 0.05). There were no differences observed between those enrolled and those not enrolled by race or insurance (both p > 0.05). Prevalence Rates
Table 1 reports the descriptive statistics for both versions of the PSS and the BSSI. Overall, 145 (15.2%) screened positive on at least one of the screeners. Two Chi-square analyses, one computed on each PSS version, demonstrated that the proportion of patients scoring positive
on the PSS did not differ based on the order of presentation (i.e., whether it came before or after the BSSI) (both p > 0.10). Consequently, for all analyses, all participants within each of the two PSS groups were combined regardless of the order of completion. Of the 459 participants who completed the PSS-2, 60 (13.1%) screened positive (Table 2). Of the 492 who took the PSS-3, 82 (16.7%) screened positive. The difference in proportions of positive screens between the two PSS versions was not statistically different, X2 (1, n ¼ 951) ¼ 2.4, p > 0.05. Additionally, sites did not differ significantly from one another on the proportion of positive screens on either version of the PSS or the BSSI (all p > 0.05). Of the 134 who reported a lifetime attempt on either the PSS-2 or PSS-3, 11 (8.2%) reported the most recent attempt occurred within 6 months of the ED visit. Agreement between the Screeners and BSSI
Table 3 reports the Kappa statistics, with 95% Confidence Intervals. Both the
TABLE 1. Descriptive Statistics for the Patient Safety Screener and Beck Scale for Suicide Ideation N (%) 2-item PSS (n ¼ 459)
3-item PSS (n ¼ 492)
PSS-2 Overall positive PSS-2 Active ideation PSS-2 Lifetime attempt BSSI Overall positive BSSI Ideation BSSI Lifetime attempt PSS Overall positive PSS Active ideation PSS Lifetime attempt BSSI Overall positive BSSI Active ideation BSSI Lifetime attempt
60 8 59 60 10 57 82 20 75 76 19 73
(13.1) (1.7) (12.9) (13.1) (2.2) (12.4) (16.7) (4.1) (15.2) (15.4) (3.9) (14.8)
95% C.I. 0.1–0.162 0.006–0.029 0.098–0.159 0.1–0.162 0.009–0.035 0.094–0.154 0.134–0.2 0.023–0.058 0.121–0.184 0.123–0.186 0.022–0.056 0.117–0.18
Note. Of the 134 who reported a lifetime attempt on either the PSS-2 or PSS-3, 3 (2.2%) reported the attempt was within the past 24 hours, 1 (0.7%) within the last month, 7 (5.2%) between 1 and 6 months ago, and 123 (91.7%) longer than 6 months ago.
ARCHIVES OF SUICIDE RESEARCH
155
Validation of the Patient Safety Screener
TABLE 2. 2 2 Tables for 2- and 3-Item Patient Safety Screener and the Beck Scale for Suicide Ideation PSS-2 BSSI
PSS-2 Negative
Downloaded by [New York University] at 07:39 19 May 2015
PSS-3
Positive
Total
3 57 60
399 60 459
4 6 10
451 8 459
0 57 57
400
Overall positive Negative 396 Positive 3 Total 399 Active ideation Negative 447 Positive 2 Total 449 Lifetime attempt Negative 400 Positive 2 Total 402
BSSI
PSS-3
459
PSS-2 and PSS-3 exhibited strong overall agreement with the BSSI, each with a Kappa of 0.95 (Landis & Koch, 1997). Similarly, agreement for the subdomain of lifetime suicide attempt was strong, with Kappas of 0.98 and 0.97, respectively. However, agreement for the subdomain of active ideation was weaker, with Kappas of 0.34 and 0.61, respectively. To determine if the magnitude of the agreement with the BSSI differed between
Negative Overall positive Negative 410 Positive 6 Total 416 Active ideation Negative 466 Positive 7 Total 473 Lifetime attempt Negative 416 Positive 3 Total 419
Positive
Total
0 76 76
410 82 492
6 13 19
472 20 492
1 72 73
417 75 492
the PSS-2 and PSS-3, the difference and the associated 95% confidence intervals between the Kappas for the PSS Overall Positive and the two subdomains (active ideation, lifetime attempt) were examined. The 95% confidence intervals for the difference in Kappas between the PSS-2 and the PSS-3 (Kappa (PSS-2, BSSI) – Kappa (PSS-3, BSSI) ¼ Kappa difference) crossed zero for both Overall Positive and lifetime attempt, suggesting they did not differ
TABLE 3. Agreement Between the 2-Item and 3-Item Patient Safety Screener with the Beck Scale for Suicide Ideation Screener assessment 2-Item Patient Safety Screener Overall positive Active ideation Lifetime attempt 3-Item Patient Safety Screener Overall positive Active ideation Lifetime attempt
156
Kappa
S.E.
C.I. lower
C.I. upper
0.94 0.34 0.98
0.02 0.10 0.02
0.90 0.14 0.95
1.00 0.54 1.00
0.95 0.61 0.97
0.02 0.08 0.02
0.91 0.45 0.94
0.99 0.77 1.00
VOLUME 19 NUMBER 2 2015
E. Boudreaux et al.
Downloaded by [New York University] at 07:39 19 May 2015
greater than chance: difference in Kappa for overall positive ¼ 0.01 (95% CI, 0.08, 0.05); lifetime attempt ¼ 0.01 (95% CI, 0.03, 0.06). However, the PSS-3 showed significantly stronger agreement with the BSSI on active ideation than the PSS-2: Kappa difference for active ideation ¼ 0.27 (95% CI, 0.52, 0.12). Finally, BSSI mean scores were markedly higher for those who screened positive (overall) on the PSS than those who screened negative (PSS-2, z ¼ 17.46, p < 0.0001 and PSS-3, z ¼ 19.29, p < 0.0001). DISCUSSION
Developing and validating an ultra-brief universal suicide risk screening tool for use with adults in the ED is important (D’Onofrio, Jauch, Jagoda et al., 2010; Randall, Colman, & Rowe, 2010). The results of this study hold intriguing implications for this line of study. Both the 2- and 3-item versions demonstrated very strong overall agreement with the BSSI, a well-established suicide risk screening and assessment measure. However, this strong agreement seems to be buoyed primarily by the subdomain of lifetime suicide attempt. While lifetime attempt exhibited near perfect agreement between the PSS and the BSSI measures, agreement across active ideation was more variable. While the 3-item version had a Kappa of 0.61, which is at the traditional cut-off indicating substantial agreement, the Kappa for the 2-items version reached only 0.34, which indicates fair agreement. The reason the PSS did not agree more strongly with the BSSI’s active ideation screening is not a simple story and highlights the complexity of assessing active ideation. The PSS assesses active ideation with a single ‘‘yes=no’’ question that directly assesses active ideation: ‘‘Over the past two weeks, have you had thoughts of killing yourself?’’ However, the BSSI
assesses active ideation in a more graduated manner. It uses two items instead of one, and each item has three ordinal response options. One item assesses the individual’s desire to kill him or herself, and is conceptually similar to the PSS’s active ideation question. The second item, however, assesses whether an individual would take efforts to save his or her life if found in a life threatening-situation. This second item is less clearly assessing active ideation, and could be interpreted as assessing passive ideation. On balance, it would seem easier, therefore, to screen positive on the BSSI than the PSS. If the threshold of the BSSI is consistently easier to reach than the PSS, one would expect less than stellar agreement between the instruments. However, the pattern observed with this data did not support such a simple explanation (see Table 4). The difference between the percentages of positive screens between the PSS and the BSSI were very small and did not suggest greater sensitivity for the BSSI. Ultra-brief screeners should continue to include questions assessing lifetime suicide attempts. The finding that the PSS’s lifetime attempt question detected as many attempters as the BSSI supports its validity. As noted by Allen, Abar, McCormick et al. (2013), most previous studies on suicide risk screening in the ED have focused entirely on ideation and have neglected past attempts. One of the best predictors of a future suicide attempt is having a past suicide attempt (Jacobs, Baldessarini, Conwell et al., 2003; Steeg, Kapur, Webb et al., 2012). While very brief screeners are needed, relying solely upon active ideation and ignoring past attempts would likely miss a significant number of patients at risk. A past attempt alone, particularly a remote attempt, may not place one at imminent risk but it may be an indicator that further assessment is warranted, and can certainly be interpreted as placing an individual with current active ideation into
ARCHIVES OF SUICIDE RESEARCH
157
Validation of the Patient Safety Screener
TABLE 4. Focus on Active Ideation with a Breakdown of the BSSI Items PSS-2
PSS-3
Active ideation
Active ideation
BSSI
PSS-2
(þ) (þ) Not (þ) Desire avoid Both to kill death () items self only only Both Total
Downloaded by [New York University] at 07:39 19 May 2015
Negative Positive Total
1 4 5
1 0 1
2 2 4
447 2 449
451 8 459
a higher risk category requiring more thorough evaluation. This reasoning is consistent with general principles of suicide risk assessment embodied in the American Psychiatric Association’s guidelines (Jacobs, Baldessarini, Conwell et al., 2003), as well as instruments like the CSSRS (Posner, Brown, Stanley et al., 2011) which assesses both ideation and behavior as components of suicide risk. One important facet of screening for past attempts that requires additional study is the importance of the timing of the last attempt. Distant attempts may not be as clinically relevant as recent attempts. The current study’s results supported those of Allen, Abar, McCormick et al. (2013) showing that the vast majority of the positive screens for lifetime attempt were for an attempt occurring more than 6 months before the ED visit. Limitations
This study has several limitations. Because the ED provides care 24 hours a day, 24 hour coverage would have been ideal to ensure sample representativeness. However, logistically, such coverage is difficult, and t most patients who present late in the evening or early morning hours are reluctant or incapable of participating in
158
BSSI
PSS-3
(þ) (þ) Not (þ) Desire avoid Both to kill death () items self only only Both Total Negative Positive Total
0 7 7
4 4 8
2 2 4
466 7 473
472 20 492
survey research. Although the screeners are intended for universal use, and the study was carefully designed to consider every patient who presented to the ED during the study shifts, patient enrollment was limited to adults who were medically, emotionally, and cognitively healthy enough to be interviewed. Overall, the sample is likely to be healthier than the average ED patient. This limitation is mitigated somewhat by the fact that this same limitation is also likely to apply to any clinical implementation of screening. Clinical staff would be unlikely to screen very ill or cognitively impaired patients who are unable to provide valid self-report. Because of requirements of informed consent, participants knew the study focused on mental health screening prior to the administration of the survey. The consent process informed them that those who are at risk for harming themselves would have additional evaluation by their treatment team. This may have biased the findings through discouraging individuals from providing accurate information to avoid possible lengthening of their ED visit. The net effect of this would be to depress the true rate of positive screens but should not affect comparisons between instruments. Finally, the PSS was validated against the BSSI rather than suicidal behavior after
VOLUME 19 NUMBER 2 2015
Downloaded by [New York University] at 07:39 19 May 2015
E. Boudreaux et al.
the ED visit. This design choice was primarily a pragmatic one. There is no universally accepted suicide risk measure, but the BSSI is one of the most widely used instruments and is commonly accepted as a reliable, valid measure. Following patients to determine suicidal behavior after the ED visit was not feasible because of the unfunded nature of the study.
Correspondence concerning this article should be addressed to Edwin D. Boudreaux, PhD, Professor, Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Emergency Medicine, LA-189, 55 Lake Avenue North, Worcester, MA 01655. E-mail: [email protected]
CONCLUSIONS
FUNDING
Despite variability in estimates of active ideation between the screeners, the overall findings support that active suicidal ideation is present in roughly 2 to 4% of general ED patients and lifetime attempts are relatively common, with estimates ranging from 12 to 15%. It appears that the 3 item PSS is substantially correlated with the BSSI for detecting suicidal ideation, and should be considered for further study.
This study was funded through internal department funds.
AUTHOR NOTE
Edwin D. Boudreaux, PhD, Departments of Emergency Medicine, Psychiatry, and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Michelle L. Jaques, BA, Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Kaitlyn M. Brady, MS, Worcester Polytechnic Institute, Worcester, Massachusetts, USA. Adam Matson, BS, Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA. Michael H. Allen, MD, Colorado Depression Center, University of Colorado School of Medicine and the VISN 19 Mental Illness Research Education and Clinical Center, Denver, Colorado, USA.
REFERENCES Allen, M. H., Abar, B. W., McCormick M., Barnes, D. H., Haukoos, J., Gamel, G. M., & Boudreaux, E. D. (2013). Screening for suicidal ideation and attempts among emergency department medical patients: Instrument and results from the Psychiatric Emergency Research Collaboration. Suicide and Life Threatening Behavior, 43, 1–11. Beck, A. T., & Steer, R. A. Manual for the Beck Scale for Suicide Ideation. (1991) San Antonio, TX: Pearson. Betz, M. E., Sullivan, A. F., Manton, A. P., Espinola, J. A., Miller, I., Camargo, C. A.,. . . ED-SAFE (2013). Knowledge, attitudes, and practices of emergency department providers in the care of suicidal patients. Academic Emergency Medicine, 20, S112. Boudreaux, E. D., Cagande, C., Kilgannon, H., Kumar, A., & Camargo. C. A. (2006). A prospective study of depression among adult patients in an urban emergency department. Journal of Clinical Psychiatry, 8, 66–70. Caterino, J., Sullivan, A. F., Betz, M. E., Espinola, J. A., Miller, I., Camargo, Jr., C. A. . . . ED-SAFE (2013). Evaluating current patterns of assessment for self-harm in emergency departments, a multicenter study. Academic Emergency Medicine, 20, 807–15. Claassen, C. A., & Larkin, G. L. (2005). Occult suicidality in an emergency department population. British Journal of Psychiatry, 186, 352–353. D’Onofrio, G., Jauch, E., Jagoda, A., Allen, M. H., Anglin, D., Barsan, W. G., . . . Zatzick, D. (2010). NIH roundtable on opportunities to advance research on neurologic and psychiatric emergencies. Annals of Emergency Medicine. 56, 551–564.
ARCHIVES OF SUICIDE RESEARCH
159
Downloaded by [New York University] at 07:39 19 May 2015
Validation of the Patient Safety Screener
Glasgow, R. E., Kaplan, R. M., Ockene, J. K., Fisher, E. B., & Emmons, K. M. (2012). Patient-reported measures of psychosocial issues and health behavior should be added to electronic health records. Health Affairs, 31, 497–504. Ilgen, M. A., Walton, M. A., Cunningham, R. M., Barry K. L., Chermack, S. T., De Chavez, P., & Blow, F. C. (2009). Recent suicidal ideation among patients in an inner city emergency department. Suicide and Life Threatening Behavior, 39, 508–517. Jacobs, D. G., Baldessarini, R. J., Conwell Y., Fawcett, J. A., Horton, L., Meltzer, H., . . . Simo, R. (2003). Practice guideline for the assessment and treatment of patients with suicidal behaviors. American Psychiatric Association. Washington DC: American Psychiatric Publishing. Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals. 32, 1–7. Landis, J. R., & Koch, G. G. (1997). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. Posner, K., Brown, G. K., Stanley, B., Yershova, K. V., Oquendo, M. A., Currier, G. W., . . . Mann, J. J. (2011). The Columbia-Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents
160
and adults. American Journal of Psychiatry, 168, 1266–1277. Randall, J. R., Colman, I., Rowe, B. H. (2011). A systematic review of psychometric assessment of self-harm risk in the emergency department. Journal of Affective Disorders, 34, 348–355. Ronquillo, L., Minassian, A., Vilke, G. M., & Wilson, M. P. (2012). Literature-based recommendations for suicide assessment in the emergency department: a review. The Journal of Emergency Medicine, 43, 836–842. Steeg, S., Kapur, N., Webb, R., Applegate, E., Stewart, S. L., Hawton, K., . . . Cooper, J. (2012). The development of a population-level clinical screening tool for self-harm repetition and suicide: the ReACT Self-Harm Rule. Psychological Medicine, 7, 1–12. Ting, S. A., Sullivan, A. F., Boudreaux, E. D., Miller, I., & Camargo Jr., C. A., (2012). Trends in U.S. emergency department visits for attempted suicide and self-inflicted injury, 1993–2008. General Hospital Psychiatry, 34, 557–565. U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention. (2012). National strategy for suicide prevention: goals and objectives for action. Washington, DC: HHS.
VOLUME 19 NUMBER 2 2015
