Readiness for inpatient psychotherapy Haggerty et al.

Determining adolescents’ suitability for inpatient psychotherapy: Utility of the clinician-rated Readiness for Inpatient Psychotherapy Scale with an adolescent inpatient sample Greg Haggerty, PhD Caleb Siefert, PhD Valentina Stoycheva, MA Samuel Justin Sinclair, PhD Matthew Baity, PhD Jennifer Zodan, PhD Ashwin Mehra, PhD Vijay Chand, MD Mark A. Blais, PsyD

Growing economic pressure on inpatient services for adolescents has resulted in fewer clinicians to provide individual psychotherapy. As a result, inpatient treatment trends have favored group psychotherapy modalities and psychopharmacological interventions. Currently, no clinician-rated measures exist to assist clinicians in determining who would be able to better utilize individual psychotherapy on inpatient units. The current study sought to demonstrate the utility of the Readiness for Inpatient Psychotherapy Scale with an adolescent inpatient sample. This study also used Greg Haggerty, Jennifer Zodan, Ashwin Mehra, and Vijay Chand are at the Von Tauber Institute for Global Psychiatry, Nassau University Medical Center, East Meadow, New York. Mark A. Blais and Samuel Justin Sinclair are at Massachusetts General Hospital and Harvard Medical School. Caleb Siefert is at the University of Michigan– Dearborn. Valentina. Stoycheva is at the Derner Institute for Advanced Psychological Studies, Adelphi University, Garden City, New York. Matthew Baity is with Alliant International University. This study was funded in part by a grant awarded to the first author from the American Psychoanalytic Association and by National Institute of Mental Health (NIMH) Grant No. 1R21MH097781-01A1 received by the first author. Correspondence may be sent to Greg Haggerty, PhD, 320 1st Ave., Massapequa Park, NY 11762; e-mail: [email protected] (Copyright © 2014 The Menninger Foundation)

Vol. 78, No. 4 (Fall 2014)

349

Haggerty et al.

the RIPS as it is intended to be used in everyday practice. Results from the authors’ analyses reveal that the RIPS demonstrates good psychometrics and interrater reliability, as well as construct validity. (Bulletin of the Menninger Clinic, 78[4], 351–372)

Because of economic pressures, many psychiatric inpatient services are not able to provide all patients with individual psychotherapy (Druss, 2010; Olfson & Marcus, 2010). Fullerton, Busch, Normand, McGuire, and Epstein. (2011) found that the use of psychotherapy in the acute inpatient episode has decreased from 42.6% to 26.9% between 1996 and 2005. As a result, a greater emphasis is being placed on the group psychotherapy modality. This change in treatment emphasis is driven more by a pragmatic solution to having fewer clinicians on inpatient units than by any demonstrated results that group therapy is more efficacious than individual psychotherapy. Inpatient hospitalization for adolescents is the most expensive treatment modality and has also followed several trends shown in adult inpatient data. Blanz and Schmidt (2000) report that, due to economic pressures, inpatient beds for adolescents have been reduced. This follows a trend of reducing the use of inpatient hospitalization overall in mental health services. Economic pressures have also led to staffing reductions. Reduction in the number of inpatient clinicians able to conduct individual psychotherapy has necessitated that inpatient staff be judicious in allocating these resources. In many cases, the clinical judgment of staff conducting the initial interviews is utilized in making the decision on which patients require inpatient individual therapy. Although clinical judgment is likely to be reasonably effective, it is possible that a brief clinician-rated scale could enhance and improve this process. Indeed, structured, quantitative clinician-rated scales have been shown to improve a number of difficult clinical judgments (e.g., Westen, Dutra, & Shedler, 2005; Westen, Shedler, Durrett, Glass, & Martens, 2003; Westen & Weinberger, 2004). Furthermore, scales that can quantify clinician decisions are beneficial because norms for such measures can be established to aid in treatment planning decisions (Westen & Harnden-Fisher, 2001; Westen & Shedler, 1999). As

350

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

such, quantitative clinician-rated scales that can aid clinicians in identifying patients who can utilize individual therapy are likely warranted. Readiness and capacity1 for psychotherapy has been defined as the ability of patients to engage in psychotherapeutic treatment and to collaborate therapeutically with their clinician(s) regarding their presenting problems and distress (Giyaur, Sharf, & Hilsenroth, 2005; Thackery, Butler, & Strupp, 1993). Ogrodniczuk, Joyce, and Piper (2009) state that readiness for psychotherapy “refers to a patient being psychologically prepared to undertake the tasks associated with engaging in, and utilizing the provisions (the therapeutic alliance, the therapist’s interventions) of psychotherapy” (p. 427). General criteria for determining the readiness of someone for psychotherapy have been posited previously. These include the need for help (Petry, Tennen, & Affleck, 2000), psychologicalmindedness (McCallum & Piper, 1997), the ability to relate to others (Moras, 2002), and the presence of interpersonal distress (Powers, 1985). Others have specifically outlined the elements to include openness in discussing personal matters (Krause, 1967), readiness to make reasonable sacrifices (Hacker, 1962), desire to change (Truant, 1999) and the level of distress the person is experiencing (Schneider & Klauer, 2001). The level of distress and readiness for psychotherapy seem to be akin to arousal and performance in the sense that too little and too much is not optimal (Ogrodniczuk et al., 2009). Derisley and Reynolds (2000) feel as though too much distress can impair a person’s ability to take part in psychotherapy. Although the idea of readiness or capacity for psychotherapy is regarded as important by clinicians (Ogrodniczuk et al., 2009), little in the way of scale development exists to assess this construct. Only two measures are available: the Counseling Readiness Scale (CRS), a subscale of the Adjective Check List (Gough & Heilbrun, 1980), and the Readiness for Psychotherapy Index (RPI; Ogrodniczuk, et al., 2009). The CRS was made to assess for people who showed openness to change and who seemed likely to benefit from counseling. Respondents 1. We will use the terms capacity and readiness for psychotherapy interchangeably.

Vol. 78, No. 4 (Fall 2014)

351

Haggerty et al.

were asked to endorse adjectives they felt were more descriptive of themselves. The adjectives for the CRS were selected by analyzing protocols of those who showed positive versus negative responses to counseling. Unfortunately, the CSR has different forms for males and females, demonstrates poor reliability for the female version, and lacks validity and reliability data in general. The measure also seems to conceptualize readiness as a character trait rather than a state (Ogrodniczuk et al., 2009). The items of the RPI were constructed from existing literature, surveyed clinicians, and the authors. Like the CRS, the RPI conceptualizes readiness as a character state rather than a trait. The 42-item measure was found to assess four factors related to readiness: disinterest, perseverance, openness, and distress. The limitation with both of these measures is that they are self-reports and are not created for use with an acute inpatient population. As we have highlighted, the CRS has questionable empirical evidence and the RPI is a longer self-report questionnaire, which patients may find difficult to complete and clinicians may find difficult to administer and score. The use of selfreport questionnaires with patients hospitalized for assessing readiness for psychotherapy is questionable because many of these patients are under duress and in crisis and may not be able to adequately assess themselves on this domain. In the present article, we report on three studies that investigated the psychometrics, construct validity, and interrater reliability of the Readiness for Inpatient Psychotherapy Scale (RIPS; Blais et al., 1999). The RIPS was constructed as part of an inpatient psychotherapy research program. The measure was a modification of the Capacity for Dynamic Process Scale (CDPS; Thackery et al., 1993) to fit the needs of an inpatient population and also to reflect the abilities needed for engaging in inpatient individual psychotherapy based on the available literature on the topic (McCallum & Piper, 1997; Moras, 2002; Ogrodniczuk et al., 2009; Petry et al., 2000). Even though the measure was an initial modification of the CDPS, the RIPS was intended to assess patients’ general capacity to engage in talk therapy and not just psychodynamic psychotherapy. The RIPS was originally made for an adult inpatient population, and ini-

352

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

tial results (Smith & Lechner, 2002) demonstrated that the RIPS was positively related to psychological health and well-being on the Schwartz Outcome Scale-10 (SOS-10; Blais et al., 1999) while it was negatively related to general symptom severity as measured by the Brief Symptom Measure-25 (BSM-25; Blais, Blagys, Rivas-Vazquez, Bello, & Sinclair, 2013; Blais et al., 1999). In the first study, we compared RIPS ratings with self-reported measures of psychological health and well-being, global symptom severity, behavioral problems, and clinician-rated engagement in group and individual therapy, as well as global functioning. We investigated the discriminative validity of the RIPS to identify those who would show better outcomes. We hypothesized that RIPS ratings would be positively related to psychological health and well-being [H1] (SOS-10; Blais et al., 1999) and global functioning [H2-concurrent validity] (Global Assessment of Functioning [GAF]; American Psychiatric Association [APA], 2000) and negatively related to global symptom severity [H3-discriminant validity] (BSM-25; Blais et al., 1999) scores at admission, as has been found in initial results using these measures with an adult inpatient sample (Smith & Lechner, 2002). We also expected that RIPS ratings would be negatively correlated with the Parent-Rated Externalizing Scale of the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) [H4-discriminant validity]. Patients who externalize are less able to engage in therapy productively. These people often prefer to act on rather than sit with and evaluate their emotions, which are important for engaging in psychotherapy (Keithy, Samples, & Strupp, 1980; Rosenbaum & Horowitz, 1983). We hypothesized that the ratings on the RIPS would be positively correlated with the individual therapists’ ratings of engagement in therapy [H5-predictive validity]. We also expected that the RIPS would demonstrate discriminative validity, showing that those who score higher than the mean and the median on the RIPS demonstrate significantly better outcomes than those who score below the mean and the median [H6-discriminative validity].

Vol. 78, No. 4 (Fall 2014)

353

Haggerty et al.

Because our first study did not contain any reliability data, we conducted a second study to investigate the interrater reliability of the RIPS. In our third study, we investigated the factor structure of the RIPS using data from Study 1 and Study 2 and hypothesized that the RIPS is unifactoral. STUDY 1. INITIAL VALIDITY Methods Participants. The sample consisted of 72 patients (45% of the total patients admitted to the unit), 52.8% male, consecutively admitted to the Adolescent Psychiatric Inpatient Unit of a large northeastern hospital. Patients were between the ages of 13 and 17 with a mean age of 15.7 (SD = 1.18). Ethnic makeup of the sample was as follows: 40.8% Caucasian, 25.4% African American, 25.4% Latino/Hispanic/Spanish, 5.6% Other, and 2.8% Asian. The primary diagnoses for these 72 patients were as follows: 64% mood disorders, 30% conduct disorder/ oppositional defiant disorder, 2% impulse control disorder, 3% psychosis, and 1% posttraumatic stress disorder. We found no significant differences in age, gender, diagnosis, or scores on admission measure between those who consented to the study versus those who did not consent. We found no significant differences in age, gender, or diagnosis between those who consented to the study versus those who did not consent. Four patients who showed cognitive impairment or IQ below 70 were excluded from this study. Procedure. Parents and legal guardians were approached for participation by the research team member who was not one of the clinicians in the study at the end of the initial family meeting. All patients who were diagnosed as mentally retarded (IQ < 70) were excluded from study participation. Each patient was given a self-report admission assessment packet at or within 1 day of admission to the unit. All patients were asked to complete an admission assessment packet as per the clinical protocol of the

354

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

unit. Only consenting patients’ admission assessment data were used in the research study. The admission packet included SOS10 (Blais et al., 1999) and the BSM-25 (Blais et al., 1999). The consenting parents completed the CBCL (Achenbach & Rescorla, 2001). At or within 1 day of discharge, patients completed a discharge assessment packet the included the SOS-10 and the BSM-25. The unit psychiatrist administered the RIPS after his initial evaluation and diagnostic interview at the patients’ admission to the unit. The RIPS ratings were given directly to one of the research staff to ensure that all other study staff would be blind to these results. All study staff and patients were blind to the results of the RIPS. The unit psychiatrist completed the RIPS ratings as part of the unit’s clinical assessment protocol. The unit psychiatrist has over 25 years of experience in working with adolescent patients and is board certified in child and adolescent psychiatry. Setting. The adolescent inpatient unit in this study is a 12-bed locked facility at a large northeastern hospital. The unit provides treatment for adolescents in acute distress. The average length of stay for patients on this unit is 10.81 days (SD = 5.23). While on the unit, adolescents receive psychopharmacological treatment as deemed necessary, two or three individual psychotherapy sessions per week, three general group sessions per week, a weekly structured anger management group, a weekly substance use psychoeducation group, daily academic programming, and daily recreation therapy. Individual therapy on the unit is structured around assisting in alleviating the patient’s acute symptomatology so as to stabilize the patient. Therapy is provided by either a licensed psychologist with over 5 years of experience or an advanced psychology graduate trainee (psychology extern or psychology intern as part of the hospital’s APA-accredited internship) who is supervised by a licensed psychologist. The therapy provided is integrative in nature. Therapists meet with adolescents individually and work together to identify goals for therapy in the first session. A mixture of cognitive-behavioral and insight-oriented interventions is then used to help clients work toward these goals. Adolescents Vol. 78, No. 4 (Fall 2014)

355

Haggerty et al.

are encouraged to discuss feelings about their struggles, explore interpersonal patterns, and then work with therapists to identify methods for working toward goals while on the unit. In addition, therapists and adolescents focus on interactions with other members of the unit staff and other patients. In the final session with the adolescents, therapists assess how the client feels he or she is doing, explore the client’s experience of the hospital, discuss how gains from the inpatient stay can be continued, and review how the client feels about discharge. On average, participants received three individual therapy sessions (SD = 1.56). Group therapy sessions on the unit were delivered by members of the psychology staff. All groups are co-led by the licensed unit psychologist, psychology extern, or psychology intern. Group work by externs and interns is supervised by a licensed unit psychologist. Groups are offered five times per week on the unit, and adolescents were encouraged to attend all groups offered. All group therapy sessions are open groups; thus participants in the group may change daily. Three groups are semistructured, free speech groups in which members were free to talk about various topics affecting them. The fourth group is focused on anger management and is informed by dialectical behavior therapy. A fifth group is focused on educating the patients about the consequences of substance use, understanding the potential triggers for use, and identifying available outpatient resources. On average, participants engaged in six group sessions (SD = 3.08) during their stay. All patients, whether they were enrolled in this study or not, received individual psychotherapy and group therapy as part of their treatment on the unit. Patients on the unit were assigned to an individual psychotherapist and a group psychotherapist in an ecologically valid manner based on clinician availability and caseload. Individual and group psychotherapists completed a GAF assessment based on the admission presentation of the patient (this assessment was based on the description of the patient on the ER history and physical assessment, the first psychiatry attending assessment note, and the first nursing assessment note in the chart). The clinicians completing the GAF were blinded to the GAF scores in the chart given by other providers

356

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

(i.e., Emergency Department attending, unit admitting nurse). The patient’s individual therapist also completed the global assessment of the patient’s engagement in individual therapy, and the group therapist did the same assessment for group therapy. The individual and group therapists were blind to each others’ ratings for both admission and discharge measures. Participating clinicians included a licensed clinical psychologist with over 5 years of experience and advanced training in assessment and five advanced clinical psychology doctoral students (interns and externs) who had completed advanced coursework in assessment at an APA-accredited clinical psychology PhD program and were supervised by a licensed clinical psychologist. All study clinicians met on a biweekly basis for reliability training for clinical rating measure. Measures. The Readiness for Inpatient Psychotherapy (RIPS; Blais et al., 1999) is an eight-item scale rated by a clinician who has interviewed the patient but who is not the patient’s therapist. The items are scored using a 6-point Likert-style scale from 0 (not at all) to 5 (totally). The items assess constructs similar to those assessed in the RPI (level of distress, desire for change, willingness to work in therapy, recognition of problems as psychological, willingness to discuss personal matters, willingness to endure discomfort in therapy, and responsibility for change) but were adjusted for the inpatient setting. This rating was taken from the consenting patients’ medical records as it was completed as part of the unit’s clinical assessment protocol. See the appendix for scale and scoring instructions. Cronbach’s alpha was .94 for this sample. The Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) is a widely used parent-report questionnaire designed to assess the behavioral problems and social competencies of children 4 to 18 years of age. The CBCL is composed of 118 problem items and 20 competence items grouped into 11 Problem Scales (including eight Syndrome Scales) and four Competence Scales. The CBCL also yields two broadband, higherorder psychopathology scales, internalizing and externalizing. The CBCL is broadly employed in both clinical and research settings because of its demonstrated reliability and validity, Vol. 78, No. 4 (Fall 2014)

357

Haggerty et al.

ease of administration and scoring, and applicability to clinical, nonclinical, and cross-cultural samples (Achenbach & Rescorla, 2001). Cronbach’s alphas for internalizing and externalizing scales were .86 and .93, respectively, for this sample. The Schwartz Outcome Scale-10 (SOS-10; Blais et al., 1999) is a 10-item Likert-style self-report measuring psychological well-being and distress. Each item is scored on a scale from 0 to 6 yielding possible total scores from 0 to 60. Higher scores on the SOS-10 are representative of better psychological health and well-being. Research has shown that the SOS-10 is a valid and reliable measure of quality of life and psychological well-being with adults and adolescents (Dragomirecka, Lenderking, Motlova, Goppoldova, & Selepova, 2006; Haggerty, Blake, Naraine, Siefert, & Blais, 2010, Haggerty, Walsh, et al., 2013; Laux & Ahern, 2003; Rivas-Vazquez et al., 2001; Young, Waehler, Laux, McDaniel, & Hilsenroth, 2003). Cronbach’s alpha in the present study was .92. The Global Assessment of Functioning (GAF; American Psychiatric Association, 2000) is a 100-point clinician-rated scale created to tap a patient’s overall level of functioning. The scale is behaviorally anchored to help guide clinical ratings. The patient’s individual and group therapists completed the GAF blind to each other’s rating at admission based on all available data about the patient’s functioning when admitted and again at discharge. The two GAF ratings were averaged together to form a mean admission GAF score, and discharge GAF ratings were also averaged together to form a mean discharge GAF score. The mean score was used to provide the most conservative score. Given that raters’ scores were averaged together, we calculated the intraclass correlation coefficient (ICC; Shrout & Fleiss, 1979) using a one-way random approach to assess reliability. ICCs of .75 is considered excellent (Shrout & Fleiss, 1979). Raters in the present study obtained an ICC(1,2) of .58 for the admission GAF. The patients’ individual and group psychotherapists rated the overall amount of engagement and participation the patients showed in the modality in which they were treated (i.e., group

358

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

or individual therapy) during the length of their hospitalization. The therapists rated them on a scale from 1 (not engaged) to 5 (very engaged). Each rating was done blind to the other clinicians’ ratings of each patient as well as to self-report questionnaire results. The rating was also completed at each patient’s discharge. To provide guidance on scoring this measure, training included reviewing patients known to the raters but not part of the study. Raters had to evidence at least an ICC of .70 during the training sessions for this measure. The Brief Symptom Measure-25 (BSM-25; Blais et al., 2013) is a unidimensional 25-item self-report of severity of psychiatric symptoms. The items generated were guided primarily by the diagnostic categories of the DSM-IV (APA, 2000). The measure was not designed to be a diagnostic tool (e.g., not intended to diagnose schizophrenia). The measure is scored by summing up the items, which are scored on a 7-point Likert scale. Previous research has shown that the BSM-25 demonstrates good sensitivity to change and good psychometric properties in adult inpatient populations (Blagys et al., 2002; Blais et al., 2013) as well as in adolescent inpatients (Haggerty, Kahoud, Walsh, Ahmed, & Blais, 2013). The measure has also shown high correlations with the Brief Symptom Inventory (BSI; Derogatis & Melisaratos, 1983) and the short form of the SCL-90 (Derogatis & Melisaratos, 1983). Cronbach’s alpha in the present study was.93. Results We performed t tests to investigate gender differences on all of the study measures. Results reveal a gender difference on the RIPS and engagement in group therapy. Females were rated higher than males on the RIPS, t(69) = 3.25, p < .01, and also on the rating of engagement in group therapy, t(62) = 2.16, p = .04. Gender differences were also revealed for the CBCL externalizing scale and the BSM-25 scale. Males scored higher on the parent-rated CBCL externalizing scale, t(36) = 2.39, p = .02. Females scored higher on the BSM-25, t(66) = 2.01, p = .05. Pearson product moment correlations were conducted to examine the divergent and concurrent validity of the RIPS. Table 1

Vol. 78, No. 4 (Fall 2014)

359

Haggerty et al. Table 1. Mean, Standard Deviation, and Pearson Product Moment Correlations Between Study Measures and RIPS Score N

Mean

SD

RIPS r

p

RIPS

72

20.80

8.61

—

—

CBCL Internalizing

38

.65

.31

.07

.66

Externalizing

38

.91

.42

−.53

< .01

BSM-25 at admission

68

50.85

30.47

.28

.02

SOS-10 at admission

70

36.76

14.09

−.24

.05

GAF mean at admission

67

41.51

6.17

.25

.04

Group therapy engagement rating

65

2.71

1.11

.20

.11

Individual therapy engagement rating

57

3.28

1.19

.57

< .001

Note: The CBCL scales were mean item scores because some of the scales had more items than others, and we used the mean scores rather than standardized scores.

shows the means and standard deviations of all study measures as well as the results of these analyses. The results reveal negative correlations between the RIPS and the SOS-10 admission total score [H1], GAF mean score at admission [H2], BSM-25 admission total score [H3], and the parent-rated CBCL externalizing scale [H4]. We also found that the RIPS score was related to engagement rating for individual psychotherapy but not for group psychotherapy [H5]. Using the RIPS as the criterion measure, we conducted two sets of discriminative validation analyses [H6] (see Steiner & Norman, 2008, p. 261). In the first analyses, groups were created by splitting the sample at the median RIPS score (19), and in the second, groups were created using the mean RIPS score (20.75) split. In the median split analyses, the above-median group had 27 patients and the below-median group had 26 patients with both admission and discharge scores for the BSM-25 and SOS-10. For the GAF, the above-median group had 31 patients and the below-median group had 33 patients. The abovemedian group had significantly higher change scores for the BSM-25 and the SOS-10 but not for GAF. The effect size for the group differences for the BSM-25 and SOS-10 change scores is .62. In the mean split analyses, the above-RIPS mean group had 25 patients and the below-RIPS mean group had 28 patients for the change scores for BSM-25 and SOS-10. For the GAF, the

360

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

above-RIPS mean group had 29 patients and the below-RIPS mean group had 35 patients. The above-mean group had a significantly higher change score on the BSM-25 and SOS-10 but not for the GAF. In addition, the effect sizes for the group differences for the BSM-25 and SOS-10 change scores were .66 and .57, respectively. We also conducted correlational analyses between the RIPS score and the change scores on the BSM-25, the SOS-10, and the GAF. Although the results all trended in the hypothesized direction, they were not statistically significant. STUDY 2: INTERRATER RELIABILITY

Methods Participants. The sample consisted of 32 patients, 59% female, consecutively admitted to the Adolescent Psychiatric Inpatient Unit of a large northeastern hospital. Average length of stay for this sample was 12.7 days. Patients were between the ages of 13 and 17, with a mean age of 16.1 (SD = 1.04). Ethnic makeup of the sample was as follows: 27% Caucasian, 23% African American, 38% Latino/Hispanic/Spanish, 8% Other, and 4% Asian. The primary diagnoses for these 32 patients were as follows: 56% mood disorders, 41% conduct disorder/oppositional defiant disorder, 1% impulse control disorder. Patients who showed cognitive impairment or IQ below 70 were excluded from this study (none met this criteria). Procedures. To investigate the RIPS interrater reliability, we had two raters complete the RIPS blind to each other’s ratings on 32 consecutively admitted patients to the same unit. These 32 patients were not included in Study 1. Setting and Measure. The setting was the same as in Study 1, and the measure was RIPS.

Vol. 78, No. 4 (Fall 2014)

361

Haggerty et al.

Figure 1. Scree Plot for RIPS

Results Given that the raters were not necessarily the same across all patients, a one-way random effects model was calculated [ICC (1,2) = .82 (95% CI: .63–.91)] and found to be in the excellent range (Shrout & Fleiss, 1979). To improve the clinical applicability of the RIPS, we also calculated the single measure ICC (1,1) because most practitioners are unlikely to use a second rater for their patients. This resulted in a reliability estimate in the good range (.69; 95% CI: .46–.84) range. STUDY 3: FACTOR STRUCTURE An important aspect of evaluating the psychometric properties of the RIPS was to investigate the factor structure of the instrument. In order to achieve a sufficient subject-to-variable ratio, the sample from this analysis consisted of all subjects in Studies 1 and 2. We had 104 subjects who had acceptable item level data, allowing for an acceptable 13:1 subject-to-variable ratio.

362

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy Table 2. RIPS Item Means, Standard Deviations, Adjusted Item-to-Scale Correlation Overall Mean (SD)

Mean (SD) for Males

Mean (SD) for Females

Item-to-Scale Correlation

RIPS 1

3.93 (1.13)

RIPS 2

1.92 (1.49)

3.43 (1.32)

4.17 (.99)

.78

.97 (1.24)

2.20 (1.53)

.84

RIPS 3 RIPS 4

1.95 (1.33)

1.38 (1.16)

2.06 (1.28)

.77

3.26 (1.22)

2.89 (1.29)

3.43 (1.04)

.76

RIPS 5

2.33 (1.30)

1.73 (1.10)

2.51 (1.20)

.78

RIPS 6

3.46 (1.21)

3.03 (1.19)

3.46 (1.04)

.75

RIPS 7

2.94 (1.18)

2.59 (1.32)

3.37 (.98)

.73

RIPS 8

2.34 (1.47)

1.70 (1.49)

2.63 (1.35)

.85

Note: α = .94, N = 72 (35 female, 37 male).

Results An exploratory principal components analysis was used. This analysis yielded one component with an eigenvalue greater than 1. Factor 1 had an eigenvalue of 5.81, accounting for 72.6% of the total variance. Figure 1 shows the scree plot. Table 2 shows the means and standard deviations for each item and the individual reports for each gender. The table also includes corrected item-to-scale correlations for the RIPS items. The results demonstrate that the RIPS is a unifactoral measure. DISCUSSION Current trends in mental health reveal a decrease in the utilization of inpatient psychotherapy services, despite data suggesting that the combination of psychotherapy with pharmacotherapy is most efficacious (Fullerton et al., 2011). At the present time, the RIPS is the only clinician-rated measure designed to assess readiness for inpatient psychotherapy for adolescents. The RIPS may aid inpatient staff in identifying patients who are likely to benefit most from individual psychotherapy. Such a tool may be particularly useful on inpatient units where the number of clinicians who can provide individual therapy is limited in relation to the number of patients the unit houses. The current study is the first to examine important scale qualities of the RIPS, as we

Vol. 78, No. 4 (Fall 2014)

363

Haggerty et al.

examined convergence and divergence with measures of engagement in therapy, psychological health and well-being, and distress, as well as interrater reliability. Furthermore, it is the first to examine the scale properties of the RIPS when the measure is utilized in an ecologically consistent manner. We hypothesized that the RIPS would relate to other measures of treatment engagement, psychological health and wellbeing, and distress. The majority of correlations between the RIPS and other measures also tended to be in anticipated directions. Most importantly, RIPS ratings that were made at admission converged with individual therapists’ ratings for treatment engagement that were made at discharge. Patients who were assessed as more ready for therapy by the intake clinician using the RIPS were independently rated by their individual therapists, who were blind to the RIPS data, as more engaged in the work of therapy during the hospitalization. Surprisingly, the RIPS seemed to better tap readiness for individual therapy as opposed to group and individual therapy. We did not explicitly hypothesize this result, and future research with the measure will need to help distinguish whether this is so and perhaps why this is so. Higher RIPS scores were associated with greater patient-reported distress, lower patient-reported well-being, and greater clinician-rated global functioning. This pattern of findings suggests that patients who view themselves as distressed and needing help, but who are also viewed by clinicians as possessing higher levels of overall global functioning, were more likely to be independently rated as more ready for inpatient psychotherapy and also are more engaged in their individual psychotherapy. Perhaps patients need to be in enough distress to be motivated to participate in therapy (Ogrodniczuk et al., 2009). The RIPS was negatively related to the parent-rated CBCL externalizing scale. We expected this result based on past research. The CBCL scale is made up of the subscales of attention problems, delinquent behavior, and aggressive behavior. High scores on any or all of these subscales may indicate that an individual has limited ability to talk with another about personal problems, has reservations about what psychotherapy can of-

364

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

fer, and is reluctant to take responsibility for personal actions (Hacker, 1962; Keijsers, Schaap, Hoogdiun, Hoogsteyns, & de Kamp, 1999; Krause, 1967). Furthermore, individuals high in externalizing symptoms are more likely to locate the source of their challenges outside of themselves, leading to significant problem behaviors (Chassin, Pitts, DeLucia, & Todd, 1999). In general, research suggests that individuals who possess antisocial characteristics are less likely to engage in therapy and benefit less from individual treatments (Woody, McLellan, Luborsky, & O’Brien, 1985). Perhaps for these patients, motivational interviewing techniques would aid in their ability to see how their problems affect their lives. We expected that the RIPS ratings would be significantly positively correlated with group therapist ratings of engagement in inpatient group therapy. However, this relationship only trended toward significance, and the overall effect size was small. This was unexpected, as we had anticipated that RIPS scores would be positively associated with engagement in therapy in general. Such a result may suggest that the construct tapped by the RIPS is more in line with that underlying engagement in individual therapy. It is possible that the characteristics that make one appropriate for individual inpatient psychotherapy may differ somewhat from the characteristics that make one appropriate for group therapy in this treatment setting. The RIPS does not have an item that taps the patients’ level of extraversion or social anxiety with peers. Although the RIPS contains an item tapping the patient’s ability to talk with the treatment team, some patients might feel more comfortable talking privately with mental health workers than in front of their peers. Gender differences with some of the study measures and scales were discovered. Female patients scored higher than males on the RIPS and the rating of engagement in group therapy. This makes sense given the developmental literature, which shows that females demonstrate better verbal skills during adolescence, a key component of the capacity to engage and utilize psychotherapy. Females also scored higher on the BSM-25. These findings are in line with admissions data suggesting that males were more often hospitalized for aggressive and conduct-disordered

Vol. 78, No. 4 (Fall 2014)

365

Haggerty et al.

behavior, while more of the females were hospitalized for behaviors in line with depression (suicidal ideation and attempts) and affect dysregulation (borderline personality disorder, nonsuicidal self-injury). Gender differences were also detected on the CBCL externalizing scale and the BSM-25 scale, where male patients scored higher than females. This would make sense because adolescent males are more prone to acting out behavior than their female counterparts. The results showed that the RIPS demonstrates good discriminative validity. Those who scored above the median and the mean on the RIPS reported greater improvement from admission to discharge on the BSM-25 and the SOS-10 than those who scored below the median and the mean. These findings are encouraging because the RIPS is a clinician-rated measure whereas the BSM-25 and the SOS-10 are patient self-reports. While the RIPS was associated with differential outcomes for self-report scales, clinician-rated change for the GAF failed to reach statistical significance (although results trended toward significance). Research has found that clinicians are more conservative with their assessments of patients than patients’ self-reports (Defife, Drill, Nakash, & Westen, 2010). Our results also showed that Pearson product correlations between the RIPS and the change scores in these outcome measures trended but was not significant. Taken together, these results may show that the RIPS may not be as sensitive as we thought but still is able to demonstrate discriminative validity. Like any study, the current one is not without its weaknesses. We gathered the RIPS ratings from the patients’ medical records and as a result do not have interrater reliability data for the initial study sample. We did find reliabilities in the excellent range when we had two raters blind to the other raters’ scores in Study 2, so we can say that the RIPS can be reliably scored. The GAF ratings at admission also showed only “fair” reliability levels. This reliability score is in line with GAF reliabilities found in clinical settings, where reliability estimates are between .54 and .65 (Hall, 1995; Jones, Thornicroft, Coffey, & Dunn , 1995; Loevdahl & Friis, 1996; Michels et al., 1996; Soderberg, Tungstrom, & Amelius, 2005). The reliability training for the GAF

366

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy

was not as extensive as some research studies that demonstrate higher reliability for this measure, and future studies should improve upon this training. Another potential weakness of the study was that the psychiatrist who was completing the RIPS assessments was also part of the treatment team. Although team members were kept blind to the RIPS results, team members could potentially glean information from how the psychiatrist presented his findings about patients in team meetings. Given this bias, we would expect to see similar magnitude effect sizes between the RIPS and both the individual therapists’ ratings and the group therapists’ ratings of patients’ engagement because we would expect this bias to affect both raters. We did not find this. In fact, the effect size for the relationship between RIPS and the individual therapists’ ratings of engagement (d = 1.39) was much larger than the group therapists’ ratings (d = .41). Future studies would benefit from having patients assessed on these measures by raters who are not clinicians on the unit and who are truly blind to other patients’ data, including staff impressions, diagnoses, and the like. We used self-report measures that did not include any validity scales and thus did not have a high degree of face validity. This, along with the fact that our sample included adolescents diagnosed with conduct disorder or oppositional defiant disorder, might point to some invalid responses on self-reports. Future research should also incorporate external raters to assess these constructs. Only 45% of the patients admitted to the unit consented to the study. We do not have any clear answer for this because those who consented and those who did not consent to the study (but completed the measures at admission as part of the clinical protocol) were not significantly different in their scores on measures, diagnosis, length of stay, ethnicity, or gender. Many of the patients were admitted against their will, which could lead to refusing to consent. Nevertheless, the current study has some significant strengths in that we used a combination of self-reports, parent reports, and clinician-rated scales. The RIPS was also used in the manner that is consistent with how it is likely to be used in everyday

Vol. 78, No. 4 (Fall 2014)

367

Haggerty et al.

practice. Readiness ratings on the RIPS were directly related to actual engagement in individual therapy. Although further research on the RIPS is certainly needed, the present study suggests that the RIPS is a promising measure for assessing readiness for individual inpatient psychotherapy. This measure can be easily integrated into standard admission interview assessments, requires relatively little time to score, and may be increasingly useful for quantitatively documenting treatment planning decisions. Furthermore, it may provide inpatient researchers with an important tool for controlling for individual differences at admission beyond initial distress. Additional research on the RIPS across inpatient treatment settings (e.g., substance abuse, acute admission, partial hospitalization) and with larger samples is needed to further assess the utility of this measure. References Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth and Families. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (rev. 4th ed.). Washington, DC: Author. Blagys, M., Blais, M., Lechner, E., Rivas-Vazquez, R., Martinez-Arrue, R., Orlandi, E., & Bello, I. (2002). Development of an ultra-brief measure of psychiatric symptom distress: The AT-25. Paper presented at the Society of Personality Assessment Mid-Winter Conference, March 19–23, 2002, San Antonio, TX. Blais, M., Blagys, M. D., Rivas-Vazquez, R., Bello, I., & Sinclair, J. S. (2013). Development and initial validation of a Brief Symptom Measure. Clinical Psychology and Psychotherapy. Advance online publication. doi:10.1002/ccp.1876 Blais, M., Lenderking, W., Baer, L., DeLorell, A., Peets, K., Leahy, L., & Burns, C. (1999). Development and initial validation of a brief mental health outcome measure. Journal of Personality Assessment, 73, 359–373. Blanz, B., & Schmidt, M. H. (2000). Practitioner review: Preconditions and outcome of inpatient treatment in child and adolescent psychiatry. Journal of Child Psychology and Psychiatry, 41(6), 703–712. Chassin, L., Pitts, S. C., DeLucia, C., Todd, M. (1999). A longitudinal study of children of alcoholics: Predicting young adult substance use disorders, anxiety, and depression. Journal of Abnormal Psychology, 108, 106–119.

368

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy Defife, J. A., Drill, R., Nakash, O., & Westen, D. (2010). Agreement between clinician and patient ratings of adaptive functioning and developmental history. American Journal of Psychiatry, 167(12), 1472–1478. Derisley, J., & Reynolds, S. (2000). The transitional stages of change as a predictor of premature termination, attendance and alliance in psychotherapy. British Journal of Clinical Psychology, 39, 371–382. Derogatis, L., & Melisaratos, N. (1983). The Brief Symptom Inventory: A brief introductory report. Psychological Medicine, 13(3), 595–605. Dragomirecka, E., Lenderking, W. R., Motlova, L., Goppoldova, E., & Selepova, P. (2006). A brief mental health outcome measure: Translation and validation of the Czech version of the Schwartz Outcome Scale-10. Quality of Life Research, 15, 307–312. Druss, B. G. (2010). The changing face of U.S. mental health care. American Journal of Psychiatry, 167(12), 1419–1421. Fullerton, C. A., Busch, A. B., Normand, S. T., McGuire, T. G., & Epstein, A. M. (2011). Ten-year trends in quality of care and spending for depression. Archives of General Psychiatry, 68(12), 1218–1226. Giyaur, K., Sharf, J., & Hilsenroth, M. J. (2005). The Capacity for Dynamic Process Scale (CDPS) and patient engagement in opiate addiction treatment. Journal of Nervous and Mental Disease, 193, 833–838. Gough, H. G., & Heilbrun, A. B. (1980). The Adjective Check List manual. Palo Alto, CA: Consulting Psychologists Press. Hacker, F. J. (1962). Treatment motivation. Bulletin of the Menninger Clinic, 26, 288–298. Haggerty, G., Blake, M., Naraine, M., Siefert, C., & Blais, M. A. (2010). Construct validity of the Schwartz Outcome Scale-10: Comparisons to interpersonal distress, adult attachment, alexithymia, the Five Factor Model, romantic relationship length, and ratings of childhood memories. Clinical Psychology and Psychotherapy, 17(1), 44–50. Haggerty, G., Kahoud, D., Walsh, E., Ahmed, Z., & Blais, M. A. (2013). A brief inpatient measure of global psychiatric symptom severity: Initial validation of the Brief Symptom Measure-25 for an adolescent psychiatric inpatient sample. Journal of Nervous and Mental Disease, 201(11), 971-976. Haggerty, G., Walsh, E., Kahoud, D., Forlenza, N., Ahmed, Z., & Ditkowsky, K. (2013). A brief measure of psychological health and well-being: Initial validation of the Schwartz Outcome Scale for an adolescent inpatient sample. Journal of Nervous and Mental Disease, 201(3), 216-221. Hall, R. (1995). Global Assessment of Functioning: A modified scale. Psychosomatics, 36, 267–275. Jones, S. H., Thornicroft, G., Coffey, M., & Dunn, G. (1995). A brief mental health outcome scale—Reliability and validity of the Global

Vol. 78, No. 4 (Fall 2014)

369

Haggerty et al. Assessment of Functioning (GAF). British Journal of Psychiatry, 166, 654–659. Keijsers, G. P. J., Schaap, C. P. D. R., Hoogdiun, C. A. L., Hoogsteyns, B., & de Kamp, E. C. M. (1999). Preliminary results of a new instrument to assess patient motivation for treatment in cognitive-behavior therapy. Behavioral and Cognitive Psychotherapy, 27, 165–179. Keithy, L. J., Samples, S. J., & Strupp, H. H. (1980). Patient motivation as a predictor of process and outcome in psychotherapy. Psychotherapy and Psychosomatics, 33, 87–97. Krause, M. S. (1967). Behavioral indexes of motivation for treatment. Journal of Counseling Psychology, 14, 426–435. Laux, J. M., & Ahern, B. (2003). Concurrent validity of the Schwartz Outcome Scale with a chemically dependent population. Journal of Addictions and Offender Counseling, 24, 2–10. Loevdahl, H., & Friis, S. (1996). Routine evaluation of mental health: Reliable information or worthless “guesstimates”? Acta Psychiatrica Scandinavica, 3, 125–128. McCallum, M., & Piper, W. E. (1997). Psychological mindedness: A contemporary understanding. Mahwah, NJ: Lawrence Erlbaum. Michels, R., Siebel, U., Freyberger, H. J., Stieglitz, R. D., Schaub, R. T., & Dilling, H. (1996). The multiaxial system of ICD-10: Evaluation of a preliminary draft in a multicentric field trial. Psychopathology, 29(6), 347–356. Moras, K. (2002). Might relationship ‘techniques’ extend the reach, efficacy and efficiency of the psychotherapies? The challenge of treatment-resistant outpatients. Journal of Contemporary Psychotherapy, 32, 41–50. Ogrodniczuk, J. S., Joyce, A. S., & Piper, W. E. (2009). Development of the Readiness for Psychotherapy Index. Journal of Nervous and Mental Disease, 197, 427–433. Olfson, M., & Marcus, S. C. (2010). National trends in outpatient psychotherapy. American Journal of Psychiatry, 167, 1456–1463. Petry, N. M., Tennen, H., & Affleck, G. (2000). Stalking the elusive client variable in psychotherapy research. In C. R. Snyder & R. E. Ingram (Eds.), Handbook of psychological change: Processes and practices for the 21st century (pp. 88–108). New York, NY: Wiley. Powers, M. J. (1985). The selection of patients for group therapy. International Journal of Social Psychiatry, 31, 290–297. Rivas-Vazquez, R. A., Rivas-Vazquez, A., Blais, M. A., Rey, G. J., RivasVazquez, F., Jacobo, M., & Carrazana, E. J. (2001). Development of a Spanish version of the Schwartz Outcome Scale-10: A brief mental health outcome measure. Journal of Personality Assessment, 77(3), 436–446.

370

Bulletin of the Menninger Clinic

Readiness for inpatient psychotherapy Rosenbaum, R. L., & Horowitz, M. J. (1983). Motivation for psychotherapy: A factor and conceptual analysis. Psychotherapy, 20, 346–354. Schneider, W., & Klauer, T. (2001). Symptom level, treatment motivation and the effects of inpatient psychotherapy. Psychotherapy Research, 11, 153–167. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428. Smith, S. R., & Lechner, E. (2002, June). Studying inpatient psychotherapy: Methods, measures, and outcome. Paper presented at the meeting of the Society for Psychotherapy Research, Santa Barbara, CA. Soderberg, P., Tungstrom, S., & Armelius, B. A. (2005). Reliability of the Global Assessment of Functioning ratings made by clinical psychiatric staff. Psychiatric Services, 56(4), 434–438. Steiner, D. L., & Norman, G. R. (2008). Health measurement scales: A practical guide to their development and use (4th ed.). New York, NY: Oxford University Press. Thackery, M., Butler, S. F., & Strupp, H .H. (1993). The Capacity for Dynamic Process Scale (CDPS). In M. L. Canfield & J. E. Canfield (Eds.), A collection of psychological scales (pp. 57–63). Bartesville, OK: Research, Evaluation, & Statistics. Truant, G. S. (1999). Assessment of suitability for psychotherapy. II. Assessment based on basic process goals. American Journal of Psychotherapy, 53, 17–34. Westen, D., Dutra, L., & Shedler, J. (2005). Assessing adolescent personality pathology. British Journal of Psychiatry, 186(3), 227–238. Westen, D., & Harnden-Fischer, J. (2001). Classifying eating disorders by personality profiles: Bridging the chasm between Axis I and Axis II. American Journal of Psychiatry, 158, 1767–1771. Westen, D., & Shedler, J. (1999). Revising and assessing Axis II, Part 1: Developing a clinically and empirically valid assessment method. American Journal of Psychiatry, 156, 258–272. Westen, D., Shedler, J., Durrett, C., Glass, S., & Martens, A. (2003). Personality diagnosis in adolescence: DSM-IV axis II diagnoses and an empirically derived alternative. American Journal of Psychiatry, 160, 952–966. Westen, D., & Weinberger, J. (2004). When clinical description becomes statistical prediction. American Psychologist, 59, 595–613. Woody, G. E., McLellan, A. T., Luborsky, L., O’Brien, C. P. (1985). Sociopathy and psychotherapy outcome. Archives of General Psychiatry, 42, 1081–1086. Young, J. L., Waehler, C. A., Laux, J. M., McDaniel, P. S., & Hilsenroth, M. J. (2003). Four studies extending the utility of the Schwartz Outcome Scale (SOS-10). Journal of Personality Assessment, 80(2), 130–138.

Vol. 78, No. 4 (Fall 2014)

371

Haggerty et al.

Appendix. The Readiness for Inpatient Psychotherapy Scale (RIPS) Non-therapist rater© Instructions: Please rate this patient on the following variables. 1. Patient is cooperative and able to be interviewed by treatment team. 0

1

2

3

4

Not at all

5 Totally

2. Patient is interested in psychotherapy. 0

1

2

3

4

Not at all

5 Totally

3. Patient recognizes the severity of his/her problem(s). 0

1

2

3

Not at all

4

5

Totally

Totally

4. Patient can focus on relevant topics during rounds or individual interviews. 0

1

2

3

Not at all

4

5

Totally

Totally

5. The patient thinks about how he/she might have contributed to his/her problem(s). 0

1

2

3

Not at all

4

5

Totally

Totally

4

5

Totally

Totally

6. Patient relates appropriately to the treatment team. 0

1

2

3

Not at all

7. Patient is able to talk about difficult subjects rounds or other interviews. 0

1

2

3

Not at all

4

5

Totally

Totally

8. Patient has an open and accepting attitude toward hospital treatment. 0 Not at all

372

1

2

3

4

5 Totally

Bulletin of the Menninger Clinic

Copyright of Bulletin of the Menninger Clinic is the property of Guilford Publications Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.