Community Ment Health J DOI 10.1007/s10597-014-9708-9
BRIEF REPORT
Predictors of Recovery-Oriented Competencies Among Mental Health Professionals in One Community Mental Health System Jennifer Stuber • Anita Rocha • Ann Christian David Johnson
•
Received: 16 November 2012 / Accepted: 30 January 2014 Ó Springer Science+Business Media New York 2014
Abstract A survey of 813 mental health professionals serving adults with severe mental illness clustered in 25 community mental health centers assessed the extent to which mental health professionals possess clinical competencies that support recovery and the predictors of these competencies. The results suggest there is room for improvement in recovery-oriented competencies. In-depth professional training in recovery, greater job variety, more years practicing in mental health, participation on an intensive case management team, and perceptions of workplace recovery culture were predictors of recoveryoriented competencies. Prioritization of on-going professional, worker retention, and management strategies that incorporate a team approach to treatment and improvements in workplace recovery culture may potentially increase recovery-oriented clinical practice. Keywords Recovery Competencies Mental health professionals
J. Stuber (&) University of Washington School of Social Work, 4101 15th Avenue NE, Seattle, WA 98105-6299, USA e-mail: [email protected] A. Rocha Center for Studies in Demography and Ecology, University of Washington, Seattle, WA, USA A. Christian Washington Community Mental Health Council, Seattle, WA, USA D. Johnson Navos Mental Health Solutions, Seattle, WA, USA
Introduction Recovery or the development of new meaning and purpose as one grows with the experience of living with a mental illness is the dominant paradigm influencing the delivery of public community mental health services (Hogan 2003; Anthony 1993). The extent to which community mental health professionals are supportive of the recovery paradigm in their clinical practice is unknown. Survey based assessments of competencies are one strategy to assess if clinicians practice with a recovery-orientation (Cradock et al. 2001; Chinman et al. 2003). Competencies are the knowledge, skills, values and attitudes of clinicians, acquired through pre-service education, in-service training, and work experience in the healthcare industry, that are needed to deliver high-quality care in specific realms of clinical practice and are a major determinant of provider performance (Coursey et al. 2000a, b). Community mental health professionals are on the front-lines implementing policy and procedures, providing therapy, case management services, medication evaluation and management, and access to other social services for people with serious mental illnesses. These professionals can directly impact how clients conceptualize recovery at a time when they are vulnerable. Thus, it is important to study the extent to which they report clinical competencies in recovery and to identify predictors of these competencies. A framework is useful for thinking about predictors of recovery-oriented clinical practice in community mental health. Such a framework should include: characteristics of the provider as a person (e.g., demographic, personal experiences), attributes of the job such as caseload, pre and in-service training and work-related experiences, contextual factors about the work-place, policies and available resources. In the current study, we examine many of these
123
Community Ment Health J
factors in relationship to provider competencies in recovery. The following hypotheses guided the study: (1) mental health professionals who have a mental illness will report greater clinical competencies than those without a personal experience; (2) in-service training in the recovery model will be positively associated with competencies; (3) high administrative burden and caseloads will be associated with lowered competencies while jobs with high levels of control and satisfaction will be associated with greater competencies; (4) the providers’ perceived workplace commitment to a recovery-culture will be a predictor of his/her own recovery-oriented competencies.
Methods Twenty-five community mental health agencies that collectively provide eighty percent of all outpatient services to adult clients receiving publically funded mental health services in Washington State participated in the study. Within the agencies 49 % of the mental health professionals serving adult clients with mental illness completed a singlesession, web-based 35 min survey. The mean number of staff responses per agency was 24.2 (SD = 17.5).The minimum agency response rate was 21 % and the maximum agency response rate was 100 %. The data collection period lasted for 90 days and providers were sent a $15 Gift Card as compensation for participating in the study. Study procedures were approved by the Institutional Review Board (IRB) at the WA State Department of Social and Health Services and by reciprocal agreement, by the IRB at the University of Washington. The outcome measure for this study, recovery-oriented clinical competencies, was assessed by the competency assessment instrument (CAI), a self-report measure developed ‘‘to represent rehabilitation, recovery, and empowerment principles found to be critical to the treatment of clients with serious mental illnesses, but often lacking in public mental health settings’’ (Chinman et al. 2003). The original CAI included 55 items—a combination of vignettes, Likert scales and multiple-choice items all requesting a numerical response on a 5 point scale—and contained fifteen subscales. The CAI was demonstrated to have good internal consistency, test–retest reliability and validity. A total score was created by adding together all 15 subscales into one score with a Cronbach alpha of 0.90 for the CAI scale (Chinman et al. 2003). In this study, items for 12 of the 15 subscales were assessed and determined using factor analysis. A total score was calculated with a Cronbach alpha of 0.82 with items coded so that higher values signal greater competency. The independent variables were as follows. To assess personal experience with mental illness providers were
123
asked, if they have ever been diagnosed by another professional as having a mental illness? Pre-service training was measured by education coded as less than a 4 year college degree, a 4 year college degree, Master’s degree, a doctoral degree, a medical school degree or other. Job title was coded with these categories: program director/manager, case manager (Bachelor’s and Master’s level), psychiatric nurse, psychiatrist, therapist/psychologist, peer support counselor, case aide, residential services counselor, employment specialist, co-occurring disorders specialist and other. The survey asked if the mental health professional was part of an intensive case management team (yes/ no), and the number of total years worked in a mental health care setting (coded continuously). Providers were asked if in the last 12 months if they had attended any inservice training(s) with content on the topic of recovery and how many hours of training on these topics they received coded as none, less than a half day, between a half and a full day, between 1 and 2 days, between 2 and 3 days, and more than 3 full days. To assess job control and demands several measures were included in the survey. Satisfaction with the agency was assessed by four items: (1) in general, I like working for this organization; (2) in general, I like working with my coworkers, (3) in general, I am satisfied with my job, and, (4) I am satisfied with the chances I have had to move into higher positions in this organization (Koeske et al. 1994). The question was asked, how long the mental health professional intends to stay in his/her current position? (coded as less than 1 year, less than 3 years, more than 3 years). Job control was measured with three questions about the amount of discretion, variety and challenge in job tasks scored from 1 to 10 with 1 being ‘‘none’’ and 10 being ‘‘a lot’’ (Karasek and Theorell 1990). Questions were asked about the number of clients on the mental health professionals’ caseload (coded continuously). Study participants were also asked, on average, what percentage of time they spend in direct contact with clients? (in-person or on the telephone) and what percentage of time they spend on documentation requirements?. To assess workplace recovery culture, 16 of 36 questions from the recovery self-assessment instrument (RSA) were used to gauge perceptions of the degree to which programs implement recovery-oriented practices (O’Connell et al. 2005). All items were asked on a 4 point response scale ranging from strongly disagree to strongly agree and items were scored so that higher scores signified more positive responses. To create the measure, we added together the individual items to arrive at a total score. The 16 item scale in our study had a Cronbach’s alpha of 0.69 and its correlation to the CAI was 0.41. Other characteristics controlled for in the study were providers’ gender, marital status (married, divorced/
Community Ment Health J
widowed/separated, or never married), race/ethnicity (Black, white, American Indian or Alaska Native, Asian, or Hispanic), age (continuous), household income (coded in categories less than $40,000, between $40,001 and $60,000, between $60,001 and $80,000, between $80,001 and $100,000 and greater than $100,000). Also, where the provider agency was located—in an urban versus a rural part of the state and the specific local administrative entity where the agency was located—was controlled for. For the independent variables, those with more than 5 percent missing data were examined for whether they were missing at random. Multiple imputation was used to handle missing data to correct for systematic inferential failings produced by either ignoring missing data or using a single imputation approach (Little and Rubin 2002). Missing data were filled-in with plausible values using the MI procedure in SAS V9.2 using a Markov Chain Monte Carlo method for arbitrary missing data. This method generated pseudorandom draws from a multivariate normal distribution via Markov chains. This process was completed five times resulting in the development of five data sets with imputed values. Estimation procedures were used separately on each dataset and the results were combined using the MIANALYZE procedure in SAS V9.2 to obtain composite estimates and standard errors. Descriptive analyses provided information about the independent and outcome variables. To present descriptive results for the CAI, each subscale was normalized to have a zero mean and a standard deviation of one. Bivariate analyses were used to determine if there were significant relationships between each independent variable and the CAI. A multivariate, 2 level hierarchical linear random intercept model was built that incorporated significant variables from the bivariate analyses. This modeling strategy accounted for the nested data structure and the non-independence of mental health professionals within the same organization (Raudenbush and Bryk 2002).
Results Descriptive The surveyed mental health professionals were representative of the gender, race/ethnicity and job titles of the workforce of mental health professionals serving adults in outpatient settings in WA State’s public mental health setting that also mirror national trends. The sample was comprised of predominately white women who reported an annual household income of less than $100,000. The mean age of the sample was 44. Thirty-three percent of the survey mental health professionals indicated they had been told by a mental health professional that they have a mental
illness. More than half of the sample reported having an advanced degree predominately, a Master’s degree (61 %). Only 6 percent of the sample had a medical or doctoral degree. One quarter of the surveyed mental health professionals held the job title of case manager or clinical case manager. Other job titles represented were therapists and psychologists (27 %), program directors or managers (11 %), psychiatric nurses (6 %), psychiatrists (3 %), peer support counselors (1 %), case aides (2 %), residential services counselors (2 %), employment specialists (2 %), co-occurring disorders specialists (3 %) and other (17 %). Eleven percent of the surveyed mental health professionals indicated that they were part of an intensive case management team and the mean number of years working in mental health settings was 13. Sixty-two percent of the surveyed mental health professionals received training on recovery or peer support in the last year. Among those who received training, only nine percent received more than 8 h of training in the prior year. The vast majority of surveyed mental health professionals reported being satisfied with their agency (means on individual satisfaction measures ranged from 0.58 to 0.95 on a 0 to 1 scale). A third of mental health professionals indicated an intention to stay in their current job for more than 3 years. Levels of perceived discretion, variety and challenged in the job ranged from 7.0 to 7.8 on a 10 point scale. The average caseload size was 28. Clinician respondents indicated, on average, they spent between 50 and 60 % of their time on the job in direct contact with clients and 30–40 % of their time on documentation requirements imposed by county, state and federal agencies. The mean RSA was 43 out of a possible score of 64. For recovery-oriented clinical competencies, the mean score was 117.13 out of a total possible score of 172. Of the 12 subscales, the three competencies mental health professionals scored the highest on included: recognition of the importance of utilizing evidence based practices (adjusted mean = 0.86), awareness of stigma and discrimination experienced by people living with mental illnesses (adjusted mean 0.84), and the importance of incorporating client preferences into treatment planning (adjusted mean = 0.78). The three competencies where mental health professionals scored the lowest: the ability to provide intensive case management for clients’ on their caseload (adjusted mean = 0.34), helping clients identify stressors that trigger mental health symptoms (adjusted mean = 0.37), and to identify and actualize personal goals (adjusted mean = 0.38). Bivariate Predictors of the CAI There were significant associations between many of variables hypothesized to associated with the CAI. Having
123
Community Ment Health J
a 4 year college degree was positively associated with the CAI as compared to having less than a 4 year college degree (p = 0.325). No other levels of education were significantly associated with the CAI. Relative to program managers, all other job titles had significantly lower CAI scores although employment specialists had higher CAI scores than program managers (p = 0.544). Having any training (p = 0.580) and having more hours of training was positively and linearly associated with the CAI. More total years of experience in mental health settings were positively associated with the CAI (p = 0.018). In addition, mental health professionals on intensive case management teams had greater recovery competencies as compared to those without such comparable experiences (p = 0.421). The four variables measuring job satisfaction were all positively associated with the CAI. Those who perceived more discretion, variety and challenge were more likely to report having greater recovery-oriented clinical competencies. Spending less time on documentation was positively associated with having greater recovery-oriented competencies (p = 0.81). Mental health professionals who perceived their agencies to have a greater recovery-orientation reported having more recovery competencies (p = 0.49).
Multivariate Predictors of the CAI The variables mentioned here are the strongest predictors of the CAI as their associations persist in a hierarchical linear multivariate model. Employment specialists as compared to program managers were more likely to score higher on the CAI (p = 0.01). Training on the recovery model remained a significant predictor of the CAI, but only for those with more in-depth training between 2 and 3 days (p = 0.007) or 3? more days in the past year (p = 0.002). Mental health professionals on intensive case management teams reported greater recovery oriented competencies than those not on teams (p = 0.004). More total years of experience in mental health was associated with a higher CAI score (p = 0.0003). Clinicians who reported more variety on the job were more likely to report recovery oriented competencies than those with less variety on their job (p = 0.003). Finally, the RSA (p \ 0.0001) measuring workplace recovery culture was a strong predictor of the CAI in the final multivariate model.
Discussion The results support there is a need room for improvement in recovery-oriented competencies and provide insight about how to improve recovery-oriented clinical competencies.
123
The mean overall score on the CAI is less than two-thirds of the total possible score. The three competencies where mental health professionals scored the lowest pertained to critical areas of recovery practice such as helping clients identify stressors that trigger mental health symptoms, and to identify and actualize personal goals. As far as improving competencies, many of our initial hypotheses were supported by this analysis although having a personal experience with mental illness was not associated with increased competency nor was high administrative burden and caseload size associated with lowered competencies. We focus in the discussion on highlighting variables significant in the final hierarchical multivariate model. These variables include job title, having more in-depth, in-service training, years on the job, participation on an intensive case management team, job variety, and perceptions of workplace recovery culture. We did not find major differences in recovery-oriented competencies by job title (e.g., case manager versus therapist) once other factors are taken into account, but we are limited by small numbers of certain types of professionals. Employment specialists do stand out in the final multivariate model as having greater recovery-oriented competencies as compared to other job titles. This may be a function of having a job description that is explicitly tied to supporting clients’ rehabilitation, independence and personal fulfillment through employment. Receiving in-depth, in-service training on recovery in the prior year was positively associated with the recoveryoriented competencies of mental health professionals. A few prior studies have shown that service providers’ attitudes, hopefulness and knowledge can improve with formal training on recovery (Pollard et al. 2008; Meehan and Glover 2009; Crowe et al. 2006). This study adds to this growing literature, but it cannot rule out the possibility that professionals who have greater clinical competencies self-select into training opportunities. Further research is needed to identify the kinds of in-service training on the recovery model most beneficial to professionals. The need for more in-depth training as opposed to one-time educational efforts is supported by this study. The mental health professionals in this study with more total years practicing in mental health settings were more likely to report greater competencies in recovery than colleagues with fewer years of experience. On the one hand, we might expect that those practicing for many years would have a more difficult time adapting to a new paradigm or experience burn out from the job. These results suggest the opposite is true. Providers with more years of experience in the mental health field have higher recovery competencies perhaps because they have gained perspective through longer term relationships with clients or through continuing education and supervision.
Community Ment Health J
The finding that mental health professionals who are part of intensive case management teams report higher levels of recovery competencies may be because these teams provide mental health and services, around-the-clock, to clients within their communities. While intensive case management teams may also practice with assertive elements that are less consistent with the recovery model (Drake and Burns 1995; Burns and Santos 1995), these results suggest how intensive case management teams are being implemented in these agencies may be more consistent with a recovery model. Variety on the job was a significant predictor of recovery competencies indicating the provision of recovery-oriented clinical services may be enabled by greater flexibility in job descriptions and the ability to step outside traditional clinical realms to help clients. Perceived workplace recovery culture was a strong predictor of the recovery competencies of mental health professionals in this study. This measure was comprised of various items that reflect the activities and recovery norms of the agency. Management strategies that seek to shift the recovery culture of the agency may improve providers’ recovery-oriented competencies according to these results. According to Jacobson and Curtis (2000), creating a recovery-wide agency culture begins with changing assumptions about clients from a deficit focus to a strengths-based focus, making visible stories of hope and, creating opportunities for healing based on a culture of acceptance and respect that permeates throughout the organization and its leadership. Organizations also have to fundamentally shift how they share power and responsibility providing opportunities for clients and providers to learn in a safe environment. We note that while personal experiences with mental illnesses were common among the providers in this sample, they did not emerge as a significant predictor of competencies. Type of graduate education beyond college was also not associated with competencies in the final model. These findings suggest recovery competencies may be learned on the job as a mental health professional. Variables controlling for geographic context were also not prominent predictors of mental health professionals’ recovery competencies. However, the fact that the workplace recovery culture, as assessed by individual mental health professionals, was strongly associated with the CAI indicates organizational culture may play a role in how mental health professionals practice with regard to recovery. Future research should discern other specific agency level predictors of competencies. Certain limitations of this study should be noted. First, there were significant challenges to obtaining high response rates among mental health professionals surveyed in a community mental health setting due to competing demands on providers’ time. Significant effort was put into
establishing a sampling frame so that a response rate and the representativeness of the sample could be established. Second, because the survey is cross-sectional, we cannot make causal statements about the relationships between independent and dependent variables. For instance, it is possible that professionals with greater competencies report also are more inclined to perceive a more recoveryoriented workplace. Causality cannot be determined. Third, some of the measures i.e., agency recovery culture and the CAI had to be shortened to accommodate the busy clinical setting in which the data were collected. Finally, the specific competencies measured in this study have not yet been validated against actual clinical practice. Thus, it is possible they represent a preference for recovery-oriented practice rather than a true measure even though the items were written to minimize social desirability bias. Despite these limitations, this study identifies several factors that are potentially important in transforming mental health systems towards a recovery paradigm, including an intensive focus on the recovery culture of agencies, offering training on the recovery model at a deep level, creating job descriptions with greater variety to help clients, the retention of mental health professionals for longer periods of time, and increasing the availability of and encouraging professionals to be part of intensive case management teams. Acknowledgments Data collection was funded by a Mental Health State Incentive Grant Award No. 6 U79 SM57648 from the Substance Abuse and Mental Health Services Administration (SAMHSA). Data analysis was funded by the National Institute of Mental Health (#R03 MH 086791-02).
References Anthony, W. A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990’s. Psychosocial Rehabilitation Journal, 16(4), 11–23. Burns, B. J., & Santos, A. B. (1995). Assertive community treatment an update of randomized trials. Psychiatric Services, 46(7), 669–675. Chinman, M., Young, A. S., Rowe, M., Forquer, S., Knight, E., & Miller, A. (2003). An instrument to assess competencies of providers treating severe mental illness. Mental Health Services Research, 5(2), 97–108. Coursey, R. D., Curtis, L., Marsh, D. T., Campbell, J., Harding, C., Spaniol, L., et al. (2000a). Competencies for direct service staff who work with adults with severe mental illnesses in outpatient public mental health/managed care services. Psychiatric Rehabilitation Journal, 23(4), 370–377. Coursey, R. D., Curtis, L., Marsh, D. T., Campbell, J., Harding, C., Spaniol, L., et al. (2000b). Competencies for direct service staff who work with adults with severe mental illnesses: Specific knowledge, skills and bibliography. Psychiatric Rehabilitation Journal, 23(4), 378–392. Cradock, J., Young, A. S., & Sullivan, G. (2001). The accuracy of medical record documentation in schizophrenia. Journal of Behavioral Health Services and Research, 28(4), 456–465.
123
Community Ment Health J Crowe, T. P., Deane, F. P., Oades, L. G., Caputi, P., & Morland, K. G. (2006). Effectiveness of a collaborative recovery training program in Australia in promoting positive views about recovery. Psychiatric Services, 57(10), 1497–1500. Drake, R. E., & Burns, B. J. (1995). Special section on assertive community treatment an introduction. Psychiatric Services, 46(7), 667–668. Hogan, M. F. (2003). New Freedom Commission Report: The President’s New Freedom Commission: Recommendations to transform mental health care in America. Psychiatric Services, 54, 1467–1474. Jacobson, N., & Curtis, L. (2000). Recovery as policy in mental health services: Strategies, emerging from the States. Psychiatric Rehabilitation Journal, 23(4), 333–341. Karasek, R., & Theorell, T. (1990). Healthy work: Stress, productivity and the reconstruction of working life. New York: Basic Books. Koeske, G., Kirk, S., Koeske, R., & Rauktis, M. (1994). Measuring the Monday blues: Validation of a job satisfaction scale for the human services. Social Work Research, 18(1), 27–35.
123
Little, R. J., Rubin, D. B. (2002). Statistical analysis with missing data, 2nd edition (pp. 2–4, 85–6), Hoboken: NJL John Wiley & Sons. Meehan, T., & Glover, H. (2009). Using the recovery knowledge inventory (RKI) to assess the effectiveness of a consumer-led recovery training program for service providers. Psychiatric Rehabilitation Journal, 32(3), 223–226. O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L. (2005). From rhetoric to routine: Assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric Rehabilitation Journal, 28, 378–386. Pollard, L., Gelbard, Y., Levy, G., & Gelkopf, M. (2008). Examining attitudes, beliefs and knowledge of effective practices in psychiatric rehabilitation in a hospital setting. Psychiatric Rehabilitation, 32(2), 124–127. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed., p. 100). Thousand Oaks, CA: Sage Publications, Inc.
BRIEF REPORT
Predictors of Recovery-Oriented Competencies Among Mental Health Professionals in One Community Mental Health System Jennifer Stuber • Anita Rocha • Ann Christian David Johnson
•
Received: 16 November 2012 / Accepted: 30 January 2014 Ó Springer Science+Business Media New York 2014
Abstract A survey of 813 mental health professionals serving adults with severe mental illness clustered in 25 community mental health centers assessed the extent to which mental health professionals possess clinical competencies that support recovery and the predictors of these competencies. The results suggest there is room for improvement in recovery-oriented competencies. In-depth professional training in recovery, greater job variety, more years practicing in mental health, participation on an intensive case management team, and perceptions of workplace recovery culture were predictors of recoveryoriented competencies. Prioritization of on-going professional, worker retention, and management strategies that incorporate a team approach to treatment and improvements in workplace recovery culture may potentially increase recovery-oriented clinical practice. Keywords Recovery Competencies Mental health professionals
J. Stuber (&) University of Washington School of Social Work, 4101 15th Avenue NE, Seattle, WA 98105-6299, USA e-mail: [email protected] A. Rocha Center for Studies in Demography and Ecology, University of Washington, Seattle, WA, USA A. Christian Washington Community Mental Health Council, Seattle, WA, USA D. Johnson Navos Mental Health Solutions, Seattle, WA, USA
Introduction Recovery or the development of new meaning and purpose as one grows with the experience of living with a mental illness is the dominant paradigm influencing the delivery of public community mental health services (Hogan 2003; Anthony 1993). The extent to which community mental health professionals are supportive of the recovery paradigm in their clinical practice is unknown. Survey based assessments of competencies are one strategy to assess if clinicians practice with a recovery-orientation (Cradock et al. 2001; Chinman et al. 2003). Competencies are the knowledge, skills, values and attitudes of clinicians, acquired through pre-service education, in-service training, and work experience in the healthcare industry, that are needed to deliver high-quality care in specific realms of clinical practice and are a major determinant of provider performance (Coursey et al. 2000a, b). Community mental health professionals are on the front-lines implementing policy and procedures, providing therapy, case management services, medication evaluation and management, and access to other social services for people with serious mental illnesses. These professionals can directly impact how clients conceptualize recovery at a time when they are vulnerable. Thus, it is important to study the extent to which they report clinical competencies in recovery and to identify predictors of these competencies. A framework is useful for thinking about predictors of recovery-oriented clinical practice in community mental health. Such a framework should include: characteristics of the provider as a person (e.g., demographic, personal experiences), attributes of the job such as caseload, pre and in-service training and work-related experiences, contextual factors about the work-place, policies and available resources. In the current study, we examine many of these
123
Community Ment Health J
factors in relationship to provider competencies in recovery. The following hypotheses guided the study: (1) mental health professionals who have a mental illness will report greater clinical competencies than those without a personal experience; (2) in-service training in the recovery model will be positively associated with competencies; (3) high administrative burden and caseloads will be associated with lowered competencies while jobs with high levels of control and satisfaction will be associated with greater competencies; (4) the providers’ perceived workplace commitment to a recovery-culture will be a predictor of his/her own recovery-oriented competencies.
Methods Twenty-five community mental health agencies that collectively provide eighty percent of all outpatient services to adult clients receiving publically funded mental health services in Washington State participated in the study. Within the agencies 49 % of the mental health professionals serving adult clients with mental illness completed a singlesession, web-based 35 min survey. The mean number of staff responses per agency was 24.2 (SD = 17.5).The minimum agency response rate was 21 % and the maximum agency response rate was 100 %. The data collection period lasted for 90 days and providers were sent a $15 Gift Card as compensation for participating in the study. Study procedures were approved by the Institutional Review Board (IRB) at the WA State Department of Social and Health Services and by reciprocal agreement, by the IRB at the University of Washington. The outcome measure for this study, recovery-oriented clinical competencies, was assessed by the competency assessment instrument (CAI), a self-report measure developed ‘‘to represent rehabilitation, recovery, and empowerment principles found to be critical to the treatment of clients with serious mental illnesses, but often lacking in public mental health settings’’ (Chinman et al. 2003). The original CAI included 55 items—a combination of vignettes, Likert scales and multiple-choice items all requesting a numerical response on a 5 point scale—and contained fifteen subscales. The CAI was demonstrated to have good internal consistency, test–retest reliability and validity. A total score was created by adding together all 15 subscales into one score with a Cronbach alpha of 0.90 for the CAI scale (Chinman et al. 2003). In this study, items for 12 of the 15 subscales were assessed and determined using factor analysis. A total score was calculated with a Cronbach alpha of 0.82 with items coded so that higher values signal greater competency. The independent variables were as follows. To assess personal experience with mental illness providers were
123
asked, if they have ever been diagnosed by another professional as having a mental illness? Pre-service training was measured by education coded as less than a 4 year college degree, a 4 year college degree, Master’s degree, a doctoral degree, a medical school degree or other. Job title was coded with these categories: program director/manager, case manager (Bachelor’s and Master’s level), psychiatric nurse, psychiatrist, therapist/psychologist, peer support counselor, case aide, residential services counselor, employment specialist, co-occurring disorders specialist and other. The survey asked if the mental health professional was part of an intensive case management team (yes/ no), and the number of total years worked in a mental health care setting (coded continuously). Providers were asked if in the last 12 months if they had attended any inservice training(s) with content on the topic of recovery and how many hours of training on these topics they received coded as none, less than a half day, between a half and a full day, between 1 and 2 days, between 2 and 3 days, and more than 3 full days. To assess job control and demands several measures were included in the survey. Satisfaction with the agency was assessed by four items: (1) in general, I like working for this organization; (2) in general, I like working with my coworkers, (3) in general, I am satisfied with my job, and, (4) I am satisfied with the chances I have had to move into higher positions in this organization (Koeske et al. 1994). The question was asked, how long the mental health professional intends to stay in his/her current position? (coded as less than 1 year, less than 3 years, more than 3 years). Job control was measured with three questions about the amount of discretion, variety and challenge in job tasks scored from 1 to 10 with 1 being ‘‘none’’ and 10 being ‘‘a lot’’ (Karasek and Theorell 1990). Questions were asked about the number of clients on the mental health professionals’ caseload (coded continuously). Study participants were also asked, on average, what percentage of time they spend in direct contact with clients? (in-person or on the telephone) and what percentage of time they spend on documentation requirements?. To assess workplace recovery culture, 16 of 36 questions from the recovery self-assessment instrument (RSA) were used to gauge perceptions of the degree to which programs implement recovery-oriented practices (O’Connell et al. 2005). All items were asked on a 4 point response scale ranging from strongly disagree to strongly agree and items were scored so that higher scores signified more positive responses. To create the measure, we added together the individual items to arrive at a total score. The 16 item scale in our study had a Cronbach’s alpha of 0.69 and its correlation to the CAI was 0.41. Other characteristics controlled for in the study were providers’ gender, marital status (married, divorced/
Community Ment Health J
widowed/separated, or never married), race/ethnicity (Black, white, American Indian or Alaska Native, Asian, or Hispanic), age (continuous), household income (coded in categories less than $40,000, between $40,001 and $60,000, between $60,001 and $80,000, between $80,001 and $100,000 and greater than $100,000). Also, where the provider agency was located—in an urban versus a rural part of the state and the specific local administrative entity where the agency was located—was controlled for. For the independent variables, those with more than 5 percent missing data were examined for whether they were missing at random. Multiple imputation was used to handle missing data to correct for systematic inferential failings produced by either ignoring missing data or using a single imputation approach (Little and Rubin 2002). Missing data were filled-in with plausible values using the MI procedure in SAS V9.2 using a Markov Chain Monte Carlo method for arbitrary missing data. This method generated pseudorandom draws from a multivariate normal distribution via Markov chains. This process was completed five times resulting in the development of five data sets with imputed values. Estimation procedures were used separately on each dataset and the results were combined using the MIANALYZE procedure in SAS V9.2 to obtain composite estimates and standard errors. Descriptive analyses provided information about the independent and outcome variables. To present descriptive results for the CAI, each subscale was normalized to have a zero mean and a standard deviation of one. Bivariate analyses were used to determine if there were significant relationships between each independent variable and the CAI. A multivariate, 2 level hierarchical linear random intercept model was built that incorporated significant variables from the bivariate analyses. This modeling strategy accounted for the nested data structure and the non-independence of mental health professionals within the same organization (Raudenbush and Bryk 2002).
Results Descriptive The surveyed mental health professionals were representative of the gender, race/ethnicity and job titles of the workforce of mental health professionals serving adults in outpatient settings in WA State’s public mental health setting that also mirror national trends. The sample was comprised of predominately white women who reported an annual household income of less than $100,000. The mean age of the sample was 44. Thirty-three percent of the survey mental health professionals indicated they had been told by a mental health professional that they have a mental
illness. More than half of the sample reported having an advanced degree predominately, a Master’s degree (61 %). Only 6 percent of the sample had a medical or doctoral degree. One quarter of the surveyed mental health professionals held the job title of case manager or clinical case manager. Other job titles represented were therapists and psychologists (27 %), program directors or managers (11 %), psychiatric nurses (6 %), psychiatrists (3 %), peer support counselors (1 %), case aides (2 %), residential services counselors (2 %), employment specialists (2 %), co-occurring disorders specialists (3 %) and other (17 %). Eleven percent of the surveyed mental health professionals indicated that they were part of an intensive case management team and the mean number of years working in mental health settings was 13. Sixty-two percent of the surveyed mental health professionals received training on recovery or peer support in the last year. Among those who received training, only nine percent received more than 8 h of training in the prior year. The vast majority of surveyed mental health professionals reported being satisfied with their agency (means on individual satisfaction measures ranged from 0.58 to 0.95 on a 0 to 1 scale). A third of mental health professionals indicated an intention to stay in their current job for more than 3 years. Levels of perceived discretion, variety and challenged in the job ranged from 7.0 to 7.8 on a 10 point scale. The average caseload size was 28. Clinician respondents indicated, on average, they spent between 50 and 60 % of their time on the job in direct contact with clients and 30–40 % of their time on documentation requirements imposed by county, state and federal agencies. The mean RSA was 43 out of a possible score of 64. For recovery-oriented clinical competencies, the mean score was 117.13 out of a total possible score of 172. Of the 12 subscales, the three competencies mental health professionals scored the highest on included: recognition of the importance of utilizing evidence based practices (adjusted mean = 0.86), awareness of stigma and discrimination experienced by people living with mental illnesses (adjusted mean 0.84), and the importance of incorporating client preferences into treatment planning (adjusted mean = 0.78). The three competencies where mental health professionals scored the lowest: the ability to provide intensive case management for clients’ on their caseload (adjusted mean = 0.34), helping clients identify stressors that trigger mental health symptoms (adjusted mean = 0.37), and to identify and actualize personal goals (adjusted mean = 0.38). Bivariate Predictors of the CAI There were significant associations between many of variables hypothesized to associated with the CAI. Having
123
Community Ment Health J
a 4 year college degree was positively associated with the CAI as compared to having less than a 4 year college degree (p = 0.325). No other levels of education were significantly associated with the CAI. Relative to program managers, all other job titles had significantly lower CAI scores although employment specialists had higher CAI scores than program managers (p = 0.544). Having any training (p = 0.580) and having more hours of training was positively and linearly associated with the CAI. More total years of experience in mental health settings were positively associated with the CAI (p = 0.018). In addition, mental health professionals on intensive case management teams had greater recovery competencies as compared to those without such comparable experiences (p = 0.421). The four variables measuring job satisfaction were all positively associated with the CAI. Those who perceived more discretion, variety and challenge were more likely to report having greater recovery-oriented clinical competencies. Spending less time on documentation was positively associated with having greater recovery-oriented competencies (p = 0.81). Mental health professionals who perceived their agencies to have a greater recovery-orientation reported having more recovery competencies (p = 0.49).
Multivariate Predictors of the CAI The variables mentioned here are the strongest predictors of the CAI as their associations persist in a hierarchical linear multivariate model. Employment specialists as compared to program managers were more likely to score higher on the CAI (p = 0.01). Training on the recovery model remained a significant predictor of the CAI, but only for those with more in-depth training between 2 and 3 days (p = 0.007) or 3? more days in the past year (p = 0.002). Mental health professionals on intensive case management teams reported greater recovery oriented competencies than those not on teams (p = 0.004). More total years of experience in mental health was associated with a higher CAI score (p = 0.0003). Clinicians who reported more variety on the job were more likely to report recovery oriented competencies than those with less variety on their job (p = 0.003). Finally, the RSA (p \ 0.0001) measuring workplace recovery culture was a strong predictor of the CAI in the final multivariate model.
Discussion The results support there is a need room for improvement in recovery-oriented competencies and provide insight about how to improve recovery-oriented clinical competencies.
123
The mean overall score on the CAI is less than two-thirds of the total possible score. The three competencies where mental health professionals scored the lowest pertained to critical areas of recovery practice such as helping clients identify stressors that trigger mental health symptoms, and to identify and actualize personal goals. As far as improving competencies, many of our initial hypotheses were supported by this analysis although having a personal experience with mental illness was not associated with increased competency nor was high administrative burden and caseload size associated with lowered competencies. We focus in the discussion on highlighting variables significant in the final hierarchical multivariate model. These variables include job title, having more in-depth, in-service training, years on the job, participation on an intensive case management team, job variety, and perceptions of workplace recovery culture. We did not find major differences in recovery-oriented competencies by job title (e.g., case manager versus therapist) once other factors are taken into account, but we are limited by small numbers of certain types of professionals. Employment specialists do stand out in the final multivariate model as having greater recovery-oriented competencies as compared to other job titles. This may be a function of having a job description that is explicitly tied to supporting clients’ rehabilitation, independence and personal fulfillment through employment. Receiving in-depth, in-service training on recovery in the prior year was positively associated with the recoveryoriented competencies of mental health professionals. A few prior studies have shown that service providers’ attitudes, hopefulness and knowledge can improve with formal training on recovery (Pollard et al. 2008; Meehan and Glover 2009; Crowe et al. 2006). This study adds to this growing literature, but it cannot rule out the possibility that professionals who have greater clinical competencies self-select into training opportunities. Further research is needed to identify the kinds of in-service training on the recovery model most beneficial to professionals. The need for more in-depth training as opposed to one-time educational efforts is supported by this study. The mental health professionals in this study with more total years practicing in mental health settings were more likely to report greater competencies in recovery than colleagues with fewer years of experience. On the one hand, we might expect that those practicing for many years would have a more difficult time adapting to a new paradigm or experience burn out from the job. These results suggest the opposite is true. Providers with more years of experience in the mental health field have higher recovery competencies perhaps because they have gained perspective through longer term relationships with clients or through continuing education and supervision.
Community Ment Health J
The finding that mental health professionals who are part of intensive case management teams report higher levels of recovery competencies may be because these teams provide mental health and services, around-the-clock, to clients within their communities. While intensive case management teams may also practice with assertive elements that are less consistent with the recovery model (Drake and Burns 1995; Burns and Santos 1995), these results suggest how intensive case management teams are being implemented in these agencies may be more consistent with a recovery model. Variety on the job was a significant predictor of recovery competencies indicating the provision of recovery-oriented clinical services may be enabled by greater flexibility in job descriptions and the ability to step outside traditional clinical realms to help clients. Perceived workplace recovery culture was a strong predictor of the recovery competencies of mental health professionals in this study. This measure was comprised of various items that reflect the activities and recovery norms of the agency. Management strategies that seek to shift the recovery culture of the agency may improve providers’ recovery-oriented competencies according to these results. According to Jacobson and Curtis (2000), creating a recovery-wide agency culture begins with changing assumptions about clients from a deficit focus to a strengths-based focus, making visible stories of hope and, creating opportunities for healing based on a culture of acceptance and respect that permeates throughout the organization and its leadership. Organizations also have to fundamentally shift how they share power and responsibility providing opportunities for clients and providers to learn in a safe environment. We note that while personal experiences with mental illnesses were common among the providers in this sample, they did not emerge as a significant predictor of competencies. Type of graduate education beyond college was also not associated with competencies in the final model. These findings suggest recovery competencies may be learned on the job as a mental health professional. Variables controlling for geographic context were also not prominent predictors of mental health professionals’ recovery competencies. However, the fact that the workplace recovery culture, as assessed by individual mental health professionals, was strongly associated with the CAI indicates organizational culture may play a role in how mental health professionals practice with regard to recovery. Future research should discern other specific agency level predictors of competencies. Certain limitations of this study should be noted. First, there were significant challenges to obtaining high response rates among mental health professionals surveyed in a community mental health setting due to competing demands on providers’ time. Significant effort was put into
establishing a sampling frame so that a response rate and the representativeness of the sample could be established. Second, because the survey is cross-sectional, we cannot make causal statements about the relationships between independent and dependent variables. For instance, it is possible that professionals with greater competencies report also are more inclined to perceive a more recoveryoriented workplace. Causality cannot be determined. Third, some of the measures i.e., agency recovery culture and the CAI had to be shortened to accommodate the busy clinical setting in which the data were collected. Finally, the specific competencies measured in this study have not yet been validated against actual clinical practice. Thus, it is possible they represent a preference for recovery-oriented practice rather than a true measure even though the items were written to minimize social desirability bias. Despite these limitations, this study identifies several factors that are potentially important in transforming mental health systems towards a recovery paradigm, including an intensive focus on the recovery culture of agencies, offering training on the recovery model at a deep level, creating job descriptions with greater variety to help clients, the retention of mental health professionals for longer periods of time, and increasing the availability of and encouraging professionals to be part of intensive case management teams. Acknowledgments Data collection was funded by a Mental Health State Incentive Grant Award No. 6 U79 SM57648 from the Substance Abuse and Mental Health Services Administration (SAMHSA). Data analysis was funded by the National Institute of Mental Health (#R03 MH 086791-02).
References Anthony, W. A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990’s. Psychosocial Rehabilitation Journal, 16(4), 11–23. Burns, B. J., & Santos, A. B. (1995). Assertive community treatment an update of randomized trials. Psychiatric Services, 46(7), 669–675. Chinman, M., Young, A. S., Rowe, M., Forquer, S., Knight, E., & Miller, A. (2003). An instrument to assess competencies of providers treating severe mental illness. Mental Health Services Research, 5(2), 97–108. Coursey, R. D., Curtis, L., Marsh, D. T., Campbell, J., Harding, C., Spaniol, L., et al. (2000a). Competencies for direct service staff who work with adults with severe mental illnesses in outpatient public mental health/managed care services. Psychiatric Rehabilitation Journal, 23(4), 370–377. Coursey, R. D., Curtis, L., Marsh, D. T., Campbell, J., Harding, C., Spaniol, L., et al. (2000b). Competencies for direct service staff who work with adults with severe mental illnesses: Specific knowledge, skills and bibliography. Psychiatric Rehabilitation Journal, 23(4), 378–392. Cradock, J., Young, A. S., & Sullivan, G. (2001). The accuracy of medical record documentation in schizophrenia. Journal of Behavioral Health Services and Research, 28(4), 456–465.
123
Community Ment Health J Crowe, T. P., Deane, F. P., Oades, L. G., Caputi, P., & Morland, K. G. (2006). Effectiveness of a collaborative recovery training program in Australia in promoting positive views about recovery. Psychiatric Services, 57(10), 1497–1500. Drake, R. E., & Burns, B. J. (1995). Special section on assertive community treatment an introduction. Psychiatric Services, 46(7), 667–668. Hogan, M. F. (2003). New Freedom Commission Report: The President’s New Freedom Commission: Recommendations to transform mental health care in America. Psychiatric Services, 54, 1467–1474. Jacobson, N., & Curtis, L. (2000). Recovery as policy in mental health services: Strategies, emerging from the States. Psychiatric Rehabilitation Journal, 23(4), 333–341. Karasek, R., & Theorell, T. (1990). Healthy work: Stress, productivity and the reconstruction of working life. New York: Basic Books. Koeske, G., Kirk, S., Koeske, R., & Rauktis, M. (1994). Measuring the Monday blues: Validation of a job satisfaction scale for the human services. Social Work Research, 18(1), 27–35.
123
Little, R. J., Rubin, D. B. (2002). Statistical analysis with missing data, 2nd edition (pp. 2–4, 85–6), Hoboken: NJL John Wiley & Sons. Meehan, T., & Glover, H. (2009). Using the recovery knowledge inventory (RKI) to assess the effectiveness of a consumer-led recovery training program for service providers. Psychiatric Rehabilitation Journal, 32(3), 223–226. O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L. (2005). From rhetoric to routine: Assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric Rehabilitation Journal, 28, 378–386. Pollard, L., Gelbard, Y., Levy, G., & Gelkopf, M. (2008). Examining attitudes, beliefs and knowledge of effective practices in psychiatric rehabilitation in a hospital setting. Psychiatric Rehabilitation, 32(2), 124–127. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed., p. 100). Thousand Oaks, CA: Sage Publications, Inc.
