Australian Occupational Therapy Journal (2015) 62, 316–325
doi: 10.1111/1440-1630.12202
Research Article
Implementing evidence-based practices in supported employment on the Gold Coast for people with severe mental illness Philip Lee Williams,1 Chris Lloyd,2 Geoffrey Waghorn3 and Tawanda Machingura4 1 Early Psychosis Gold Coast, Gold Coast University Hospital, 2 Behavioural Basis of Health, Griffith University Gold Coast Campus, Southport, 3Department of Recovery and Social Inclusion, The Queensland Centre for Mental Health Research, West Moreton Hospital and Health Service, The Park Centre for Mental health, Wacol, and 4Department of Occupational Therapy, Mental Health Recovery Service, Gold Coast University Hospital, Southport, Queensland, Australia
Aim: The aim of this project was to evaluate program outcomes following the implementation of an evidence-based approach to supported employment on the Gold Coast, Queensland, Australia. Method: A prospective observational design was used to evaluate employment outcomes and fidelity to the evidence-based principles and practices of a specialised form of supported employment. The cohort was defined as all those (n = 114) that entered the program at each of three sites within a 21-month period. Each participant was followed up for a minimum of six months. All three sites implemented the employment program by establishing a partnership between a non-government organisation and the Gold Coast community mental health service. Results: The primary outcome variable was the proportion commencing competitive employment during the follow-up period from among those that commenced receiving assistance (the denominator). This ranged from 12% at Site C to 33.3% at Site A, and 37% at Site B. Fidelity to evidence-based principles was fair at Sites A
Philip Lee Williams BaOccThy, MPH; Team Leader. Chris Lloyd BaOccThy, PhD; Senior Research Fellow. Geoffrey Waghorn BaSocial Sciences, PhD; Head of Recovery and Social Inclusion. Tawanda Machingura BaOccThy; Assistant Director of Occupational Therapy. Correspondence: Philip Williams, Headspace Youth Early Psychosis Program Southport, 26 Railway Street, Southport 4215, Qld, Australia. Email: philip.williams@head spacesouthport.org.au Conflict of interest: This report was jointly supported by the organisations with which the authors are affiliated. The authors have no conflicts of interest. Accepted for publication 9 March 2015. © 2015 Occupational Therapy Australia
and C and good at Site B. These results were below expectations based on international-controlled trials. The variation in site effectiveness appeared related to both fidelity to evidence-based principles and to other factors at each site, which could not be clearly identified. Conclusions: Delivering an effective supported employment program using an inter-agency partnership method is challenging. There are several roles in which occupational therapists can be involved that facilitate improving both the implementation and the effectiveness of supported employment for people with severe mental illness in Australia. KEY WORDS competitive employment, mental health services, psychosis, severe mental illness.
Introduction Competitive employment is defined as part time or full-time work in the open labour market at or above minimum wages, in jobs not reserved for people with disabilities, with supervision provided by personnel regularly employed by the business (Bond, Drake & Becker, 2008; Waghorn, Saha & McGrath, 2014). Community residents with severe mental illness are interested in employment as an important part of their recovery goals. A recent study of 255 community residents with schizophrenia or schizoaffective disorder found that 85% were either employed or interested in employment as a future goal. Despite this interest, participation in employment remains low at 22%, which is 3.5 times lower than the 77% employed proportion of healthy working age Australians (Waghorn et al., 2014). In Australia, the Queensland Mental Health Plan 2007–2017 (Queensland Health, 2008) identifies
EVIDENCE-BASED SUPPORTED EMPLOYMENT
vocational rehabilitation as a priority area for mental health service development. Consistent with this mental health plan, the Gold Coast Hospital and Health Service District commenced implementing a specialised form of supported employment designed specifically for people with severe mental illnesses. This method is known as the Individual Placement and Support (IPS) approach, a widely accepted evidence-based approach to supported employment (Bond, 2004; Bond, Drake & Becker, 2012a; Bond et al., 2008; Kinoshita et al., 2013). This approach offers important new opportunities for occupational therapists to support this program. Occupational therapists have acted as program leaders at both strategic and operational levels (Waghorn et al., 2012). In a New Zealand example, an innovative team leader of a youth mental health team, who was also an occupational therapist, designed and implemented an IPS program as both the program leader and the employment specialist delivering intensive one-on-one employment assistance (Porteous & Waghorn, 2007, 2009). Other occupational therapists have used this program as the basis to deliver recovery- enhancing interventions, such as training for other mental health team members in how consumer employment goals can facilitate recovery (Gladman, Waghorn, Wishart & Dias, 2015). Other authors have suggested how the positive impact of evidence-based supported employment can be used to update and redefine the role of occupational therapists working in public mental health (Lloyd, Deane, Tse & Waghorn, 2009; Lloyd & Waghorn, 2008; Waghorn, Lloyd & Clune, 2009a). These authors highlight how the continued development of vocational rehabilitation within mental health services includes opportunities for occupational therapy to add value to public mental health programs. The terms psychiatric rehabilitation, vocational rehabilitation and supported employment are not often clearly defined. In the mental health context, vocational rehabilitation means a form of psychiatric rehabilitation where the person’s recovery goals involve specified employment, education or training goals. Supported employment is a form of vocational rehabilitation, where the focus is on providing support to the person to obtain and retain competitive employment in the open labour market. While there are other approaches to vocational rehabilitation described in the literature, such as pre-vocational training, social enterprises, sheltered employment and clubhouse transitional employment (Waghorn & Lloyd, 2005), these are out of scope for this report. The focus is on the IPS approach due to it being a specialised and well-defined model with a substantial evidence base (Bond, Peterson, Becker & Drake, 2012b; Kinoshita et al., 2013).
A specialised form of supported employment The IPS approach was designed for consumers of public mental health services living with a severe mental
317 illness who have a recovery goal of competitive employment in the open labour market (Bond, 2004; Bond et al., 2008, 2012a). This approach involves intensive and individualised support coordinated with publicly funded mental health services. Employment specialists are added to the mental health team specifically to assist service users with their competitive employment goals. All mental health team consumers are invited to participate, usually via discussions with their mental health case manager. Those with severe disabilities and complex comorbid conditions, such as psychotic disorders and comorbid substance dependence, are not discouraged from participating. Becker et al. identified eight core principles of IPS (Bond et al., 2012a,b). These include: (i) Every person with severe mental illness who wants to work is eligible for IPS supported employment; (ii) Employment services are integrated with mental health treatment services; (iii) Competitive employment is the goal; (iv) Personalised benefits counselling is provided; (v) The job search starts soon after a person expresses interest in working; (vi) Employment specialists systematically develop relationships with employers based upon their client’s preferences; (vii) Job supports are continuous and (viii) Client preferences are honoured. This intensive and individualised approach to providing employment assistance differs from other approaches to vocational rehabilitation through its focus on mainstream competitive employment, rather than set aside jobs or sheltered employment. IPS also avoids the lengthy pre-employment preparation or training phases found in other forms of vocational rehabilitation, and does not negatively screen for work readiness or employability (Rinaldi, Miller & Perkins, 2010). This approach has been successfully implemented in Australia (Killackey, Jackson & McGorry, 2008; Morris, Waghorn, Robson, Moore & Edwards, 2014; Waghorn, Chant & Jaeger, 2007; Waghorn, Stephenson & Browne, 2011; Waghorn et al., 2014). The most recent IPS implementation occurred in regional New South Wales, using a partnership approach between existing services. Morris et al. (2014) reported 12-month outcomes for this program at four implementation locations. The four sites achieved a mean proportion commencing competitive employment of 57% (54 of 95). The proportion commencing employment over 12 months varied from 39% at Newcastle to 72% at Peel’ (Morris et al., 2014, p. 146). In terms of continuing employment for 13 weeks or more, 45% attained this milestone in 12 months and 32.6% of participants attained 26 weeks employment. To achieve these results, Hunter New England rehabilitation staff (two were occupational therapists) provided five hours per week of external technical support to each site to facilitate the adoption of IPS principles and practices. Despite this support, two sites had lower than expected competitive employment results that could not be explained by low fidelity scores. Morris et al. (2014) © 2015 Occupational Therapy Australia
318 concluded that while sustainable partnerships between the health and employment sectors are feasible, they require further investigation to understand how local factors other than fidelity to IPS contribute to their success or failure. Killackey et al. (2008) implemented IPS in Melbourne by creating a new integrated employment service without utilising existing employment services. Using a randomised controlled design (n = 41), IPS participants (n = 20) in a youth mental health service were compared with those receiving treatment as usual (n = 21), which involved the mental health case managers facilitating the vocational goals of their clients during the provision of usual mental health care. The results at six months favoured the IPS group in terms of commencing employment, hours worked per week, number of jobs acquired and duration of employment. The authors concluded that IPS can be implemented in Australia with high fidelity using this approach, and can successfully address the vocational assistance needs of young people with first episode psychosis. When mental health services restrict their focus to routine clinical treatment and care, without directly supporting service users’ employment goals, participation in competitive employment declines accordingly. Bertram and Howard (2006) examined mental health case notes in a South London mental health service to identify employment activity among mental health service users. They examined 297 treatment and care plans for notes about vocational interventions. They found 12% were employed with little evidence of vocational rehabilitation planning. Only 8% of records included notes about vocational interventions. In Hong Kong, Tsang, Chan, Wong and Liberman (2009) examined the effectiveness of an integrated supported employment program, which aimed to enhance IPS with social skills training. Participants (n = 163) were randomly assigned to three forms of vocational rehabilitation: enhanced IPS, IPS and traditional vocational rehabilitation. After 15 months of assistance enhanced IPS participants had significantly higher employment commencements (78.8%), and longer job tenure (24 weeks) compared with standard IPS, and traditional vocational rehabilitation participants. This report outlines the employment outcomes attained from implementing IPS on the Gold Coast using a partnership approach between the Gold Coast Mental Health Service and two non government organisations contracted to the Australian Government to deliver disability employment services in the Gold Coast region. The aim was to evaluate program implementation and effectiveness by examining (i) fidelity with respect to the principles and practices of IPS, and (ii) employment outcomes attained in comparison to similar controlled trials in Australia and in other developed countries. © 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
Method The primary employment outcome was defined as the proportion commencing one day or more of competitive employment (the numerator) from among those that commenced receiving employment assistance (the cohort as the denominator). A prospective observational design was used to measure this outcome. Fidelity to the evidence-based practices of supported employment was assessed once at each site using the 15-item IPS Fidelity scale (Bond, Becker, Drake & Vogler, 1997). The cohort was defined as all those that entered the program within a 21-month period, across three sites (Sites A, B and C, see Tables 1–3). On entry to the IPS program, referral information, participant characteristics and mental health information were compiled in a MS Excel file by the occupational therapist. The employment specialist for each site provided monthly updates on employment progress to track program outcomes for each participant. Each participant was followed up for a minimum of six months after commencing to receive employment assistance. The evaluation included an examination of referrals to the program over the first 21 months. Ethics approval was obtained from the Gold Coast Hospital and Health Service prior to commencement of the study. No additional funds were obtained to support this project.
Participants All referrals to the program received from 1 February 2012 to 30 September 2013 were examined. The cohort consisted of the subset of those referrals who met the inclusion criteria and who commenced receiving vocational assistance. Inclusion criteria were: (i) a current client of the Gold Coast mental health service at the time of referral; (ii) diagnosed with a severe mental illness or psychiatric disability; (iii) of working age, namely 18–64 years; (iv) expressing interest in competitive employment as a recovery goal; (v) not currently employed or enrolled with another employment service provider; (vi) speaks English sufficiently not to need an interpreter and (vii) considered by the clinical team as able to safely participate in the program. Clients with severe, complex or comorbid psychiatric disorders were not excluded. Participant characteristics are shown in Table 1.
The IPS intervention The Gold Coast mental health service selected the IPS model for implementation, because it provided an evidence-based approach to supported employment. Two occupational therapists (Assistant Director of occupational therapy and a Team Leader) were identified as leaders for the new program. Expressions of interest were sought from local disability employment services about forming an employment partnership with the mental health service to provide dedicated employment services
319
EVIDENCE-BASED SUPPORTED EMPLOYMENT
TABLE 1: IPS site and participant characteristics Characteristic
Site A
Site B
Site C
Number of mental health consumers with access to the program at 1 Oct 2013 Number of full-time employment specialists Program implementation commenced Cohort inclusion period
350
56
257
1.0 1 Aug 2012 1 Aug 2012 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 64 out of 75 33 of 52 38.1 (10.1) 29 of 52
1.0 1 Feb 2012 1 Feb 2012 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 67 out of 75 22 of 28 22.3 (2.2) 25 of 28
1.0 1 March 2013 1 March 2013 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 63 out of 75 24 of 34 39.4 (9.2) 30 of 34
21 of 52 2 of 52
3 of 28 0
3 of 34 1 of 34
Follow-up period IPS fidelity review conducted IPS fidelity review total score Males Age in years [Mean (SD)] Diagnosis of psychotic disorder (including First Episode Psychosis) Diagnosis of bipolar affective disorder (BPAD) Diagnosis of anxiety disorder
TABLE 2: Fidelity to evidence-based principles and practices IPS 15-item Fidelity Scale
Site A
Site B
Site C
1. Case load size 2. Vocational services 3. Vocational generalists 4. Integration of services 5. Vocational unit 6. Zero exclusion criteria 7. Ongoing work-based assessments 8. Rapid job search 9. Individualised job search 10. Diversity of jobs 11. Permanence of jobs 12. Jobs as transitions 13. Follow along supports 14. Community-based services 15. Assertive engagement and outreach Total score
5 5 5 3 3 4 5 4 4 5 4 5 5 3 4 64 of 75. Fair fidelity
5 4 4 3 5 5 5 4 5 3 4 5 5 5 5 67 of 75. Good fidelity
5 5 5 3 3 3 5 4 4 5 4 5 5 3 4 63 of 75. Fair fidelity
Notes: Item descriptions and scoring method for each item can be found in Bond, Becker, Drake, & Vogler (1997).
to the service users of three community mental health teams. Each of the three mental health teams was allocated a full-time employment specialist by one of three different disability employment service providers. This method followed the partnership approach described previously (Morris et al., 2014; Waghorn et al., 2012, 2014). The implementation process was guided by a regional steering committee consisting of representatives
of each mental health team, each disability employment service and the Gold Coast mental health district governance committee.
Measures Program outcome variables were selected to match the variables previously reported for this type of implementation (Morris et al., 2014). While the primary outcome © 2015 Occupational Therapy Australia
320
P. L. WILLIAMS ET AL.
TABLE 3: The nature of support provided to three Gold Coast IPS sites Regional level
Local level
Resources allocated
Gold Coast Steering Group was established to coordinate and support all three sites where IPS was to be implemented. Members include Assistant Director of occupational therapy (chair), managers of the two employment services, team leaders of the three mental health teams, and employment service and mental health staff from all three sites
Frequency and intensity of support
The steering committee met monthly for the first 12 months of the program and then two monthly for the next 12 months
Each site was allocated 0.5 FTE for a local mental health Team Leader or occupational therapist to be the IPS coordinator. At Site A the position was filled by an occupational therapist with good support from the mental health Team Leader. At Site B, the position was filled by an occupational therapist with good support from the Team Leader who was also an occupational therapist. At Site C, the position was filled by an occupational therapist who received limited support from the mental health Team Leader An occupational at each site provided onsite support: Site A, 3 h per week – desk space provided for employment consultant within team office; Site B, 5 h per week (plus direct support from the team leader) – desk space provided within team office; Site C, 30 min per week – desk space provided outside team office (treatment room)
Nature of support provided
1. Regular meetings provide stakeholders with opportunities to identify system problems to program delivery and to solve these at a regional level 2. Third parties such as the Government Income Support provider (Centrelink) are invited to this meeting to help solve systemic issues affecting employment service providers and mental health service consumers
was the proportion commencing competitive employment, the proportion commencing other vocational activity was also recorded. Other vocational activity included non-competitive employment (sheltered employment, voluntary work and unpaid work experience) and enrolment in formal education or training courses. Job diversity was measured by comparing the number of different job types to the total number of jobs recorded. Independent variables collected at an individual level included: age, sex, diagnostic category, date of referral, date of program commencement, reasons for non-acceptance of referrals, time to commence job seeking and time to commence the first job. Fidelity to evidence-based practices in supported employment was measured at each site using the Supported Employment Fidelity Scale (Bond et al., 1997). This measure consists of 15 items covering staffing (three items), organisation (three items) and services © 2015 Occupational Therapy Australia
1. Facilitate communication between mental health staff and employment services staff 2. Update an evaluation database with details of all new referrals, health information and employment progress 3. Assist the employment specialist to engage with mental health consumers and obtain medical information from the mental health teams for mandatory program eligibility assessments. Support the employment specialist, and facilitate their acceptance into the mental health team 4. Provide other interventions when needed to support the vocational goals of consumers. 5. Advocate for consumers’ vocational goals and support employment specialists to assist all interested consumers
(nine items). Each item is rated from one to five where five represents best practice. Mental health staff, from teams not involved in the study, were trained in the use of this measure. Each assessor was then asked to complete the IPS Fidelity scale for one site using a variety of information sources, including existing records such as minutes of meetings, and interviews with staff of both mental health and employment services. Each site had one IPS fidelity review during the period of data collection in this paper. Fidelity results at the item level are shown in Table 2.
Resources No additional resources were provided to mental health teams or to the partner disability employment services. Disability employment services provided the dedicated employment specialist from their existing resources. Equal amounts of in-kind support were offered to each site by the program leaders and by the
321
EVIDENCE-BASED SUPPORTED EMPLOYMENT
regional steering group. However, due to local differences in leadership and staffing, each site differed in how the program was actually governed and supported at each site. For example, in addition to the regular regional steering group meetings Site A held monthly meetings between the occupational therapist and the employment specialist. Site B held weekly meetings involving the occupational therapist, the employment specialist and the mental health Team Leader. No additional meetings were arranged to support Site C. Challenges in leading and resourcing this type of program have been documented (Waghorn, Lockett, Bacon, Gorman & Durie, 2009b).
Results Sites A and C reported similar participant characteristics. A higher proportion of people diagnosed with bipolar affective disorder were found at Site A compared with Site C (38% vs. 8%). Site B is a dedicated early psychosis service and this was reflected both in the higher proportions diagnosed with a primary psychotic disorder (92%), and the younger mean age of 22 years (see Table 1). Site B achieved good fidelity to IPS principles while sites A and C achieved fair fidelity (see Table 2). However, all sites show room for improvement in the application of evidence-based practices, with respect to at least six or more of the 15 practices examined. All three
sites had room to increase integration between mental health and vocational services. This could be achieved at each site by co-locating the dedicated employment specialist four days per week into the partner mental health team. All sites were allocated the same in-kind resources to support the implementation of the IPS model. However, these resources were utilised differently at each site. Site B provided the most support on a weekly basis, followed by Site A, and Site C (see Table 3). The support provided by the Early Psychosis team leader at Site B included discussing all referrals with mental health case managers, and meeting fortnightly with the employment specialist to share relevant information and to review and update evaluation data. This ranking of sites (B>A>C) also held with respect to IPS fidelity (see Table 2) and also reflected relative performance on the primary outcome variable, namely the proportion of participants commencing competitive employment (see Table 4). Primary outcomes in Table 4 indicate that Site B achieved the highest proportion of participants commencing competitive employment, at 37.0% compared with 33.3% at Site A and 12% at Site C. Site A had the highest proportion (70.3%) of participants attaining any vocational outcome (competitive employment, other vocational outcome, or formal study and training), compared with Site B (55.5%), and Site C (20.0%) (see Table 4).
TABLE 4: Employment outcomes by site Employment outcomes
Site A
Site B
Site C
Number of referrals received in 21 months Program commencements Non-commencement reason Participant declined Ineligible for disability Employment services program Unknown Commenced competitive employment Commenced any vocational activity including competitive employment Completed 13 weeks competitive employment Time taken from referral to commence job searching in days, Mean (SD) Time in days from referral to commencing competitive employment, Mean (SD) Number of jobs held by those who commenced employment, Mean (SD) Hours worked per week, Mean (SD) Job diversity: the number of different job types compared with the total number of jobs
52 27/52 (51.9%)
28 27/28 (96.4%)
34 25/34 (73.5%)
11/25
1/1
7/9
4/25 10/25 9/27 (33.3%) 19/27 (70.3%)
10/27 (37.0%) 15/27 (55.5%)
0/9 2/9 3/25 (12.0%) 5/25 (20.0%)
8/27 (29.6%) 51.2 (39.8)
4/27 (14.8%) 40.4 (17.6)
1/25 (4.0%) 47.2 (32.3)
97.3 (31.8)
91.4 (82.8)
275.6 (296.3)
1.5 (0.8), Range 1–3
1.6 (0.9), Range 1–3
Unknown
19.8 (5.6) n = 9 8/9
23.0 (11.5), n = 10 8/10
11.0 (3.6), n = 3 1/3
© 2015 Occupational Therapy Australia
322 In terms of completing 13 weeks employment, site A (29.6%) outperformed both sites B (14.8%) and C (4.0%). This result suggests that retaining employment is more challenging for Site B compared with site A. The results also show performance at site C to be disappointing on all metrics except the proportion of referrals that converted to actual program commencements (73.5%). Site C also had the lowest fidelity score and received the least amount of support from the mental health team (see Table 4).
Discussion Kennedy-Jones, Cooper and Fossey (2005) explored the experience of engaging in work from a mental health service user point of view. They concluded that health benefits for the individual were significant and urged occupational therapists to provide ongoing support to people with severe mental illness who seek employment. Introducing the IPS model as a means to promote vocational recovery also provides opportunities to occupational therapists to support these important service user goals. In this project, promising employment outcomes were achieved at sites A and B (see Table 4). Yet these results are disappointing in comparison to previous Australian trials of IPS (Killackey et al., 2008; Morris et al., 2014; Waghorn et al., 2014) and with respect to USA trials (Bond et al., 2012a). As Morris et al. (2014) have shown, a competent co-located employment specialist supported by a strong partnership with good fidelity to evidence-based practices, and with regular external technical support, can enable 60% or more participants referred to an IPS program to commence competitive employment. That two of the three sites in this study achieved just over half of this target indicates that much more can be done to improve employment outcomes for Gold Coast participants. Sites A and C were similar in terms of client mix (see Table 1). However, there were marked differences with respect to integration with mental health services, strength of local leadership, and commitment to evidence-based practices. Sites A and B achieved higher fidelity to IPS practices. In addition, the ranking in terms of amount of in-kind support to the program where Site B>Site A>Site C, was consistent with fidelity scores and overall performance on the primary outcome. The lack of support for the program at Site C, and its subsequent fair fidelity, seemed to predetermine its disappointing results. Site B represented the strongest local leadership and most engagement between the treating team and the disability employment service provider. However, Site B did not perform as well as site A in terms of job retention, or in terms of any vocational outcome. The former appears related to the diagnostic mix of mostly young people with first episode psychosis. However, the difference in terms of any © 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
vocational outcomes most reflects a lack of focus on competitive employment at Site A. These results suggest that the program could be strengthened at all three sites through: (i) full-time co-location of the employment specialist within each mental health team; (ii) better integration of mental health services with employment services building on full-time co-location and (iii) more technical support for each site for implementing other evidence-based practices (see Table 2). While the regional steering committee provided overall governance and a strategic direction for the program, its existence did not appear to positively influence employment outcomes at each site. It appears that the strength of leadership provided at a site level may be a factor in both better implementation and in the better primary outcomes attained. Any allied health discipline such as Psychology, Mental Health Nursing, Social Work, Speech Therapy or Physiotherapy could in principle drive this system change. However, other allied health professionals in this study seemed to step back to allow occupational therapists to take the lead. Occupational therapists were well suited to supporting this program because the teams with the greatest involvement by occupational therapists also achieved the best results. This is consistent with prior implementations of IPS in Australia and in New Zealand, where occupational therapists have been critically involved as: program advocates; program leaders; program evaluators (Morris et al., 2014); site champions; trainers; mental health team leaders; researchers (Waghorn et al., 2012, 2014); employment specialists and mental health case managers (Porteous & Waghorn, 2007, 2009).
Limitations The main limitation of the evaluation design is that it was an implementation study, not a randomised controlled trial. In a controlled trial, every effort would normally be made to control for between site differences such as different use of resources in supporting the program at a local level. However, this was an observational study of an implementation strategy where the investigators had little control over what happened within each site. Although issues relating to a particular site could be raised at the regional steering committee meeting, it was not possible to force change on any particular site or partnership. Hence, in this study, the actual strength of implementation of IPS achieved at each site is best considered a mediating variable rather than as an independent variable or independent covariate. In terms of analysis, we relied mostly upon descriptive differences and anecdotal impressions, because power was too low to confidently compare sites for statistically significant differences. While this is appropriate for generating ideas to improve the program, it is not sufficient for reaching conclusions about program
323
EVIDENCE-BASED SUPPORTED EMPLOYMENT
effectiveness in this context. We also note that data quality could be managed by monitoring the database and its updating more regularly to maintain optimal accuracy of information, to prevent missing information and to discuss fidelity scores more regularly with assessors to achieve greater consistency of fidelity scoring across sites. The quality of implementation was assessed using an earlier 15-item version of the IPS fidelity scale (Bond et al., 1997) that has since been updated to a potentially more useful 25-item version (Bond et al., 2012b). The most salient shortcoming was the failure of each site to achieve co-location of the employment specialist for four of five days per week in the mental health team premises. The aim of this step is to provide the basis for ongoing informal communication with the mental health team staff and with service users. Although the employment specialist is expected to be out of the office for 70% of the time visiting employers, co-location has been found useful for developing seamless coordination of the two services. This in term is expected to lead to greater service integration through the increased opportunities for clinicians to consider vocational issues, and the opportunities for the employment specialist to understand the clinical issues that may be impacting on work performance (Sherring, Robson, Morris, Frost & Tirupati, 2010; Waghorn et al., 2007, 2012). Other weak practices were working as a vocational unit, zero exclusion criteria and community-based services. Not working as a vocational unit, suggests a lack of regular supervision for employment specialists that is needed to ensure they develop positive relationships with local employers. Not sufficiently implementing the zero exclusion principle was also reflected in the proportion of referrals that did not lead to program commencements. This is important because not gaining timely access to a sought after program can adversely impact on service user motivation. In addition, the failure to implement the practice of community-based services that can leave employment specialists office bound and out of touch with local employers. It can also lead to a reliance on participants finding their own jobs, rather than the employment specialist generating opportunities matching job seeker preferences, from a well-known network of employers.
Implementation challenges Implementing evidence-based supported employment at the Gold Coast involved several practical challenges. The initial process of establishing a formal agreement between the Gold Coast mental health service and the disability employment services took approximately two years. This delay meant it was difficult to gather momentum and difficult to sustain any gains from efforts to promote the program. The program had official executive support from the health service, but lacked an identified executive sponsor to advocate for
the program and drive decisions supporting high-fidelity implementation. As a result, operational barriers emerged that were difficult to resolve quickly resolve. These included: procedures for managing confidential information; physical space and facilities for the employment specialists within each mental health team; and keeping to an agreed schedule for employment specialists to attend the health premises. The challenging nature of the Gold Coast labour market may have impacted on employment outcomes. The Australian Bureau of Statistics (2011) reported that background unemployment in the Gold Coast area at the time of this study was 7.4% compared with the greater Brisbane area of 5.4%. In addition, the Gold Coast has three transient elements to its population, consisting of students, young people on working holidays and tourists. The first two groups compete for entry level employment opportunities. Whilst, it is unlikely that adverse labour market factors explain these results, it is possible that the failure to achieve high IPS fidelity made all sites more prone to adverse effects from these labour market factors. Just such an effect has been suggested previously to explain why the more intensive IPS practices are less prone to moderation by labour market factors (Bond, 2004; Bond et al., 2008).
Opportunities for occupational therapists This study reveals the many ways occupational therapists can be involved to support the introduction of evidence-based practices in supported employment for adults with severe mental illness. The evidence suggests that the IPS model can be implemented in Australia using a partnership approach, provided persistent efforts are made to establish and maintain high-fidelity IPS practices. While there are other approaches to vocational rehabilitation, they are typically less than half as successful as the IPS approach in attaining competitive employment for people with severe mental illness (Bond et al., 2012a; Waghorn & Lloyd, 2005). This study highlights the importance of strong local leadership in supporting the introduction of evidencebased practices in supported employment. Of all the allied health disciplines, occupational therapy seems best placed to provide this leadership within mental health services. Once a good knowledge of IPS practices is acquired occupational therapists can also assist staff of disability employment services in adopting these practices. There is potential for the role of facilitating the integration of supported employment into community mental health services, to become a major role for occupational therapists, complementing their existing work in mental health case management, treatment and care and general psychiatric rehabilitation. Consequently, it may be important to ensure that undergraduate and postgraduate programs adequately prepare students for these roles particularly for those seeking to specialise in mental health practice. © 2015 Occupational Therapy Australia
324
Conclusion IPS implementation using existing systems and resources in Australia is challenging yet achievable. It requires two existing services to operate in close partnership to jointly deliver a new service that complements existing mental health treatment, care and rehabilitation services. The evidence suggests that this requires strong local leadership, good knowledge of IPS principles and practices, local technical support and a strong commitment by all stakeholders to adopting all the evidence-based practices in supported employment. In addition, ongoing evaluation is critical, not only to monitor fidelity with evidence-based practices but also to ensure that the adoption of the new IPS practices actually leads to the level of employment outcomes expected. Occupational therapists working in both community mental health services and in disability employment services have new opportunities to advocate, implement, support and facilitate this essential yet challenging program.
References Australian Bureau of Statistics. (2011). Census of Population and Housing: 2006–2011. Retrieved from: http://profile.id.com.au/gold-coast/employment-status on 14/4/14. Bertram, M. & Howard, L. (2006). Employment status and occupational care planning for people using mental health services. Psychiatric Bulletin, 30, 48–51. doi: 10. 1192/pb.30.2.48 Bond, G. (2004). Supported employment: Evidence for an evidence based practice. Psychiatric Rehabilitation Journal, 27 (4), 345–359. doi: 10.2975/27.2004.345.359 Bond, G. R., Becker, D. R., Drake, R. E. & Vogler, K. (1997). A fidelity scale for the Individual Placement and Support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265–284. Bond, G. R., Drake, R. E. & Becker, D. R. (2008). An update on randomised controlled trials of evidence-based supported employment. Psychiatric Rehabilitation Journal, 31, 280–290. doi: 10.2975/31.4.2008.280.290 Bond, G. R., Drake, R. E. & Becker, D. R. (2012a). Generalizability of the Individual Placement and Support (IPS) model of supported employment outside the US. World Psychiatry, 11 (1), 32–39. doi: 10.1016/j.wpsyc.2012.01.005 Bond, G. R., Peterson, A. E., Becker, D. R. & Drake, R. E. (2012b). Validating the revised Individual Placement and Support fidelity scale (IPS-25). Psychiatric Services, 63, 758–763. doi: 10.1176/appi.ps.201100476 Gladman, B., Waghorn, G., Wishart, L. & Dias, S. (2015). Reliability of a measure of health professionals’ perceptions of integrated employment services for people with psychiatric disabilities. Journal of Rehabilitation, 81 (1), 3–8. Kennedy-Jones, M., Cooper, J. & Fossey, E. (2005). Developing a worker role: Stories of four people with mental illness. Australian Occupational Therapy Journal, 52 (2), 116–126. doi: 10.1111/j.1440-1630.2005.00475.x
© 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
Killackey, E., Jackson, H. J. & McGorry, P. D. (2008). Vocational intervention in first-episode psychosis: Individual placement and support vs. treatment as usual. British Journal of Psychiatry, 193 (2), 114–120. doi: 10.1192/bjp. bp.107.043109. Kinoshita, Y., Furukawa, T. A., Kinoshita, K., Honyashiki, M., Omori, M., Marshall, M. et al. (2013). Supported employment for adults with severe mental illness. Cochrane Database of Systematic Review, 2013 (9), doi: 10.1002/ 14651858.CD008297.pub2. Lloyd, C. & Waghorn, G. (2008). Reconfiguring evidencebased practice of occupational therapists - commentary. International Journal of Therapy and Rehabilitation, 15 (10), 436. Lloyd, C., Deane, F., Tse, S. & Waghorn, G. (2009). Supporting recovery oriented services for people with severe mental illness. Australian Journal of Rehabilitation Counselling, 15 (1), 1–11. doi: 10.1375/jrc.15.1.1. Morris, A., Waghorn, G., Robson, E., Moore, L. & Edwards, E. (2014). Implementation of evidence-based supported employment in regional Australia. Psychiatric Rehabilitation Journal, 37 (2), 144–147. doi: 10.1037/prj0000051. Porteous, N. & Waghorn, G. (2007). Implementing evidence-based employment services in New Zealand for young adults with psychosis: Progress during the first five years. British Journal of Occupational Therapy, 70, 521– 526. doi: 10.1177/030802260707001204. Porteous, N. & Waghorn, G. (2009). Developing evidencebased supported employment services for young adults receiving public mental health services. New Zealand Journal of Occupational Therapy, 56 (1), 34–39. Queensland Health. (2008). Queensland plan for mental health 2007–2012. Brisbane: Mental Health Branch, Queensland Health. Rinaldi, M., Miller, L. & Perkins, R. (2010). Implementing the individual placement and support (IPS) approach for people with mental health conditions in England. International Review of Psychiatry, 22 (2), 163–172. doi:10.3109/ 09540261003720456. Sherring, J., Robson, E., Morris, A., Frost, B. & Tirupati, S. (2010). A working reality: Evaluating enhanced intersectoral links in supported employment for people with psychiatric disabilities. Australian Occupational Therapy Journal, 57, 261–267. doi: 10.1111/j.1440-1630.2009.00844.x. Tsang, H. W. H., Chan, A., Wong, A. & Liberman, P. L. (2009). Vocational outcomes of an integrated supported employment programme for individuals with persistent and severe mental illness. Journal of Behaviour Therapy and Experimental Psychiatry, 40, 292–305. doi: 10.1016/j.jbtep. 2008.12.007. Waghorn, G. & Lloyd, C. (2005). The employment of people with mental illness. Advances in Mental Health, 4 (2), 129– 171. doi: 10.5172/jamh.4.2.129. Waghorn, G., Chant, D. & Jaeger, J. (2007). Employment functioning and disability among community residents with bipolar affective disorder : results from an Australian community survey. Bipolar Disorders, 9 (1), 166–182. doi: 10.1111/j.1399-5618.2007.00417.x. Waghorn, G., Lloyd, C. & Clune, A. (2009a). Reviewing the theory and practice of occupational therapy in mental
EVIDENCE-BASED SUPPORTED EMPLOYMENT
health rehabilitation. British Journal of Occupational Therapy, 72, 314–322. doi: 10.1177/030802260907200708. Waghorn, G., Lockett, H., Bacon, J., Gorman, P. & Durie, S. (2009b). The importance of leadership in implementing evidence-based supported employment services for people with severe mental illness. The International Journal of Leadership in Public Services, 5 (suppl), 51–56. Waghorn, G., Stephenson, A. & Browne, D. (2011). The importance of service integration in developing effective employment services for people with severe mental health conditions. British Journal Of Occupational Therapy, 74 (1), 339–347. doi: 10.4276/030802211X13099513661153.
325 Waghorn, G., Childs, S., Hampton, E., Gladman, B., Greaves, A. & Bowman, D. (2012). Enhancing community mental health services through formal partnerships with supported employment providers. American Journal of Psychiatric Rehabilitation, 15 (2), 157–180. doi: 10.1080/ 15487768.2012.679582. Waghorn, G., Saha, S. & McGrath, J. (2014). Correlates of competitive versus non-competitive employment among adults with psychotic disorders. Psychiatric Services, 65, 476–482. doi: 10.1176/appi.ps.201300096.
© 2015 Occupational Therapy Australia
doi: 10.1111/1440-1630.12202
Research Article
Implementing evidence-based practices in supported employment on the Gold Coast for people with severe mental illness Philip Lee Williams,1 Chris Lloyd,2 Geoffrey Waghorn3 and Tawanda Machingura4 1 Early Psychosis Gold Coast, Gold Coast University Hospital, 2 Behavioural Basis of Health, Griffith University Gold Coast Campus, Southport, 3Department of Recovery and Social Inclusion, The Queensland Centre for Mental Health Research, West Moreton Hospital and Health Service, The Park Centre for Mental health, Wacol, and 4Department of Occupational Therapy, Mental Health Recovery Service, Gold Coast University Hospital, Southport, Queensland, Australia
Aim: The aim of this project was to evaluate program outcomes following the implementation of an evidence-based approach to supported employment on the Gold Coast, Queensland, Australia. Method: A prospective observational design was used to evaluate employment outcomes and fidelity to the evidence-based principles and practices of a specialised form of supported employment. The cohort was defined as all those (n = 114) that entered the program at each of three sites within a 21-month period. Each participant was followed up for a minimum of six months. All three sites implemented the employment program by establishing a partnership between a non-government organisation and the Gold Coast community mental health service. Results: The primary outcome variable was the proportion commencing competitive employment during the follow-up period from among those that commenced receiving assistance (the denominator). This ranged from 12% at Site C to 33.3% at Site A, and 37% at Site B. Fidelity to evidence-based principles was fair at Sites A
Philip Lee Williams BaOccThy, MPH; Team Leader. Chris Lloyd BaOccThy, PhD; Senior Research Fellow. Geoffrey Waghorn BaSocial Sciences, PhD; Head of Recovery and Social Inclusion. Tawanda Machingura BaOccThy; Assistant Director of Occupational Therapy. Correspondence: Philip Williams, Headspace Youth Early Psychosis Program Southport, 26 Railway Street, Southport 4215, Qld, Australia. Email: philip.williams@head spacesouthport.org.au Conflict of interest: This report was jointly supported by the organisations with which the authors are affiliated. The authors have no conflicts of interest. Accepted for publication 9 March 2015. © 2015 Occupational Therapy Australia
and C and good at Site B. These results were below expectations based on international-controlled trials. The variation in site effectiveness appeared related to both fidelity to evidence-based principles and to other factors at each site, which could not be clearly identified. Conclusions: Delivering an effective supported employment program using an inter-agency partnership method is challenging. There are several roles in which occupational therapists can be involved that facilitate improving both the implementation and the effectiveness of supported employment for people with severe mental illness in Australia. KEY WORDS competitive employment, mental health services, psychosis, severe mental illness.
Introduction Competitive employment is defined as part time or full-time work in the open labour market at or above minimum wages, in jobs not reserved for people with disabilities, with supervision provided by personnel regularly employed by the business (Bond, Drake & Becker, 2008; Waghorn, Saha & McGrath, 2014). Community residents with severe mental illness are interested in employment as an important part of their recovery goals. A recent study of 255 community residents with schizophrenia or schizoaffective disorder found that 85% were either employed or interested in employment as a future goal. Despite this interest, participation in employment remains low at 22%, which is 3.5 times lower than the 77% employed proportion of healthy working age Australians (Waghorn et al., 2014). In Australia, the Queensland Mental Health Plan 2007–2017 (Queensland Health, 2008) identifies
EVIDENCE-BASED SUPPORTED EMPLOYMENT
vocational rehabilitation as a priority area for mental health service development. Consistent with this mental health plan, the Gold Coast Hospital and Health Service District commenced implementing a specialised form of supported employment designed specifically for people with severe mental illnesses. This method is known as the Individual Placement and Support (IPS) approach, a widely accepted evidence-based approach to supported employment (Bond, 2004; Bond, Drake & Becker, 2012a; Bond et al., 2008; Kinoshita et al., 2013). This approach offers important new opportunities for occupational therapists to support this program. Occupational therapists have acted as program leaders at both strategic and operational levels (Waghorn et al., 2012). In a New Zealand example, an innovative team leader of a youth mental health team, who was also an occupational therapist, designed and implemented an IPS program as both the program leader and the employment specialist delivering intensive one-on-one employment assistance (Porteous & Waghorn, 2007, 2009). Other occupational therapists have used this program as the basis to deliver recovery- enhancing interventions, such as training for other mental health team members in how consumer employment goals can facilitate recovery (Gladman, Waghorn, Wishart & Dias, 2015). Other authors have suggested how the positive impact of evidence-based supported employment can be used to update and redefine the role of occupational therapists working in public mental health (Lloyd, Deane, Tse & Waghorn, 2009; Lloyd & Waghorn, 2008; Waghorn, Lloyd & Clune, 2009a). These authors highlight how the continued development of vocational rehabilitation within mental health services includes opportunities for occupational therapy to add value to public mental health programs. The terms psychiatric rehabilitation, vocational rehabilitation and supported employment are not often clearly defined. In the mental health context, vocational rehabilitation means a form of psychiatric rehabilitation where the person’s recovery goals involve specified employment, education or training goals. Supported employment is a form of vocational rehabilitation, where the focus is on providing support to the person to obtain and retain competitive employment in the open labour market. While there are other approaches to vocational rehabilitation described in the literature, such as pre-vocational training, social enterprises, sheltered employment and clubhouse transitional employment (Waghorn & Lloyd, 2005), these are out of scope for this report. The focus is on the IPS approach due to it being a specialised and well-defined model with a substantial evidence base (Bond, Peterson, Becker & Drake, 2012b; Kinoshita et al., 2013).
A specialised form of supported employment The IPS approach was designed for consumers of public mental health services living with a severe mental
317 illness who have a recovery goal of competitive employment in the open labour market (Bond, 2004; Bond et al., 2008, 2012a). This approach involves intensive and individualised support coordinated with publicly funded mental health services. Employment specialists are added to the mental health team specifically to assist service users with their competitive employment goals. All mental health team consumers are invited to participate, usually via discussions with their mental health case manager. Those with severe disabilities and complex comorbid conditions, such as psychotic disorders and comorbid substance dependence, are not discouraged from participating. Becker et al. identified eight core principles of IPS (Bond et al., 2012a,b). These include: (i) Every person with severe mental illness who wants to work is eligible for IPS supported employment; (ii) Employment services are integrated with mental health treatment services; (iii) Competitive employment is the goal; (iv) Personalised benefits counselling is provided; (v) The job search starts soon after a person expresses interest in working; (vi) Employment specialists systematically develop relationships with employers based upon their client’s preferences; (vii) Job supports are continuous and (viii) Client preferences are honoured. This intensive and individualised approach to providing employment assistance differs from other approaches to vocational rehabilitation through its focus on mainstream competitive employment, rather than set aside jobs or sheltered employment. IPS also avoids the lengthy pre-employment preparation or training phases found in other forms of vocational rehabilitation, and does not negatively screen for work readiness or employability (Rinaldi, Miller & Perkins, 2010). This approach has been successfully implemented in Australia (Killackey, Jackson & McGorry, 2008; Morris, Waghorn, Robson, Moore & Edwards, 2014; Waghorn, Chant & Jaeger, 2007; Waghorn, Stephenson & Browne, 2011; Waghorn et al., 2014). The most recent IPS implementation occurred in regional New South Wales, using a partnership approach between existing services. Morris et al. (2014) reported 12-month outcomes for this program at four implementation locations. The four sites achieved a mean proportion commencing competitive employment of 57% (54 of 95). The proportion commencing employment over 12 months varied from 39% at Newcastle to 72% at Peel’ (Morris et al., 2014, p. 146). In terms of continuing employment for 13 weeks or more, 45% attained this milestone in 12 months and 32.6% of participants attained 26 weeks employment. To achieve these results, Hunter New England rehabilitation staff (two were occupational therapists) provided five hours per week of external technical support to each site to facilitate the adoption of IPS principles and practices. Despite this support, two sites had lower than expected competitive employment results that could not be explained by low fidelity scores. Morris et al. (2014) © 2015 Occupational Therapy Australia
318 concluded that while sustainable partnerships between the health and employment sectors are feasible, they require further investigation to understand how local factors other than fidelity to IPS contribute to their success or failure. Killackey et al. (2008) implemented IPS in Melbourne by creating a new integrated employment service without utilising existing employment services. Using a randomised controlled design (n = 41), IPS participants (n = 20) in a youth mental health service were compared with those receiving treatment as usual (n = 21), which involved the mental health case managers facilitating the vocational goals of their clients during the provision of usual mental health care. The results at six months favoured the IPS group in terms of commencing employment, hours worked per week, number of jobs acquired and duration of employment. The authors concluded that IPS can be implemented in Australia with high fidelity using this approach, and can successfully address the vocational assistance needs of young people with first episode psychosis. When mental health services restrict their focus to routine clinical treatment and care, without directly supporting service users’ employment goals, participation in competitive employment declines accordingly. Bertram and Howard (2006) examined mental health case notes in a South London mental health service to identify employment activity among mental health service users. They examined 297 treatment and care plans for notes about vocational interventions. They found 12% were employed with little evidence of vocational rehabilitation planning. Only 8% of records included notes about vocational interventions. In Hong Kong, Tsang, Chan, Wong and Liberman (2009) examined the effectiveness of an integrated supported employment program, which aimed to enhance IPS with social skills training. Participants (n = 163) were randomly assigned to three forms of vocational rehabilitation: enhanced IPS, IPS and traditional vocational rehabilitation. After 15 months of assistance enhanced IPS participants had significantly higher employment commencements (78.8%), and longer job tenure (24 weeks) compared with standard IPS, and traditional vocational rehabilitation participants. This report outlines the employment outcomes attained from implementing IPS on the Gold Coast using a partnership approach between the Gold Coast Mental Health Service and two non government organisations contracted to the Australian Government to deliver disability employment services in the Gold Coast region. The aim was to evaluate program implementation and effectiveness by examining (i) fidelity with respect to the principles and practices of IPS, and (ii) employment outcomes attained in comparison to similar controlled trials in Australia and in other developed countries. © 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
Method The primary employment outcome was defined as the proportion commencing one day or more of competitive employment (the numerator) from among those that commenced receiving employment assistance (the cohort as the denominator). A prospective observational design was used to measure this outcome. Fidelity to the evidence-based practices of supported employment was assessed once at each site using the 15-item IPS Fidelity scale (Bond, Becker, Drake & Vogler, 1997). The cohort was defined as all those that entered the program within a 21-month period, across three sites (Sites A, B and C, see Tables 1–3). On entry to the IPS program, referral information, participant characteristics and mental health information were compiled in a MS Excel file by the occupational therapist. The employment specialist for each site provided monthly updates on employment progress to track program outcomes for each participant. Each participant was followed up for a minimum of six months after commencing to receive employment assistance. The evaluation included an examination of referrals to the program over the first 21 months. Ethics approval was obtained from the Gold Coast Hospital and Health Service prior to commencement of the study. No additional funds were obtained to support this project.
Participants All referrals to the program received from 1 February 2012 to 30 September 2013 were examined. The cohort consisted of the subset of those referrals who met the inclusion criteria and who commenced receiving vocational assistance. Inclusion criteria were: (i) a current client of the Gold Coast mental health service at the time of referral; (ii) diagnosed with a severe mental illness or psychiatric disability; (iii) of working age, namely 18–64 years; (iv) expressing interest in competitive employment as a recovery goal; (v) not currently employed or enrolled with another employment service provider; (vi) speaks English sufficiently not to need an interpreter and (vii) considered by the clinical team as able to safely participate in the program. Clients with severe, complex or comorbid psychiatric disorders were not excluded. Participant characteristics are shown in Table 1.
The IPS intervention The Gold Coast mental health service selected the IPS model for implementation, because it provided an evidence-based approach to supported employment. Two occupational therapists (Assistant Director of occupational therapy and a Team Leader) were identified as leaders for the new program. Expressions of interest were sought from local disability employment services about forming an employment partnership with the mental health service to provide dedicated employment services
319
EVIDENCE-BASED SUPPORTED EMPLOYMENT
TABLE 1: IPS site and participant characteristics Characteristic
Site A
Site B
Site C
Number of mental health consumers with access to the program at 1 Oct 2013 Number of full-time employment specialists Program implementation commenced Cohort inclusion period
350
56
257
1.0 1 Aug 2012 1 Aug 2012 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 64 out of 75 33 of 52 38.1 (10.1) 29 of 52
1.0 1 Feb 2012 1 Feb 2012 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 67 out of 75 22 of 28 22.3 (2.2) 25 of 28
1.0 1 March 2013 1 March 2013 to 1 Oct 2013 1 Oct 2013 to 1 April 2014 Feb 2013 63 out of 75 24 of 34 39.4 (9.2) 30 of 34
21 of 52 2 of 52
3 of 28 0
3 of 34 1 of 34
Follow-up period IPS fidelity review conducted IPS fidelity review total score Males Age in years [Mean (SD)] Diagnosis of psychotic disorder (including First Episode Psychosis) Diagnosis of bipolar affective disorder (BPAD) Diagnosis of anxiety disorder
TABLE 2: Fidelity to evidence-based principles and practices IPS 15-item Fidelity Scale
Site A
Site B
Site C
1. Case load size 2. Vocational services 3. Vocational generalists 4. Integration of services 5. Vocational unit 6. Zero exclusion criteria 7. Ongoing work-based assessments 8. Rapid job search 9. Individualised job search 10. Diversity of jobs 11. Permanence of jobs 12. Jobs as transitions 13. Follow along supports 14. Community-based services 15. Assertive engagement and outreach Total score
5 5 5 3 3 4 5 4 4 5 4 5 5 3 4 64 of 75. Fair fidelity
5 4 4 3 5 5 5 4 5 3 4 5 5 5 5 67 of 75. Good fidelity
5 5 5 3 3 3 5 4 4 5 4 5 5 3 4 63 of 75. Fair fidelity
Notes: Item descriptions and scoring method for each item can be found in Bond, Becker, Drake, & Vogler (1997).
to the service users of three community mental health teams. Each of the three mental health teams was allocated a full-time employment specialist by one of three different disability employment service providers. This method followed the partnership approach described previously (Morris et al., 2014; Waghorn et al., 2012, 2014). The implementation process was guided by a regional steering committee consisting of representatives
of each mental health team, each disability employment service and the Gold Coast mental health district governance committee.
Measures Program outcome variables were selected to match the variables previously reported for this type of implementation (Morris et al., 2014). While the primary outcome © 2015 Occupational Therapy Australia
320
P. L. WILLIAMS ET AL.
TABLE 3: The nature of support provided to three Gold Coast IPS sites Regional level
Local level
Resources allocated
Gold Coast Steering Group was established to coordinate and support all three sites where IPS was to be implemented. Members include Assistant Director of occupational therapy (chair), managers of the two employment services, team leaders of the three mental health teams, and employment service and mental health staff from all three sites
Frequency and intensity of support
The steering committee met monthly for the first 12 months of the program and then two monthly for the next 12 months
Each site was allocated 0.5 FTE for a local mental health Team Leader or occupational therapist to be the IPS coordinator. At Site A the position was filled by an occupational therapist with good support from the mental health Team Leader. At Site B, the position was filled by an occupational therapist with good support from the Team Leader who was also an occupational therapist. At Site C, the position was filled by an occupational therapist who received limited support from the mental health Team Leader An occupational at each site provided onsite support: Site A, 3 h per week – desk space provided for employment consultant within team office; Site B, 5 h per week (plus direct support from the team leader) – desk space provided within team office; Site C, 30 min per week – desk space provided outside team office (treatment room)
Nature of support provided
1. Regular meetings provide stakeholders with opportunities to identify system problems to program delivery and to solve these at a regional level 2. Third parties such as the Government Income Support provider (Centrelink) are invited to this meeting to help solve systemic issues affecting employment service providers and mental health service consumers
was the proportion commencing competitive employment, the proportion commencing other vocational activity was also recorded. Other vocational activity included non-competitive employment (sheltered employment, voluntary work and unpaid work experience) and enrolment in formal education or training courses. Job diversity was measured by comparing the number of different job types to the total number of jobs recorded. Independent variables collected at an individual level included: age, sex, diagnostic category, date of referral, date of program commencement, reasons for non-acceptance of referrals, time to commence job seeking and time to commence the first job. Fidelity to evidence-based practices in supported employment was measured at each site using the Supported Employment Fidelity Scale (Bond et al., 1997). This measure consists of 15 items covering staffing (three items), organisation (three items) and services © 2015 Occupational Therapy Australia
1. Facilitate communication between mental health staff and employment services staff 2. Update an evaluation database with details of all new referrals, health information and employment progress 3. Assist the employment specialist to engage with mental health consumers and obtain medical information from the mental health teams for mandatory program eligibility assessments. Support the employment specialist, and facilitate their acceptance into the mental health team 4. Provide other interventions when needed to support the vocational goals of consumers. 5. Advocate for consumers’ vocational goals and support employment specialists to assist all interested consumers
(nine items). Each item is rated from one to five where five represents best practice. Mental health staff, from teams not involved in the study, were trained in the use of this measure. Each assessor was then asked to complete the IPS Fidelity scale for one site using a variety of information sources, including existing records such as minutes of meetings, and interviews with staff of both mental health and employment services. Each site had one IPS fidelity review during the period of data collection in this paper. Fidelity results at the item level are shown in Table 2.
Resources No additional resources were provided to mental health teams or to the partner disability employment services. Disability employment services provided the dedicated employment specialist from their existing resources. Equal amounts of in-kind support were offered to each site by the program leaders and by the
321
EVIDENCE-BASED SUPPORTED EMPLOYMENT
regional steering group. However, due to local differences in leadership and staffing, each site differed in how the program was actually governed and supported at each site. For example, in addition to the regular regional steering group meetings Site A held monthly meetings between the occupational therapist and the employment specialist. Site B held weekly meetings involving the occupational therapist, the employment specialist and the mental health Team Leader. No additional meetings were arranged to support Site C. Challenges in leading and resourcing this type of program have been documented (Waghorn, Lockett, Bacon, Gorman & Durie, 2009b).
Results Sites A and C reported similar participant characteristics. A higher proportion of people diagnosed with bipolar affective disorder were found at Site A compared with Site C (38% vs. 8%). Site B is a dedicated early psychosis service and this was reflected both in the higher proportions diagnosed with a primary psychotic disorder (92%), and the younger mean age of 22 years (see Table 1). Site B achieved good fidelity to IPS principles while sites A and C achieved fair fidelity (see Table 2). However, all sites show room for improvement in the application of evidence-based practices, with respect to at least six or more of the 15 practices examined. All three
sites had room to increase integration between mental health and vocational services. This could be achieved at each site by co-locating the dedicated employment specialist four days per week into the partner mental health team. All sites were allocated the same in-kind resources to support the implementation of the IPS model. However, these resources were utilised differently at each site. Site B provided the most support on a weekly basis, followed by Site A, and Site C (see Table 3). The support provided by the Early Psychosis team leader at Site B included discussing all referrals with mental health case managers, and meeting fortnightly with the employment specialist to share relevant information and to review and update evaluation data. This ranking of sites (B>A>C) also held with respect to IPS fidelity (see Table 2) and also reflected relative performance on the primary outcome variable, namely the proportion of participants commencing competitive employment (see Table 4). Primary outcomes in Table 4 indicate that Site B achieved the highest proportion of participants commencing competitive employment, at 37.0% compared with 33.3% at Site A and 12% at Site C. Site A had the highest proportion (70.3%) of participants attaining any vocational outcome (competitive employment, other vocational outcome, or formal study and training), compared with Site B (55.5%), and Site C (20.0%) (see Table 4).
TABLE 4: Employment outcomes by site Employment outcomes
Site A
Site B
Site C
Number of referrals received in 21 months Program commencements Non-commencement reason Participant declined Ineligible for disability Employment services program Unknown Commenced competitive employment Commenced any vocational activity including competitive employment Completed 13 weeks competitive employment Time taken from referral to commence job searching in days, Mean (SD) Time in days from referral to commencing competitive employment, Mean (SD) Number of jobs held by those who commenced employment, Mean (SD) Hours worked per week, Mean (SD) Job diversity: the number of different job types compared with the total number of jobs
52 27/52 (51.9%)
28 27/28 (96.4%)
34 25/34 (73.5%)
11/25
1/1
7/9
4/25 10/25 9/27 (33.3%) 19/27 (70.3%)
10/27 (37.0%) 15/27 (55.5%)
0/9 2/9 3/25 (12.0%) 5/25 (20.0%)
8/27 (29.6%) 51.2 (39.8)
4/27 (14.8%) 40.4 (17.6)
1/25 (4.0%) 47.2 (32.3)
97.3 (31.8)
91.4 (82.8)
275.6 (296.3)
1.5 (0.8), Range 1–3
1.6 (0.9), Range 1–3
Unknown
19.8 (5.6) n = 9 8/9
23.0 (11.5), n = 10 8/10
11.0 (3.6), n = 3 1/3
© 2015 Occupational Therapy Australia
322 In terms of completing 13 weeks employment, site A (29.6%) outperformed both sites B (14.8%) and C (4.0%). This result suggests that retaining employment is more challenging for Site B compared with site A. The results also show performance at site C to be disappointing on all metrics except the proportion of referrals that converted to actual program commencements (73.5%). Site C also had the lowest fidelity score and received the least amount of support from the mental health team (see Table 4).
Discussion Kennedy-Jones, Cooper and Fossey (2005) explored the experience of engaging in work from a mental health service user point of view. They concluded that health benefits for the individual were significant and urged occupational therapists to provide ongoing support to people with severe mental illness who seek employment. Introducing the IPS model as a means to promote vocational recovery also provides opportunities to occupational therapists to support these important service user goals. In this project, promising employment outcomes were achieved at sites A and B (see Table 4). Yet these results are disappointing in comparison to previous Australian trials of IPS (Killackey et al., 2008; Morris et al., 2014; Waghorn et al., 2014) and with respect to USA trials (Bond et al., 2012a). As Morris et al. (2014) have shown, a competent co-located employment specialist supported by a strong partnership with good fidelity to evidence-based practices, and with regular external technical support, can enable 60% or more participants referred to an IPS program to commence competitive employment. That two of the three sites in this study achieved just over half of this target indicates that much more can be done to improve employment outcomes for Gold Coast participants. Sites A and C were similar in terms of client mix (see Table 1). However, there were marked differences with respect to integration with mental health services, strength of local leadership, and commitment to evidence-based practices. Sites A and B achieved higher fidelity to IPS practices. In addition, the ranking in terms of amount of in-kind support to the program where Site B>Site A>Site C, was consistent with fidelity scores and overall performance on the primary outcome. The lack of support for the program at Site C, and its subsequent fair fidelity, seemed to predetermine its disappointing results. Site B represented the strongest local leadership and most engagement between the treating team and the disability employment service provider. However, Site B did not perform as well as site A in terms of job retention, or in terms of any vocational outcome. The former appears related to the diagnostic mix of mostly young people with first episode psychosis. However, the difference in terms of any © 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
vocational outcomes most reflects a lack of focus on competitive employment at Site A. These results suggest that the program could be strengthened at all three sites through: (i) full-time co-location of the employment specialist within each mental health team; (ii) better integration of mental health services with employment services building on full-time co-location and (iii) more technical support for each site for implementing other evidence-based practices (see Table 2). While the regional steering committee provided overall governance and a strategic direction for the program, its existence did not appear to positively influence employment outcomes at each site. It appears that the strength of leadership provided at a site level may be a factor in both better implementation and in the better primary outcomes attained. Any allied health discipline such as Psychology, Mental Health Nursing, Social Work, Speech Therapy or Physiotherapy could in principle drive this system change. However, other allied health professionals in this study seemed to step back to allow occupational therapists to take the lead. Occupational therapists were well suited to supporting this program because the teams with the greatest involvement by occupational therapists also achieved the best results. This is consistent with prior implementations of IPS in Australia and in New Zealand, where occupational therapists have been critically involved as: program advocates; program leaders; program evaluators (Morris et al., 2014); site champions; trainers; mental health team leaders; researchers (Waghorn et al., 2012, 2014); employment specialists and mental health case managers (Porteous & Waghorn, 2007, 2009).
Limitations The main limitation of the evaluation design is that it was an implementation study, not a randomised controlled trial. In a controlled trial, every effort would normally be made to control for between site differences such as different use of resources in supporting the program at a local level. However, this was an observational study of an implementation strategy where the investigators had little control over what happened within each site. Although issues relating to a particular site could be raised at the regional steering committee meeting, it was not possible to force change on any particular site or partnership. Hence, in this study, the actual strength of implementation of IPS achieved at each site is best considered a mediating variable rather than as an independent variable or independent covariate. In terms of analysis, we relied mostly upon descriptive differences and anecdotal impressions, because power was too low to confidently compare sites for statistically significant differences. While this is appropriate for generating ideas to improve the program, it is not sufficient for reaching conclusions about program
323
EVIDENCE-BASED SUPPORTED EMPLOYMENT
effectiveness in this context. We also note that data quality could be managed by monitoring the database and its updating more regularly to maintain optimal accuracy of information, to prevent missing information and to discuss fidelity scores more regularly with assessors to achieve greater consistency of fidelity scoring across sites. The quality of implementation was assessed using an earlier 15-item version of the IPS fidelity scale (Bond et al., 1997) that has since been updated to a potentially more useful 25-item version (Bond et al., 2012b). The most salient shortcoming was the failure of each site to achieve co-location of the employment specialist for four of five days per week in the mental health team premises. The aim of this step is to provide the basis for ongoing informal communication with the mental health team staff and with service users. Although the employment specialist is expected to be out of the office for 70% of the time visiting employers, co-location has been found useful for developing seamless coordination of the two services. This in term is expected to lead to greater service integration through the increased opportunities for clinicians to consider vocational issues, and the opportunities for the employment specialist to understand the clinical issues that may be impacting on work performance (Sherring, Robson, Morris, Frost & Tirupati, 2010; Waghorn et al., 2007, 2012). Other weak practices were working as a vocational unit, zero exclusion criteria and community-based services. Not working as a vocational unit, suggests a lack of regular supervision for employment specialists that is needed to ensure they develop positive relationships with local employers. Not sufficiently implementing the zero exclusion principle was also reflected in the proportion of referrals that did not lead to program commencements. This is important because not gaining timely access to a sought after program can adversely impact on service user motivation. In addition, the failure to implement the practice of community-based services that can leave employment specialists office bound and out of touch with local employers. It can also lead to a reliance on participants finding their own jobs, rather than the employment specialist generating opportunities matching job seeker preferences, from a well-known network of employers.
Implementation challenges Implementing evidence-based supported employment at the Gold Coast involved several practical challenges. The initial process of establishing a formal agreement between the Gold Coast mental health service and the disability employment services took approximately two years. This delay meant it was difficult to gather momentum and difficult to sustain any gains from efforts to promote the program. The program had official executive support from the health service, but lacked an identified executive sponsor to advocate for
the program and drive decisions supporting high-fidelity implementation. As a result, operational barriers emerged that were difficult to resolve quickly resolve. These included: procedures for managing confidential information; physical space and facilities for the employment specialists within each mental health team; and keeping to an agreed schedule for employment specialists to attend the health premises. The challenging nature of the Gold Coast labour market may have impacted on employment outcomes. The Australian Bureau of Statistics (2011) reported that background unemployment in the Gold Coast area at the time of this study was 7.4% compared with the greater Brisbane area of 5.4%. In addition, the Gold Coast has three transient elements to its population, consisting of students, young people on working holidays and tourists. The first two groups compete for entry level employment opportunities. Whilst, it is unlikely that adverse labour market factors explain these results, it is possible that the failure to achieve high IPS fidelity made all sites more prone to adverse effects from these labour market factors. Just such an effect has been suggested previously to explain why the more intensive IPS practices are less prone to moderation by labour market factors (Bond, 2004; Bond et al., 2008).
Opportunities for occupational therapists This study reveals the many ways occupational therapists can be involved to support the introduction of evidence-based practices in supported employment for adults with severe mental illness. The evidence suggests that the IPS model can be implemented in Australia using a partnership approach, provided persistent efforts are made to establish and maintain high-fidelity IPS practices. While there are other approaches to vocational rehabilitation, they are typically less than half as successful as the IPS approach in attaining competitive employment for people with severe mental illness (Bond et al., 2012a; Waghorn & Lloyd, 2005). This study highlights the importance of strong local leadership in supporting the introduction of evidencebased practices in supported employment. Of all the allied health disciplines, occupational therapy seems best placed to provide this leadership within mental health services. Once a good knowledge of IPS practices is acquired occupational therapists can also assist staff of disability employment services in adopting these practices. There is potential for the role of facilitating the integration of supported employment into community mental health services, to become a major role for occupational therapists, complementing their existing work in mental health case management, treatment and care and general psychiatric rehabilitation. Consequently, it may be important to ensure that undergraduate and postgraduate programs adequately prepare students for these roles particularly for those seeking to specialise in mental health practice. © 2015 Occupational Therapy Australia
324
Conclusion IPS implementation using existing systems and resources in Australia is challenging yet achievable. It requires two existing services to operate in close partnership to jointly deliver a new service that complements existing mental health treatment, care and rehabilitation services. The evidence suggests that this requires strong local leadership, good knowledge of IPS principles and practices, local technical support and a strong commitment by all stakeholders to adopting all the evidence-based practices in supported employment. In addition, ongoing evaluation is critical, not only to monitor fidelity with evidence-based practices but also to ensure that the adoption of the new IPS practices actually leads to the level of employment outcomes expected. Occupational therapists working in both community mental health services and in disability employment services have new opportunities to advocate, implement, support and facilitate this essential yet challenging program.
References Australian Bureau of Statistics. (2011). Census of Population and Housing: 2006–2011. Retrieved from: http://profile.id.com.au/gold-coast/employment-status on 14/4/14. Bertram, M. & Howard, L. (2006). Employment status and occupational care planning for people using mental health services. Psychiatric Bulletin, 30, 48–51. doi: 10. 1192/pb.30.2.48 Bond, G. (2004). Supported employment: Evidence for an evidence based practice. Psychiatric Rehabilitation Journal, 27 (4), 345–359. doi: 10.2975/27.2004.345.359 Bond, G. R., Becker, D. R., Drake, R. E. & Vogler, K. (1997). A fidelity scale for the Individual Placement and Support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265–284. Bond, G. R., Drake, R. E. & Becker, D. R. (2008). An update on randomised controlled trials of evidence-based supported employment. Psychiatric Rehabilitation Journal, 31, 280–290. doi: 10.2975/31.4.2008.280.290 Bond, G. R., Drake, R. E. & Becker, D. R. (2012a). Generalizability of the Individual Placement and Support (IPS) model of supported employment outside the US. World Psychiatry, 11 (1), 32–39. doi: 10.1016/j.wpsyc.2012.01.005 Bond, G. R., Peterson, A. E., Becker, D. R. & Drake, R. E. (2012b). Validating the revised Individual Placement and Support fidelity scale (IPS-25). Psychiatric Services, 63, 758–763. doi: 10.1176/appi.ps.201100476 Gladman, B., Waghorn, G., Wishart, L. & Dias, S. (2015). Reliability of a measure of health professionals’ perceptions of integrated employment services for people with psychiatric disabilities. Journal of Rehabilitation, 81 (1), 3–8. Kennedy-Jones, M., Cooper, J. & Fossey, E. (2005). Developing a worker role: Stories of four people with mental illness. Australian Occupational Therapy Journal, 52 (2), 116–126. doi: 10.1111/j.1440-1630.2005.00475.x
© 2015 Occupational Therapy Australia
P. L. WILLIAMS ET AL.
Killackey, E., Jackson, H. J. & McGorry, P. D. (2008). Vocational intervention in first-episode psychosis: Individual placement and support vs. treatment as usual. British Journal of Psychiatry, 193 (2), 114–120. doi: 10.1192/bjp. bp.107.043109. Kinoshita, Y., Furukawa, T. A., Kinoshita, K., Honyashiki, M., Omori, M., Marshall, M. et al. (2013). Supported employment for adults with severe mental illness. Cochrane Database of Systematic Review, 2013 (9), doi: 10.1002/ 14651858.CD008297.pub2. Lloyd, C. & Waghorn, G. (2008). Reconfiguring evidencebased practice of occupational therapists - commentary. International Journal of Therapy and Rehabilitation, 15 (10), 436. Lloyd, C., Deane, F., Tse, S. & Waghorn, G. (2009). Supporting recovery oriented services for people with severe mental illness. Australian Journal of Rehabilitation Counselling, 15 (1), 1–11. doi: 10.1375/jrc.15.1.1. Morris, A., Waghorn, G., Robson, E., Moore, L. & Edwards, E. (2014). Implementation of evidence-based supported employment in regional Australia. Psychiatric Rehabilitation Journal, 37 (2), 144–147. doi: 10.1037/prj0000051. Porteous, N. & Waghorn, G. (2007). Implementing evidence-based employment services in New Zealand for young adults with psychosis: Progress during the first five years. British Journal of Occupational Therapy, 70, 521– 526. doi: 10.1177/030802260707001204. Porteous, N. & Waghorn, G. (2009). Developing evidencebased supported employment services for young adults receiving public mental health services. New Zealand Journal of Occupational Therapy, 56 (1), 34–39. Queensland Health. (2008). Queensland plan for mental health 2007–2012. Brisbane: Mental Health Branch, Queensland Health. Rinaldi, M., Miller, L. & Perkins, R. (2010). Implementing the individual placement and support (IPS) approach for people with mental health conditions in England. International Review of Psychiatry, 22 (2), 163–172. doi:10.3109/ 09540261003720456. Sherring, J., Robson, E., Morris, A., Frost, B. & Tirupati, S. (2010). A working reality: Evaluating enhanced intersectoral links in supported employment for people with psychiatric disabilities. Australian Occupational Therapy Journal, 57, 261–267. doi: 10.1111/j.1440-1630.2009.00844.x. Tsang, H. W. H., Chan, A., Wong, A. & Liberman, P. L. (2009). Vocational outcomes of an integrated supported employment programme for individuals with persistent and severe mental illness. Journal of Behaviour Therapy and Experimental Psychiatry, 40, 292–305. doi: 10.1016/j.jbtep. 2008.12.007. Waghorn, G. & Lloyd, C. (2005). The employment of people with mental illness. Advances in Mental Health, 4 (2), 129– 171. doi: 10.5172/jamh.4.2.129. Waghorn, G., Chant, D. & Jaeger, J. (2007). Employment functioning and disability among community residents with bipolar affective disorder : results from an Australian community survey. Bipolar Disorders, 9 (1), 166–182. doi: 10.1111/j.1399-5618.2007.00417.x. Waghorn, G., Lloyd, C. & Clune, A. (2009a). Reviewing the theory and practice of occupational therapy in mental
EVIDENCE-BASED SUPPORTED EMPLOYMENT
health rehabilitation. British Journal of Occupational Therapy, 72, 314–322. doi: 10.1177/030802260907200708. Waghorn, G., Lockett, H., Bacon, J., Gorman, P. & Durie, S. (2009b). The importance of leadership in implementing evidence-based supported employment services for people with severe mental illness. The International Journal of Leadership in Public Services, 5 (suppl), 51–56. Waghorn, G., Stephenson, A. & Browne, D. (2011). The importance of service integration in developing effective employment services for people with severe mental health conditions. British Journal Of Occupational Therapy, 74 (1), 339–347. doi: 10.4276/030802211X13099513661153.
325 Waghorn, G., Childs, S., Hampton, E., Gladman, B., Greaves, A. & Bowman, D. (2012). Enhancing community mental health services through formal partnerships with supported employment providers. American Journal of Psychiatric Rehabilitation, 15 (2), 157–180. doi: 10.1080/ 15487768.2012.679582. Waghorn, G., Saha, S. & McGrath, J. (2014). Correlates of competitive versus non-competitive employment among adults with psychotic disorders. Psychiatric Services, 65, 476–482. doi: 10.1176/appi.ps.201300096.
© 2015 Occupational Therapy Australia
