Acad Psychiatry DOI 10.1007/s40596-014-0069-z
COLUMN: EDUCATIONAL CASE REPORT
Quality Improvement (QI) Education in a Psychiatry Residency Program: “Diagnosis and Consultation” as Key QI Skills Susan J. Stagno & Mark Cheren
Received: 5 November 2013 / Accepted: 23 February 2014 # Academic Psychiatry 2014
Systematic approaches to quality improvement (QI) have been an important part of the health care field for many years, but teaching the skills and providing experience in QI are relatively new to residency training. The development of curriculum for and assessment of these competencies is particularly challenging for residency programs [1], and residency training directors struggle with how best to introduce these concepts, develop these skills, and provide adequate guidance to residents in time-efficient ways [2]. These efforts have taken on new importance for both residents and practicing clinicians as the demonstration of competence in this area has come to be required by accrediting and certifying bodies across the health professions education continuum. The ACGME core competencies include systems-based practice (SBP) and practicebased learning and improvement (PBLI) [3], which involve QI skills, and the American Board of Psychiatry and Neurology (ABPN) now assesses “performance in practice” (a form of QI) as part of maintenance of certification [4]. Most residents are new to QI, and the methods and language used can seem somewhat unfamiliar to them, initially. Taking an approach to this topic that allows them to appreciate that they already possess many of the skills and much of the knowledge necessary to plan and implement meaningful QI projects can reduce the unfamiliarity and attendant anxiety, and it can foster engagement.
Method The informing design principle in introducing QI in this curriculum is to encourage residents to view QI as a S. J. Stagno (*) : M. Cheren University Hospitals Case Medical Center, Cleveland, OH, USA e-mail: [email protected]
systematic way to “diagnose” and respond to problems in practice procedures, processes, and systems and to see that this approach is directly analogous to the approaches they have been learning to employ addressing the clinical issues of their patients. At the same time, as they work on a QI project, they are strongly encouraged to identify the help they need and to reach out for that help just as routinely as they would seek consults for challenging clinical issues. It is in this respectful and supportive context that they are also asked to learn more about systematic quality improvement. This educational program uses four modalities: 1. Group seminars: QI seminars are incorporated into the educational schedule for third and fourth year residents. During the initial session, residents are introduced to the concept of QI by two faculty, a psychiatrist and a medical educator with expertise in QI. The purpose and rationale of QI is explained along with several key QI concepts including the difference between QI and research and the “Plan, Do, Study, Act” (PDSA) cycle. Expectations, including independent study using online modules and involvement in a QI project, are laid out, and resources available in the program/department as well as the larger hospital system are discussed. Residents also complete the Healthcare Improvement Skills Center (HISC) Quality Improvement Skills Self-Assessment Inventory at this first session. The inventory allows them to (1) recognize the QI skills they already possessed, (2) anticipate areas where they might want help, (3) articulate the steps and strategies used in carrying out a QI project, and (4) begin to develop specific learning goals. During the next three seminars, the content of the online modules is discussed and QI projects are selected. Resident
Acad Psychiatry
teams select a faculty coach by the end of this sequence and complete plans for their respective projects using a prescribed format. Additional in-class sessions are scheduled at approximately 2 to 3-month intervals to provide a venue in which residents can obtain peer and QI seminar faculty “consultation” and report progress with their projects. 2. Independent work online: Six learning modules available through the Healthcare Improvement Skills Center website [5] provide much of the “didactic” portion of the learning through a web-based curriculum. These case-driven, interactive modules cover identifying the problem, setting a precise objective, building a team, selecting appropriate measures, collecting and interpreting data (including run charts), midcourse corrections, and sustaining and spreading results (when appropriate). 3. Involvement in a QI project: Residents select situations in which they observed procedural or systems issues that created impediments to efficient care or best practices in clinical settings where they are currently providing care, or in which they have previously rotated. Projects that residents have completed included (1) using a preclinical encounter patient questionnaire to deliver more time-efficient care in a community mental health center, (2) addressing smoking cessation in a free clinic, (3) clinically and legally appropriate completion of involuntary commitment forms, and (4) developing an intake process for a resident psychotherapy clinic. Project residents are currently involved with (1) identifying the most frequent reasons for hospital readmissions to a psychiatry inpatient unit and designing interventions to reduce readmission rates and (2) developing guidelines for management of prolonged corrected QT (QTc) intervals for patients requiring psychotropic medications that could contribute to prolonged QTc. They are given a time line of 6–12 months to complete their projects. 4. Faculty coaching: Faculty who volunteer to coach residents receive instruction by way of a process that include many of the same methods and resources provided to residents. They are provided with a manual that concisely describes the QI process as well as worksheets that help guide residents through their projects. The role of the coach is to guide resident teams through measurable, short-cycle interventions. The coach may serve as part of their QI project team, or may choose to remain advisory only. Coaches continue to meet approximately two to three times yearly to review QI concepts and share their experiences working with residents.
The figure which follows depicts the interplay of these modalities as the program progresses (Fig. 1). Two classes of residents at the PGY3 and PGY4 levels during one academic year participated in the educational program (10 residents total, 100 % participation). All participants completed the self-assessment instrument (HISC QI Skills Self-assessment Inventory) at the first meeting of the seminar, and a postintervention program evaluation developed for this pilot on the last day of the seminar. The HISC inventory includes six categories (describe the issue, build a team, define the problem, choose the target, test the change, reconsider, or extend improvement efforts) with 19 specific skills (paralleling the online learning modules and their respective learning objectives) important in QI. On a Likert scale of 0 (indicating no skills) to 5 (indicating expert), residents rated their perception of their current skills and also the goal they wished to reach. This project was reviewed by the University Hospitals Institutional Review Board and approved as an exempted study. Residents participated anonymously after giving informed consent.
Results Responses to the QI skills self-assessment revealed that the category in which residents felt most skilled was in “describing the issue,” and the category they rated themselves as least skilled was in “reconsidering or extending the improvement efforts,” followed closely by “choosing the target” which includes skills of collecting, aggregating, displaying, and analyzing the data. A post-intervention survey was administered that asks whether the experience helped to reduce or eliminate concerns about improving their practices in a systematic way, identifying their strengths in QI work, and if their participation contributed to an improvement project and satisfied the requirements of the program. The survey queries them about their expectation of continuing to engage in future QI work and if they expect to train others in QI. Finally, the survey asks about whether the process of learning QI was worthwhile, if the coaching aspect was helpful, and whether the resources (online and in-class) were useful. No resident indicated that this experience was not helpful or that he or she did not meet the goal of developing their QI skills. The majority (83 %) of residents indicated that this experience helped them to be significantly more comfortable in asking for help from others with QI work and that their participation contributed to the enhancement of one or more improvement projects. Fifty percent of residents reported that the improvement skills modules were “very helpful,” and 50 % indicated they were “somewhat helpful.” All residents indicated that they expected to continue in systematic improvement efforts after completing residency, and most residents (83 %) indicated that they intend to train others in QI following this experience.
Acad Psychiatry
Quality Improvement Training Activities for Residents Introductory Session* • Describe why QI is important • Complete QI skills self- assessment • Identifying QI skills strengths, areas where help or learning are needed • Begin to consider QI project and coach
On Line Learning Healthcare Improvement Skills Center (HISC) website
Coaching
Facilitated
In Person
Peer Consulting Sessions*
By Phone or Computer By eMail As Needed
Discuss progress on projects and HISC modules; set new goals 60 Min. Sessions
* During residency educational seminar
Coaching by Faculty and Fellows
1st Coaching Session In Person • Talk about project ( or support selection) • Go over resident’s QI Summary together • Draw up or refine action plan • Identify needed resources • Set goals & time for next meeting
Subsequent
Coaching Sessions
• Consult • Set goals & time for next session
Approximately 60 min.
15 – 60 min. (as needed)
Fig. 1 Depicts the interplay of modalities as the program progresses
Discussion Both a systematic review [6] and a study looking at a curriculum for psychiatry residents [7] show that combining adultlearning principles and modalities are effective in teaching QI. Some lessons learned in piloting this multimodality approach included the need to set tighter time lines and clearer expectations about progress through the modules and on the projects themselves. Requiring completion of the six modules of online learning prior to the third seminar lays the foundation for the development of core skills early and moves the process along more rapidly. Residents often tended to "think big" and have trouble limiting their projects to achievable short-cycle efforts. Strong guidance and, if necessary, direction from faculty with this aspect appears warranted. Several had trouble identifying baseline measurement of a systems problem before beginning their intervention. Therefore, we addressed this issue during several seminars and introduced them to a resource person
with measurement expertise early on who served as a consultant to them throughout the balance of the program. The small size of this pilot limits the generalizability of this experience, as does the absence of objective, uniform data concerning participants’ skill development, and the ultimate quality of improvement team efforts and/or project outcomes. What we do have is modest proof of concept, worthy of larger and more rigorous replication. In addition, gathering data from the faculty coaches to assess their skills before involvement and after coaching a resident team would be useful in future studies. Overall, participants described their engagement in this program as a positive experience and appreciated the fact that this mechanism enhanced the quality of their own practice experience in one of the settings through which they rotated by allowing them to serve as a change agent on behalf of their patients’ well-being, thereby potentially contributing to the improvement of patient care. Further, the majority indicate that they intend to continue to participate in QI and are likely
Acad Psychiatry
to train others in systematic improvement in the future, which we find to be a positive outcome. Implications for Educators: & QI can seem like a foreign concept to residents. Helping them to see that clinical skills they possess (diagnosing, consulting) can be applied to systems, processes, and procedures can engage them more quickly and more comfortably. & Allowing residents to identify quality issues in their own practice environments as the focus for completion of a project engages them because they are invested in the outcome. & Using multiple pedagogical approaches including online learning, seminars, coaching, and project engagement provides for timeefficient and effective teaching and learning.
Disclosures Mark Cheren is president of Improvement Learning, LLC and Editor in Chief of Improvement Learning’s Healthcare Improvement Skills Center (HISC). The HISC website is a resource used in the educational program described in this article. Susan Stagno has nothing to disclose.
References 1. Tomolo AM, Lawrence RH, Aron DC. A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum. Qual Saf Health Care. 2009;18:217–24. 2. Heard J, Allen R, Clardy J. Assessing the needs of residency program directors to meet the ACGME general competencies. Acad Med. 2002;77:750. 3. Accreditation Council for Graduate Medical Education Common Program Requirements, Section IV, Subsection A.5.c).(4), 2013; p.8. 4. American Board of Psychiatry and Neurology Maintenance of Certification; available at http://www.abpn.com/moc.html. 5. Healthcare Improvement Skills Center; available at http:// improvementskills.org. 6. Boonyasai R, Windish D, Chakroborti C, et al. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023–37. 7. Arbuckle MR, Weinberg M, Cabaniss DL, et al. Training psychiatry residents in quality improvement: an integrated, year-long curriculum. Acad Psychiatry. 2013;37:42–5.
COLUMN: EDUCATIONAL CASE REPORT
Quality Improvement (QI) Education in a Psychiatry Residency Program: “Diagnosis and Consultation” as Key QI Skills Susan J. Stagno & Mark Cheren
Received: 5 November 2013 / Accepted: 23 February 2014 # Academic Psychiatry 2014
Systematic approaches to quality improvement (QI) have been an important part of the health care field for many years, but teaching the skills and providing experience in QI are relatively new to residency training. The development of curriculum for and assessment of these competencies is particularly challenging for residency programs [1], and residency training directors struggle with how best to introduce these concepts, develop these skills, and provide adequate guidance to residents in time-efficient ways [2]. These efforts have taken on new importance for both residents and practicing clinicians as the demonstration of competence in this area has come to be required by accrediting and certifying bodies across the health professions education continuum. The ACGME core competencies include systems-based practice (SBP) and practicebased learning and improvement (PBLI) [3], which involve QI skills, and the American Board of Psychiatry and Neurology (ABPN) now assesses “performance in practice” (a form of QI) as part of maintenance of certification [4]. Most residents are new to QI, and the methods and language used can seem somewhat unfamiliar to them, initially. Taking an approach to this topic that allows them to appreciate that they already possess many of the skills and much of the knowledge necessary to plan and implement meaningful QI projects can reduce the unfamiliarity and attendant anxiety, and it can foster engagement.
Method The informing design principle in introducing QI in this curriculum is to encourage residents to view QI as a S. J. Stagno (*) : M. Cheren University Hospitals Case Medical Center, Cleveland, OH, USA e-mail: [email protected]
systematic way to “diagnose” and respond to problems in practice procedures, processes, and systems and to see that this approach is directly analogous to the approaches they have been learning to employ addressing the clinical issues of their patients. At the same time, as they work on a QI project, they are strongly encouraged to identify the help they need and to reach out for that help just as routinely as they would seek consults for challenging clinical issues. It is in this respectful and supportive context that they are also asked to learn more about systematic quality improvement. This educational program uses four modalities: 1. Group seminars: QI seminars are incorporated into the educational schedule for third and fourth year residents. During the initial session, residents are introduced to the concept of QI by two faculty, a psychiatrist and a medical educator with expertise in QI. The purpose and rationale of QI is explained along with several key QI concepts including the difference between QI and research and the “Plan, Do, Study, Act” (PDSA) cycle. Expectations, including independent study using online modules and involvement in a QI project, are laid out, and resources available in the program/department as well as the larger hospital system are discussed. Residents also complete the Healthcare Improvement Skills Center (HISC) Quality Improvement Skills Self-Assessment Inventory at this first session. The inventory allows them to (1) recognize the QI skills they already possessed, (2) anticipate areas where they might want help, (3) articulate the steps and strategies used in carrying out a QI project, and (4) begin to develop specific learning goals. During the next three seminars, the content of the online modules is discussed and QI projects are selected. Resident
Acad Psychiatry
teams select a faculty coach by the end of this sequence and complete plans for their respective projects using a prescribed format. Additional in-class sessions are scheduled at approximately 2 to 3-month intervals to provide a venue in which residents can obtain peer and QI seminar faculty “consultation” and report progress with their projects. 2. Independent work online: Six learning modules available through the Healthcare Improvement Skills Center website [5] provide much of the “didactic” portion of the learning through a web-based curriculum. These case-driven, interactive modules cover identifying the problem, setting a precise objective, building a team, selecting appropriate measures, collecting and interpreting data (including run charts), midcourse corrections, and sustaining and spreading results (when appropriate). 3. Involvement in a QI project: Residents select situations in which they observed procedural or systems issues that created impediments to efficient care or best practices in clinical settings where they are currently providing care, or in which they have previously rotated. Projects that residents have completed included (1) using a preclinical encounter patient questionnaire to deliver more time-efficient care in a community mental health center, (2) addressing smoking cessation in a free clinic, (3) clinically and legally appropriate completion of involuntary commitment forms, and (4) developing an intake process for a resident psychotherapy clinic. Project residents are currently involved with (1) identifying the most frequent reasons for hospital readmissions to a psychiatry inpatient unit and designing interventions to reduce readmission rates and (2) developing guidelines for management of prolonged corrected QT (QTc) intervals for patients requiring psychotropic medications that could contribute to prolonged QTc. They are given a time line of 6–12 months to complete their projects. 4. Faculty coaching: Faculty who volunteer to coach residents receive instruction by way of a process that include many of the same methods and resources provided to residents. They are provided with a manual that concisely describes the QI process as well as worksheets that help guide residents through their projects. The role of the coach is to guide resident teams through measurable, short-cycle interventions. The coach may serve as part of their QI project team, or may choose to remain advisory only. Coaches continue to meet approximately two to three times yearly to review QI concepts and share their experiences working with residents.
The figure which follows depicts the interplay of these modalities as the program progresses (Fig. 1). Two classes of residents at the PGY3 and PGY4 levels during one academic year participated in the educational program (10 residents total, 100 % participation). All participants completed the self-assessment instrument (HISC QI Skills Self-assessment Inventory) at the first meeting of the seminar, and a postintervention program evaluation developed for this pilot on the last day of the seminar. The HISC inventory includes six categories (describe the issue, build a team, define the problem, choose the target, test the change, reconsider, or extend improvement efforts) with 19 specific skills (paralleling the online learning modules and their respective learning objectives) important in QI. On a Likert scale of 0 (indicating no skills) to 5 (indicating expert), residents rated their perception of their current skills and also the goal they wished to reach. This project was reviewed by the University Hospitals Institutional Review Board and approved as an exempted study. Residents participated anonymously after giving informed consent.
Results Responses to the QI skills self-assessment revealed that the category in which residents felt most skilled was in “describing the issue,” and the category they rated themselves as least skilled was in “reconsidering or extending the improvement efforts,” followed closely by “choosing the target” which includes skills of collecting, aggregating, displaying, and analyzing the data. A post-intervention survey was administered that asks whether the experience helped to reduce or eliminate concerns about improving their practices in a systematic way, identifying their strengths in QI work, and if their participation contributed to an improvement project and satisfied the requirements of the program. The survey queries them about their expectation of continuing to engage in future QI work and if they expect to train others in QI. Finally, the survey asks about whether the process of learning QI was worthwhile, if the coaching aspect was helpful, and whether the resources (online and in-class) were useful. No resident indicated that this experience was not helpful or that he or she did not meet the goal of developing their QI skills. The majority (83 %) of residents indicated that this experience helped them to be significantly more comfortable in asking for help from others with QI work and that their participation contributed to the enhancement of one or more improvement projects. Fifty percent of residents reported that the improvement skills modules were “very helpful,” and 50 % indicated they were “somewhat helpful.” All residents indicated that they expected to continue in systematic improvement efforts after completing residency, and most residents (83 %) indicated that they intend to train others in QI following this experience.
Acad Psychiatry
Quality Improvement Training Activities for Residents Introductory Session* • Describe why QI is important • Complete QI skills self- assessment • Identifying QI skills strengths, areas where help or learning are needed • Begin to consider QI project and coach
On Line Learning Healthcare Improvement Skills Center (HISC) website
Coaching
Facilitated
In Person
Peer Consulting Sessions*
By Phone or Computer By eMail As Needed
Discuss progress on projects and HISC modules; set new goals 60 Min. Sessions
* During residency educational seminar
Coaching by Faculty and Fellows
1st Coaching Session In Person • Talk about project ( or support selection) • Go over resident’s QI Summary together • Draw up or refine action plan • Identify needed resources • Set goals & time for next meeting
Subsequent
Coaching Sessions
• Consult • Set goals & time for next session
Approximately 60 min.
15 – 60 min. (as needed)
Fig. 1 Depicts the interplay of modalities as the program progresses
Discussion Both a systematic review [6] and a study looking at a curriculum for psychiatry residents [7] show that combining adultlearning principles and modalities are effective in teaching QI. Some lessons learned in piloting this multimodality approach included the need to set tighter time lines and clearer expectations about progress through the modules and on the projects themselves. Requiring completion of the six modules of online learning prior to the third seminar lays the foundation for the development of core skills early and moves the process along more rapidly. Residents often tended to "think big" and have trouble limiting their projects to achievable short-cycle efforts. Strong guidance and, if necessary, direction from faculty with this aspect appears warranted. Several had trouble identifying baseline measurement of a systems problem before beginning their intervention. Therefore, we addressed this issue during several seminars and introduced them to a resource person
with measurement expertise early on who served as a consultant to them throughout the balance of the program. The small size of this pilot limits the generalizability of this experience, as does the absence of objective, uniform data concerning participants’ skill development, and the ultimate quality of improvement team efforts and/or project outcomes. What we do have is modest proof of concept, worthy of larger and more rigorous replication. In addition, gathering data from the faculty coaches to assess their skills before involvement and after coaching a resident team would be useful in future studies. Overall, participants described their engagement in this program as a positive experience and appreciated the fact that this mechanism enhanced the quality of their own practice experience in one of the settings through which they rotated by allowing them to serve as a change agent on behalf of their patients’ well-being, thereby potentially contributing to the improvement of patient care. Further, the majority indicate that they intend to continue to participate in QI and are likely
Acad Psychiatry
to train others in systematic improvement in the future, which we find to be a positive outcome. Implications for Educators: & QI can seem like a foreign concept to residents. Helping them to see that clinical skills they possess (diagnosing, consulting) can be applied to systems, processes, and procedures can engage them more quickly and more comfortably. & Allowing residents to identify quality issues in their own practice environments as the focus for completion of a project engages them because they are invested in the outcome. & Using multiple pedagogical approaches including online learning, seminars, coaching, and project engagement provides for timeefficient and effective teaching and learning.
Disclosures Mark Cheren is president of Improvement Learning, LLC and Editor in Chief of Improvement Learning’s Healthcare Improvement Skills Center (HISC). The HISC website is a resource used in the educational program described in this article. Susan Stagno has nothing to disclose.
References 1. Tomolo AM, Lawrence RH, Aron DC. A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum. Qual Saf Health Care. 2009;18:217–24. 2. Heard J, Allen R, Clardy J. Assessing the needs of residency program directors to meet the ACGME general competencies. Acad Med. 2002;77:750. 3. Accreditation Council for Graduate Medical Education Common Program Requirements, Section IV, Subsection A.5.c).(4), 2013; p.8. 4. American Board of Psychiatry and Neurology Maintenance of Certification; available at http://www.abpn.com/moc.html. 5. Healthcare Improvement Skills Center; available at http:// improvementskills.org. 6. Boonyasai R, Windish D, Chakroborti C, et al. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023–37. 7. Arbuckle MR, Weinberg M, Cabaniss DL, et al. Training psychiatry residents in quality improvement: an integrated, year-long curriculum. Acad Psychiatry. 2013;37:42–5.
