ORIGINAL REPORTS
Putting Residents in the Office: An Effective Method to Teach the Systems-Based Practice Competency Marisa Pulcrano, BA,* A. Alfred Chahine, MD,*,† Amanda Saratsis, MD,* Jamie Divine-Cadavid, MS,* Vinod Narra, MD,* and Stephen R.T. Evans, MD* *
Department of Surgery, MedStar Georgetown University Hospital, Washington, District of Columbia; and Department of Surgery, Children’s National Health System, Washington, District of Columbia
†
OBJECTIVES: Systems-based practice (SBP) was 1 of 6 core
competencies established by the Accreditation Council for Graduate Medical Education and has proven to be one of the most difficult to effectively implement. This pilot study presents an immersion workshop as an effective tool to teach the SBP competency in a way that could easily be integrated into a residency curriculum. DESIGN: In 2006, 16 surgical residents rotated through 3
stations for 30 minutes each: coding and billing, scheduling operations and return appointments, and patient check-in. Participants were administered a pretest and posttest questionnaire evaluating their knowledge of SBP, and were asked to evaluate the workshop. SETTING: Outpatient clinic at MedStar Georgetown Uni-
versity Hospital, Washington, DC. PARTICIPANTS: Residents in the general surgery residency
training program at MedStar Georgetown University Hospital. RESULTS: Most residents (62.5%) improved their score after the workshop, whereas 31.25% showed no change and 6.25% demonstrated a decrease in score. Overall within their training levels, all groups demonstrated an increase in mean test score. Postgraduate year-2 residents demonstrated the greatest change in mean score (20%), whereas postgraduate year-4 residents demonstrated the smallest change in mean score (3.3%). CONCLUSIONS: An immersion workshop where general
surgery residents gained direct exposure to SBP concepts in situ was an effective and practical method of integrating this core competency into the residency curriculum. Such a workshop could complement more formal didactic teaching
Poster presentation at Surgical Education Week, The Association of Surgical Education, Washington, DC, April 10-14, 2007. Correspondence: Inquiries to A. Alfred Chahine, MD, Division of Pediatric Surgery, MedStar Georgetown University Hospital, 3800 Reservoir Road, NW # 4PHC, Washington, DC 20007; e-mail: [email protected]
286
and be easily incorporated into the curriculum. For example, this workshop could be integrated into the ambulatory care requirement that each resident must fulfill as part of C 2014 Associtheir clinical training. ( J Surg 72:286-290. J ation of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: systems-based practice, general surgery
residency, graduate medical education, competencies COMPETENCIES: Professionalism, Practice-Based Learning and Improvement, Systems-Based Practice
INTRODUCTION Training in systems-based practice (SBP) is 1 of the 6 core competencies mandated by the Accreditation Council for Graduate Medical Education (ACGME) of all participants of residency programs. The ACGME defines SBP as “awareness of and responsiveness to the larger context and system of health care, and the ability to effectively call on system resources to provide care that is of optimal value.”1 Competency in SBP is intended to enhance a physician’s effectiveness in the context of the modern health care system, providing knowledge of health care resources, delivery systems, advocacy, and access to care. Yet, of the 6 competencies, SBP was found to be the competency that residency program directors felt they needed the most assistance with when educating their residents.2 Program directors across specialties have integrated various creative methods in an effort to introduce these concepts, including interdepartmental conferences, small-group workshops, and role-play, research projects and presentations, and standardized patient interactions in clinical settings. Owing to its broad and varied nature, however, systems-based content has been difficult to incorporate into the graduate medical education curriculum. Measuring learning outcomes and
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.09.001
individual competency in SBP has proven equally challenging. A scientifically sound and reproducible method of integrating systems-based training has yet to be identified. Furthermore, we are not aware of any publications describing a surgical department’s successful implementation of an educational program designed to address their residents’ understanding of SBP. To explore methods by which residents in an academic surgical residency program may gain knowledge of and facility with SBP concepts, a workshop was designed whereby residents observed administrative staff in an outpatient clinic at MedStar Georgetown University Hospital. This pilot study presents our successful experience using an immersion workshop as an effective tool to teach the SBP competency to surgical residents, and our conclusion that such a tool can be easily integrated into the existing curriculum.
METHOD Residents in the General Surgery Residency Training Program at MedStar Georgetown University Hospital participated in this study in October 2006. Training levels ranged from postgraduate year (PGY)-2 to PGY-5. Participants were administered a questionnaire with 10 items at the outset of the workshop as a pretest (Fig. 1). The questions assessed the resident’s familiarity with common SBP situations that may arise in a surgical practice. The questions were multiple choice or true/false, apart from 2 open-ended questions. Following the pretest, residents participated in an immersion workshop that took place in the front and back offices of the outpatient clinic in the Department of Surgery. The workshop required residents to rotate through 3 stations for 30 minutes each: coding and billing, scheduling operations and
FIGURE 1. Questionnaire administered pretest and posttest to evaluate surgical residents’ knowledge of systems-based competency. Journal of Surgical Education Volume 72/Number 2 March/April 2015
287
return appointments, and patient check-in. We briefed the administrative staff before the workshop as to the expectations of the exercise, and residents were encouraged to ask questions to the staff. The entire workshop lasted 2 hours and took place during the residency training program’s protected time for medical education, thus all residents working at that time were invited to participate in this pilot study. At the conclusion of the workshop, participants completed the same questionnaire to demonstrate knowledge acquired through the activity. Raw pretest and posttest scores were compared for each individual, and mean scores were compared across training levels to determine the effectiveness of the workshop in increasing facility with SBP concepts. Each participant completed a short 10-point Likert scale evaluation of the workshop after the exercise was complete. The median and modes of the evaluation responses were then calculated.
greatest change in mean score (20% increase between pretest and posttest), whereas residents at the PGY-3 level demonstrated a 10% increase in scores between the pretest and the posttest. PGY-4 residents demonstrated the smallest change in mean score (3.3% increase between pretest and posttest), and PGY-5 residents also exhibited only a small improvement (5% increase between pretest and posttest) (Fig. 3). Participant evaluation of the workshop was assessed using a post-workshop survey. Survey questions were based on a 10-point Likert scale of positive statements, with 1 being Strongly agree and 10 being Strongly disagree. When asked whether the workshop was helpful, residents rated it on an average 4.7/10 (mode ¼ 3/10) but felt that 30 minutes per station was too long (mode ¼ 9/10, median ¼ 5.6). Participant evaluations of administrative staff were very positive (mode ¼ 2/10, median ¼ 3.1), with many of the open-ended comments emphasizing their appreciation of the staff’s time and effort.
RESULTS Of a total of 40 residents, 16 (40%) participated in the pilot study. None of the residents had prior formal SBP exposure. Of the 16 participants, 10 (62.5%) improved their score, whereas 5 (31.25%) showed no change and 1 (6.25%) resident demonstrated a decrease in score (Fig. 2). The pretest score was lowest in the group of residents at PGY-2 level (mean ¼ 70/100), whereas the highest pretest score was in the group of PGY-4 residents (mean ¼ 86.7/100). PGY-3 residents scored on an average 77.5/100, and PGY-5 residents scored on an average 85/100 on their pretest. All participants who had no change in their score answered the same questions incorrectly on the pretest and the posttest. Within training levels, all groups demonstrated an overall increase in mean test score across the pretest and the posttest. Residents at the PGY-2 level demonstrated the
DISCUSSIONS In 2001, the ACGME required residency training programs to implement 6 core competencies. SBP was found to be one of the more difficult competencies to successfully teach. Program directors have been provided references and tools to facilitate this, but a specific curriculum has not been delineated by the ACGME.3 Furthermore, residents who are already stretched for time and largely focused on acquiring medical knowledge and developing technical skills often deem other nontechnical areas less important.4 Compounding this problem is the fact that most graduating medical students have received minimal exposure to SBP principles. Wasnick et al.5 found that only 10% of fourthyear medical students were knowledgeable about SBP.
FIGURE 2. Pretest and posttest scores of each participant, grouped by PGY resident level. 288
Journal of Surgical Education Volume 72/Number 2 March/April 2015
FIGURE 3. Change in test score by participant, grouped by PGY resident level.
Residency programs from various fields have explored different methods to improve the resident’s understanding of SBP. Several published studies have designed interventions based on longitudinal schedules employing a combination of lectures, small-group workshops, and conferences.6–8 Several institutions incorporated their unique SBP intervention into the residents’ existing curriculum.3,4,9,10 Buchmann et al.3 required their radiology residents to design and complete a research project relating to systems errors, over a 1-year period. Oyler et al.9 encouraged understanding of SBP issue in internal medicine residents by integrating a longitudinal quality assessment and improvement curriculum into their 2 required 1-month ambulatory rotations. Three studies have evaluated discreet, less time-intensive, SBP experiences more similar to our pilot study. In these examples, primary care residents were introduced to frontoffice responsibilities.11-13 Unlike our pilot project, however, the goal of these previously published interventions was for residents to gain an appreciation for the training and responsibilities of the staff. These studies did not evaluate whether the SBP knowledge of the residents had improved as a result of the experience. With this pilot study, we sought to implement an intervention that was discreet and in situ, providing residents with a realistic sense of SBP. Furthermore, it piggybacked on existing teaching encounters as the intervention was carried out during the weekly morning dedicated to resident education. The teaching and subsequent assessment of SBP competency was performed during the intervention, an approach that is believed to improve the success of an SBP-related teaching session.14 We believe that implementing a short, well-defined, program such as this is sufficient to make a positive effect on residents’ knowledge as it relates to SBP, and is easier for residency
program directors to incorporate into their existing curriculum when compared with a more longitudinal approach. This was a pilot study designed to assess the feasibility of a larger SBP study targeting all surgical residents. The low number of participants reflects the original design of the pilot, which enrolled only residents available on a given morning of their weekly protected educational time. Given the successful results of this pilot study, a more comprehensive study designed to capture a larger proportion of surgical residents is warranted. Important information regarding the ideal target learner was also discovered. PGY-4 and -5 residents had better pretest scores than the more junior residents (PGY-2 and -3). This kind of workshop may, therefore, be more useful in the first few years of residency. Since, at the time of the study, there were no SBP-specific workshops or educational sessions within the resident curriculum, this finding may be a reflection of the fact that residents learn SBP inherently through their routine clinical training. Enrolling the same number of senior and junior residents would permit a better comparison of the effect of this type of training on SBP knowledge. It is also important to note that because the posttest was administered immediately after the intervention we cannot ascertain the effect of the workshop on long-term retention of SBP principles. Another potential limitation of this study relates to the inherent lack of standardization in the individual’s experience as they rotate through the various workshop stations. The study was designed to give residents a realistic understanding of SBP activities. What a given participating resident experienced was, therefore, dependent upon what occurred during their 30-minute rotation. For example, the workshop did not try to standardize which patients came in to the clinic or called the appointment line during a given 30-minute period. This potential limitation,
Journal of Surgical Education Volume 72/Number 2 March/April 2015
289
however, was considered less significant than the perceived benefit of implementing an immersion workshop in situ in an outpatient clinic. Moreover, the lack of standardization was minimized by implementing this workshop over the course of one morning, with the same staff explaining SBP concepts to all participating residents. In a future study, it may be beneficial to consider a short didactic session in addition to the in situ workshop to ascertain that all residents are at least exposed to the same fundamental SBP concepts. Conversely, the administrative staff could be instructed to ensure the key concepts have been touched upon during the 30-minute rotation, to help standardize the residents’ experience. Based on the results of our pilot study, we propose a larger SBP study where PGY-1 to -3 residents rotate through a similar workshop (perhaps without the patient check-in station that seemed redundant) and are administered the pretest and posttest. A debriefing meeting to go over their experience in the workshop and their posttest answers could help ensure that the SBP concepts have been truly assimilated and discussed. Subsequently, we would administer a third test 6 months after the workshop, to help define the long-term retention effect of this discreet training event. A second workshop could be administered to the same residents in their PGY-4 or -5, which would pertain to more sophisticated SBP concepts, as SBP competency encompasses much more than the 3 notions that we tested. An effort to schedule the workshop on a day when the maximal numbers of residents are working is critical to improve the 40% participation rate in our pilot study. To our knowledge, this is the first published report of a discreet immersion workshop in SBP principles aimed at general surgery residents. We were able to demonstrate that an immersion experience exposing residents to in situ SBP concepts is an effective and practical method of integrating this core competency into the residency curriculum. This workshop could complement more formal didactic teaching and easily be incorporated into the existing educational format.
systems-based practice project requirement for radiology residents. Acad Radiol. 2008;15(8):1040-1045. doi:http://dx.doi.org/10.1016/j.acra.2008.02.016. 4. David RA, Reich LM. The creation and evaluation of a
systems-based practice/managed care curriculum in a primary care internal medicine residency program. Mt Sinai J Med. 2005;72(5):296-299. 5. Wasnick JD, Chang L, Russell C, Gadsden J. Do
residency applicants know what the ACGME core competencies are? one program’s experience Acad Med. 2010;85(5):791-793. 6. Doezema D, McLaughlin S, Sklar DP. An approach to
fulfilling the systems‐based practice competency requirement. Acad Emerg Med. 2002;9(11):1355-1359. [Accessed 03.04.14, 6:07:05 PM]. 7. Panek RC, Deloney LA, Park J, Goodwin W, Klein S,
Ferris EJ. Interdepartmental problem-solving as a method for teaching and learning systems-based practice. Acad Radiol. 2006;13(9):1150-1154. doi:http: //dx.doi.org/10.1016/j.acra.2006.06.003 [Accessed 03.04.14, 6:09:28 PM]. 8. Allen E, Zerzan J, Choo C, Shenson D, Saha S.
Teaching systems-based practice to residents by using independent study projects. Acad Med. 2005;80 (2):125-128. [Accessed 03.04.14, 6:20:15 PM]. 9. Oyler J, Vinci L, Arora V, Johnson J. Teaching
internal medicine residents quality improvement techniques using the ABIM’s practice improvement modules. J Gen Intern Med. 2008;23(7):927-930. doi:http: //dx.doi.org/10.1007/s11606-008-0549-5. 10. Ziegelstein RC, Fiebach NH. “The mirror” and “the
village”: a new method for teaching practice-based learning and improvement and systems-based practice. Acad Med. 2004;79(1):83-88. 11. Kellerman R. Practice management training in family
practice residencies. J Fam Pract. 1983;16(1):173-176. 12. Rose EA, Neale AV, Rathur WA. Teaching practice
management during residency. Fam Med. 1999;31 (2):107-113.
REFERENCES 1. Accreditation Council for Graduate Medical Education.
ACGME outcome project. Available at: 〈http://cor es33webs.mede.uic.edu/gmenext/ui/portal/external/gc_ about.htm〉; Accessed 04.03.14.
2. Heard JK, Allen RM, Clardy J. Assessing the needs of
residency program directors to meet the ACGME general competencies. Acad Med. 2002;77(7):750. 3. Buchmann RF, Deloney LA, Donepudi SK, Mitchell
CM, Klein SG. Development and implementation of a 290
13. Sutkin G, Aronoff CK. Resident front office experi-
ence: a systems-based practice activity. Med Educ Online. 2008;13:6. doi:http://dx.doi.org/10.3885/ meo.2008.T0000120. 14. Lee AG, Beaver HA, Greenlee E, et al. Teaching and
assessing systems-based competency in ophthalmology residency training programs. Surv Ophthalmol. 2007;52(6):680-689, doi:http://dx.doi.org/10.1016/j. survophthal.2007.08.021.
Journal of Surgical Education Volume 72/Number 2 March/April 2015
Putting Residents in the Office: An Effective Method to Teach the Systems-Based Practice Competency Marisa Pulcrano, BA,* A. Alfred Chahine, MD,*,† Amanda Saratsis, MD,* Jamie Divine-Cadavid, MS,* Vinod Narra, MD,* and Stephen R.T. Evans, MD* *
Department of Surgery, MedStar Georgetown University Hospital, Washington, District of Columbia; and Department of Surgery, Children’s National Health System, Washington, District of Columbia
†
OBJECTIVES: Systems-based practice (SBP) was 1 of 6 core
competencies established by the Accreditation Council for Graduate Medical Education and has proven to be one of the most difficult to effectively implement. This pilot study presents an immersion workshop as an effective tool to teach the SBP competency in a way that could easily be integrated into a residency curriculum. DESIGN: In 2006, 16 surgical residents rotated through 3
stations for 30 minutes each: coding and billing, scheduling operations and return appointments, and patient check-in. Participants were administered a pretest and posttest questionnaire evaluating their knowledge of SBP, and were asked to evaluate the workshop. SETTING: Outpatient clinic at MedStar Georgetown Uni-
versity Hospital, Washington, DC. PARTICIPANTS: Residents in the general surgery residency
training program at MedStar Georgetown University Hospital. RESULTS: Most residents (62.5%) improved their score after the workshop, whereas 31.25% showed no change and 6.25% demonstrated a decrease in score. Overall within their training levels, all groups demonstrated an increase in mean test score. Postgraduate year-2 residents demonstrated the greatest change in mean score (20%), whereas postgraduate year-4 residents demonstrated the smallest change in mean score (3.3%). CONCLUSIONS: An immersion workshop where general
surgery residents gained direct exposure to SBP concepts in situ was an effective and practical method of integrating this core competency into the residency curriculum. Such a workshop could complement more formal didactic teaching
Poster presentation at Surgical Education Week, The Association of Surgical Education, Washington, DC, April 10-14, 2007. Correspondence: Inquiries to A. Alfred Chahine, MD, Division of Pediatric Surgery, MedStar Georgetown University Hospital, 3800 Reservoir Road, NW # 4PHC, Washington, DC 20007; e-mail: [email protected]
286
and be easily incorporated into the curriculum. For example, this workshop could be integrated into the ambulatory care requirement that each resident must fulfill as part of C 2014 Associtheir clinical training. ( J Surg 72:286-290. J ation of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: systems-based practice, general surgery
residency, graduate medical education, competencies COMPETENCIES: Professionalism, Practice-Based Learning and Improvement, Systems-Based Practice
INTRODUCTION Training in systems-based practice (SBP) is 1 of the 6 core competencies mandated by the Accreditation Council for Graduate Medical Education (ACGME) of all participants of residency programs. The ACGME defines SBP as “awareness of and responsiveness to the larger context and system of health care, and the ability to effectively call on system resources to provide care that is of optimal value.”1 Competency in SBP is intended to enhance a physician’s effectiveness in the context of the modern health care system, providing knowledge of health care resources, delivery systems, advocacy, and access to care. Yet, of the 6 competencies, SBP was found to be the competency that residency program directors felt they needed the most assistance with when educating their residents.2 Program directors across specialties have integrated various creative methods in an effort to introduce these concepts, including interdepartmental conferences, small-group workshops, and role-play, research projects and presentations, and standardized patient interactions in clinical settings. Owing to its broad and varied nature, however, systems-based content has been difficult to incorporate into the graduate medical education curriculum. Measuring learning outcomes and
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.09.001
individual competency in SBP has proven equally challenging. A scientifically sound and reproducible method of integrating systems-based training has yet to be identified. Furthermore, we are not aware of any publications describing a surgical department’s successful implementation of an educational program designed to address their residents’ understanding of SBP. To explore methods by which residents in an academic surgical residency program may gain knowledge of and facility with SBP concepts, a workshop was designed whereby residents observed administrative staff in an outpatient clinic at MedStar Georgetown University Hospital. This pilot study presents our successful experience using an immersion workshop as an effective tool to teach the SBP competency to surgical residents, and our conclusion that such a tool can be easily integrated into the existing curriculum.
METHOD Residents in the General Surgery Residency Training Program at MedStar Georgetown University Hospital participated in this study in October 2006. Training levels ranged from postgraduate year (PGY)-2 to PGY-5. Participants were administered a questionnaire with 10 items at the outset of the workshop as a pretest (Fig. 1). The questions assessed the resident’s familiarity with common SBP situations that may arise in a surgical practice. The questions were multiple choice or true/false, apart from 2 open-ended questions. Following the pretest, residents participated in an immersion workshop that took place in the front and back offices of the outpatient clinic in the Department of Surgery. The workshop required residents to rotate through 3 stations for 30 minutes each: coding and billing, scheduling operations and
FIGURE 1. Questionnaire administered pretest and posttest to evaluate surgical residents’ knowledge of systems-based competency. Journal of Surgical Education Volume 72/Number 2 March/April 2015
287
return appointments, and patient check-in. We briefed the administrative staff before the workshop as to the expectations of the exercise, and residents were encouraged to ask questions to the staff. The entire workshop lasted 2 hours and took place during the residency training program’s protected time for medical education, thus all residents working at that time were invited to participate in this pilot study. At the conclusion of the workshop, participants completed the same questionnaire to demonstrate knowledge acquired through the activity. Raw pretest and posttest scores were compared for each individual, and mean scores were compared across training levels to determine the effectiveness of the workshop in increasing facility with SBP concepts. Each participant completed a short 10-point Likert scale evaluation of the workshop after the exercise was complete. The median and modes of the evaluation responses were then calculated.
greatest change in mean score (20% increase between pretest and posttest), whereas residents at the PGY-3 level demonstrated a 10% increase in scores between the pretest and the posttest. PGY-4 residents demonstrated the smallest change in mean score (3.3% increase between pretest and posttest), and PGY-5 residents also exhibited only a small improvement (5% increase between pretest and posttest) (Fig. 3). Participant evaluation of the workshop was assessed using a post-workshop survey. Survey questions were based on a 10-point Likert scale of positive statements, with 1 being Strongly agree and 10 being Strongly disagree. When asked whether the workshop was helpful, residents rated it on an average 4.7/10 (mode ¼ 3/10) but felt that 30 minutes per station was too long (mode ¼ 9/10, median ¼ 5.6). Participant evaluations of administrative staff were very positive (mode ¼ 2/10, median ¼ 3.1), with many of the open-ended comments emphasizing their appreciation of the staff’s time and effort.
RESULTS Of a total of 40 residents, 16 (40%) participated in the pilot study. None of the residents had prior formal SBP exposure. Of the 16 participants, 10 (62.5%) improved their score, whereas 5 (31.25%) showed no change and 1 (6.25%) resident demonstrated a decrease in score (Fig. 2). The pretest score was lowest in the group of residents at PGY-2 level (mean ¼ 70/100), whereas the highest pretest score was in the group of PGY-4 residents (mean ¼ 86.7/100). PGY-3 residents scored on an average 77.5/100, and PGY-5 residents scored on an average 85/100 on their pretest. All participants who had no change in their score answered the same questions incorrectly on the pretest and the posttest. Within training levels, all groups demonstrated an overall increase in mean test score across the pretest and the posttest. Residents at the PGY-2 level demonstrated the
DISCUSSIONS In 2001, the ACGME required residency training programs to implement 6 core competencies. SBP was found to be one of the more difficult competencies to successfully teach. Program directors have been provided references and tools to facilitate this, but a specific curriculum has not been delineated by the ACGME.3 Furthermore, residents who are already stretched for time and largely focused on acquiring medical knowledge and developing technical skills often deem other nontechnical areas less important.4 Compounding this problem is the fact that most graduating medical students have received minimal exposure to SBP principles. Wasnick et al.5 found that only 10% of fourthyear medical students were knowledgeable about SBP.
FIGURE 2. Pretest and posttest scores of each participant, grouped by PGY resident level. 288
Journal of Surgical Education Volume 72/Number 2 March/April 2015
FIGURE 3. Change in test score by participant, grouped by PGY resident level.
Residency programs from various fields have explored different methods to improve the resident’s understanding of SBP. Several published studies have designed interventions based on longitudinal schedules employing a combination of lectures, small-group workshops, and conferences.6–8 Several institutions incorporated their unique SBP intervention into the residents’ existing curriculum.3,4,9,10 Buchmann et al.3 required their radiology residents to design and complete a research project relating to systems errors, over a 1-year period. Oyler et al.9 encouraged understanding of SBP issue in internal medicine residents by integrating a longitudinal quality assessment and improvement curriculum into their 2 required 1-month ambulatory rotations. Three studies have evaluated discreet, less time-intensive, SBP experiences more similar to our pilot study. In these examples, primary care residents were introduced to frontoffice responsibilities.11-13 Unlike our pilot project, however, the goal of these previously published interventions was for residents to gain an appreciation for the training and responsibilities of the staff. These studies did not evaluate whether the SBP knowledge of the residents had improved as a result of the experience. With this pilot study, we sought to implement an intervention that was discreet and in situ, providing residents with a realistic sense of SBP. Furthermore, it piggybacked on existing teaching encounters as the intervention was carried out during the weekly morning dedicated to resident education. The teaching and subsequent assessment of SBP competency was performed during the intervention, an approach that is believed to improve the success of an SBP-related teaching session.14 We believe that implementing a short, well-defined, program such as this is sufficient to make a positive effect on residents’ knowledge as it relates to SBP, and is easier for residency
program directors to incorporate into their existing curriculum when compared with a more longitudinal approach. This was a pilot study designed to assess the feasibility of a larger SBP study targeting all surgical residents. The low number of participants reflects the original design of the pilot, which enrolled only residents available on a given morning of their weekly protected educational time. Given the successful results of this pilot study, a more comprehensive study designed to capture a larger proportion of surgical residents is warranted. Important information regarding the ideal target learner was also discovered. PGY-4 and -5 residents had better pretest scores than the more junior residents (PGY-2 and -3). This kind of workshop may, therefore, be more useful in the first few years of residency. Since, at the time of the study, there were no SBP-specific workshops or educational sessions within the resident curriculum, this finding may be a reflection of the fact that residents learn SBP inherently through their routine clinical training. Enrolling the same number of senior and junior residents would permit a better comparison of the effect of this type of training on SBP knowledge. It is also important to note that because the posttest was administered immediately after the intervention we cannot ascertain the effect of the workshop on long-term retention of SBP principles. Another potential limitation of this study relates to the inherent lack of standardization in the individual’s experience as they rotate through the various workshop stations. The study was designed to give residents a realistic understanding of SBP activities. What a given participating resident experienced was, therefore, dependent upon what occurred during their 30-minute rotation. For example, the workshop did not try to standardize which patients came in to the clinic or called the appointment line during a given 30-minute period. This potential limitation,
Journal of Surgical Education Volume 72/Number 2 March/April 2015
289
however, was considered less significant than the perceived benefit of implementing an immersion workshop in situ in an outpatient clinic. Moreover, the lack of standardization was minimized by implementing this workshop over the course of one morning, with the same staff explaining SBP concepts to all participating residents. In a future study, it may be beneficial to consider a short didactic session in addition to the in situ workshop to ascertain that all residents are at least exposed to the same fundamental SBP concepts. Conversely, the administrative staff could be instructed to ensure the key concepts have been touched upon during the 30-minute rotation, to help standardize the residents’ experience. Based on the results of our pilot study, we propose a larger SBP study where PGY-1 to -3 residents rotate through a similar workshop (perhaps without the patient check-in station that seemed redundant) and are administered the pretest and posttest. A debriefing meeting to go over their experience in the workshop and their posttest answers could help ensure that the SBP concepts have been truly assimilated and discussed. Subsequently, we would administer a third test 6 months after the workshop, to help define the long-term retention effect of this discreet training event. A second workshop could be administered to the same residents in their PGY-4 or -5, which would pertain to more sophisticated SBP concepts, as SBP competency encompasses much more than the 3 notions that we tested. An effort to schedule the workshop on a day when the maximal numbers of residents are working is critical to improve the 40% participation rate in our pilot study. To our knowledge, this is the first published report of a discreet immersion workshop in SBP principles aimed at general surgery residents. We were able to demonstrate that an immersion experience exposing residents to in situ SBP concepts is an effective and practical method of integrating this core competency into the residency curriculum. This workshop could complement more formal didactic teaching and easily be incorporated into the existing educational format.
systems-based practice project requirement for radiology residents. Acad Radiol. 2008;15(8):1040-1045. doi:http://dx.doi.org/10.1016/j.acra.2008.02.016. 4. David RA, Reich LM. The creation and evaluation of a
systems-based practice/managed care curriculum in a primary care internal medicine residency program. Mt Sinai J Med. 2005;72(5):296-299. 5. Wasnick JD, Chang L, Russell C, Gadsden J. Do
residency applicants know what the ACGME core competencies are? one program’s experience Acad Med. 2010;85(5):791-793. 6. Doezema D, McLaughlin S, Sklar DP. An approach to
fulfilling the systems‐based practice competency requirement. Acad Emerg Med. 2002;9(11):1355-1359. [Accessed 03.04.14, 6:07:05 PM]. 7. Panek RC, Deloney LA, Park J, Goodwin W, Klein S,
Ferris EJ. Interdepartmental problem-solving as a method for teaching and learning systems-based practice. Acad Radiol. 2006;13(9):1150-1154. doi:http: //dx.doi.org/10.1016/j.acra.2006.06.003 [Accessed 03.04.14, 6:09:28 PM]. 8. Allen E, Zerzan J, Choo C, Shenson D, Saha S.
Teaching systems-based practice to residents by using independent study projects. Acad Med. 2005;80 (2):125-128. [Accessed 03.04.14, 6:20:15 PM]. 9. Oyler J, Vinci L, Arora V, Johnson J. Teaching
internal medicine residents quality improvement techniques using the ABIM’s practice improvement modules. J Gen Intern Med. 2008;23(7):927-930. doi:http: //dx.doi.org/10.1007/s11606-008-0549-5. 10. Ziegelstein RC, Fiebach NH. “The mirror” and “the
village”: a new method for teaching practice-based learning and improvement and systems-based practice. Acad Med. 2004;79(1):83-88. 11. Kellerman R. Practice management training in family
practice residencies. J Fam Pract. 1983;16(1):173-176. 12. Rose EA, Neale AV, Rathur WA. Teaching practice
management during residency. Fam Med. 1999;31 (2):107-113.
REFERENCES 1. Accreditation Council for Graduate Medical Education.
ACGME outcome project. Available at: 〈http://cor es33webs.mede.uic.edu/gmenext/ui/portal/external/gc_ about.htm〉; Accessed 04.03.14.
2. Heard JK, Allen RM, Clardy J. Assessing the needs of
residency program directors to meet the ACGME general competencies. Acad Med. 2002;77(7):750. 3. Buchmann RF, Deloney LA, Donepudi SK, Mitchell
CM, Klein SG. Development and implementation of a 290
13. Sutkin G, Aronoff CK. Resident front office experi-
ence: a systems-based practice activity. Med Educ Online. 2008;13:6. doi:http://dx.doi.org/10.3885/ meo.2008.T0000120. 14. Lee AG, Beaver HA, Greenlee E, et al. Teaching and
assessing systems-based competency in ophthalmology residency training programs. Surv Ophthalmol. 2007;52(6):680-689, doi:http://dx.doi.org/10.1016/j. survophthal.2007.08.021.
Journal of Surgical Education Volume 72/Number 2 March/April 2015
