Surg Endosc DOI 10.1007/s00464-013-3241-8
and Other Interventional Techniques
Simulation-based training improves the operative performance of totally extraperitoneal (TEP) laparoscopic inguinal hernia repair: a prospective randomized controlled trial Yo Kurashima • Liane S. Feldman • Pepa A. Kaneva • Gerald M. Fried • Simon Bergman • Sebastian V. Demyttenaere Chao Li • Melina C. Vassiliou
•
Received: 15 July 2013 / Accepted: 26 September 2013 Ó Springer Science+Business Media New York 2013
Abstract Background Laparoscopic surgery has an important role to play in the care of patients with inguinal hernias, but the procedure is difficult to learn. This study aimed to assess whether training to proficiency using a novel laparoscopic inguinal hernia repair (LIHR) simulation curriculum improved operating room (OR) performance. Methods For this study, 17 surgical residents [postgraduate years (PGYs) 2–5] participated in a didactic LIHR course and then were randomized to a training (T) or a control (C, standard residency) group. Performance of totally extraperitoneal (TEP) LIHR in the OR at baseline and after the study was measured using the Global Operative Assessment of Laparoscopic Skills–Groin Hernia (GOALS-GH). Results Of the 17 residents, 14 (5 T and 9 C) completed their final evaluations. The two groups showed no differences in terms of LIHR experience. The baseline GOALSGH scores in the OR were similar (T 14.8; range 12.8–16.8 Presented at the 2013 SAGES 2013 Annual meeting, April 17–20, 2013, Baltimore, MD, USA. Y. Kurashima L. S. Feldman P. A. Kaneva G. M. Fried C. Li M. C. Vassiliou Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University, Montreal, QC, Canada e-mail: [email protected] Y. Kurashima L. S. Feldman P. A. Kaneva G. M. Fried C. Li M. C. Vassiliou Arnold and Blema Steinberg Medical Simulation Centre, Montreal, QC, Canada
vs. C 13.6; range 12.3–14.8; P = 0.20). The mean number of training sessions needed to achieve proficiency was 4.8 (range 4.4–5.2), and the mean total training time was 109 min (range 61.9–149.1 min). After training, OR performance improved in the T group by 3.4 points (range 2.0–4.8 points; P = 0.002), whereas no significant change was seen in the C group [1.2; (range -1.1 to 3.6; P = 0.27)]. The final total GOALS-GH scores showed a trend toward better performance in the T group than in the C group [18.2; (range 14.9–21.5) vs. 14.8; (range 12.4–17.1); P = 0.06). Conclusions This study demonstrated the skills required for transfer of LIHR to the OR using a low-cost procedurespecific simulator. Residents who trained to proficiency on the McGill Laparoscopic Inguinal Hernia Simulator (MLIHS) showed greater skill improvement than their colleagues who did not. These results provide evidence supporting the use of simulation to teach and assess LIHR. Keywords Inguinal hernia Simulation training Laparoscopy
S. Bergman Department of Surgery, Jewish General Hospital, McGill University, Montreal, QC, Canada S. V. Demyttenaere Department of Surgery, Saint Mary’s Hospital, McGill University, Montreal, QC, Canada
Y. Kurashima (&) Department of Gastroenterological Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan e-mail: [email protected]
123
Surg Endosc
Inguinal hernia repair is one of the most common procedures performed by general surgeons, and minimally invasive techniques are becoming increasingly important in the management of hernia patients [1, 2]. The procedures, however, are challenging to teach and learn, and reports describe increased complications and recurrences during the learning curve, which is reported to require as many as 250 procedures [3]. When performed by experienced surgeons, the laparoscopic approach is beneficial for the repair of bilateral or recurrent inguinal hernias, and some studies have shown that laparoscopic inguinal hernia repair (LIHR) is associated with less postoperative pain and earlier return to normal activities than the open technique [3, 4]. Despite compelling data, only 14–17.5 % of inguinal hernia repairs are performed laparoscopically [5, 6]. Furthermore, a gap still exists in the training of residents to perform LIHR. In a recent survey, 79.5 % of Canadian residents logged fewer than five LIHR cases as primary operators during their entire residency program [7]. Clear evidence supports the role of simulation in the acquisition of fundamental surgical skills, particularly laparoscopic skills [8, 9]. The use of simulation to teach procedures, however, is less well established. Surgical procedures that require complex integration of different types of knowledge and skill by the surgeon while interacting with the environment can be challenging to simulate effectively. Previous studies have demonstrated improved performance of totally extraperitoneal (TEP) hernia repair in the OR after training with different TEP simulators, and one study also demonstrated improvements in patient outcomes [10, 11]. However, the simulators used in these studies focused only on parts of the procedure such as mesh placement and sac reduction. Moreover, they used intraoperative measures of TEP performance not specific to TEP and not validated specifically for that purpose. We developed and established validity evidence for a simple, low-cost model, the McGill Laparoscopic Inguinal Hernia Simulator (MLIHS), which allows learners to practice the entire TEP procedure [12]. In addition, we created and also obtained validity evidence for a procedure-specific assessment tool that can be used in the simulator and in the operating room (OR): the Global Operative Assessment of Laparoscopic Skills–Groin Hernia (GOALS-GH) [13]. Both of these tools were used to design a proficiency curriculum for LIHR. This randomized controlled trial aimed to compare the performance of TEP in the OR after a proficiency-based LIHR simulation curriculum with that of standard clinical training.
A07-M71-10B). General surgery residents in postgraduate years (PGYs) 2–5 at McGill University participated in the study from September 2010 to May 2012. The residents all participated in a didactic teaching session about inguinal hernia repair and anatomy during their usual protected teaching time. They then were assessed at baseline in the OR during a TEP repair using the GOALS-GH. Incarcerated or recurrent inguinal hernias were excluded from this study. After baseline testing, the residents were randomly assigned to the simulation training or control group. The MLIHS training was proctored by a surgeon experienced in minimally invasive surgery (MIS). Proficiency metrics for the simulator were based on expert performance established in a previous study [12]. Both groups continued their regular residency training, and the participants were asked to record their clinical laparoscopic experience including all inguinal hernia repairs throughout the study period. Both groups were tested again in the OR during a TEP hernia repair within 3 months after baseline testing, and the training group was tested additionally within 2 weeks after achievement of proficiency. The evaluators were blinded to the training status of the participants. The flow of the participants in the study is depicted in Fig. 1. Randomization Randomization was performed by having the participants draw assignments from a box with a 50 % chance of allotment to the T or C group. The participants were asked to keep their randomization status confidential for the study period. The investigator who performed the randomization was not involved in the evaluation of the TEP performances in the OR. The MLIHS The TEP training for this study was done using the MLIHS, described in detail elsewhere [12]. The MLIHS is a physical simulator that models the inguinal anatomy. Created from low-cost materials, the MLIHS can be used to practice both transabdominal preperitoneal (TAPP) and TEP hernia repairs. The MLIHS uses the same laparoscopic tower and instruments used in the OR. Learners can practice each step of the operative procedure including trocar insertion and mesh placement. Assessment of TEP performance
Methods The institutional review board (IRB) of the Faculty of Medicine at McGill University approved this study (IRB:
123
Intraoperative TEP performance was evaluated using the GOALS-GH, which evaluates the operative judgment and technical skills specific to LIHR. Validity evidence for use
Surg Endosc Fig. 1 Flow of study participants
of the GOALS-GH as a measure of surgical skills in the OR and the simulator has been published [13]. The scale can be used to assess the performance of both the TAPP and TEP procedures using the following five items: (1) trocar location and placement, (2) creation of a peritoneal flap (TAPP) or creation of preperitoneal space (TEP), (3) hernia sac identification and reduction, (4) mesh placement and fixation, and (5) knowledge of anatomy and flow of procedure. Each domain is scored on a Likert scale from 1 to 5, with anchoring descriptors at 1, 3, and 5. The highest score is 25. All the participants were evaluated in the OR by the attending surgeon blinded to their randomization status. Some residents were evaluated by the same surgeons at both baseline and final assessment, whereas others were evaluated by different surgeons. The surgeon evaluators were trained to use the GOALS-GH. Simulation-based training The participants randomized to the simulation-training group practiced bilateral TEP procedures using the MLIHS. Experienced MIS surgeons (fellows and researchers) assisted the residents by holding the laparoscope and providing instruction or feedback at the resident’s request. The proficiency goal was to achieve a GOALS-GH score of 24 within 15 min. The training schedule was determined by the learners, who contacted the proctors independently. The learners were asked to repeat practice sessions until they achieved proficiency as assessed by the proctor. The number and duration of the training sessions were recorded for each participant. The
interval between the last MLIHS practice and the final evaluation in the OR was set at a maximum of 15 days. At the end of the training, the learners were asked about their impressions of the TEP simulator and its value as a training tool. Statistical analysis The primary outcome of the study was the difference between baseline and final performance in the OR measured by the GOALS-GH. A power analysis was conducted before the study based on pilot and previous data. The mean total GOALS-GH score in the OR for novices was 13.8 ± 3. A posttraining increase in performance to a GOALS-GH score of 20 ± 3 was considered significant compared with an estimated score of 15 for the control group. To demonstrate significant differences between the two groups with an alpha of 0.05 and a beta error of 0.20, the analysis indicated a need for a minimum of seven participants in each group. The GOALS-GH scores were compared between the T and C groups using Student’s t test. The results are reported as means and 95 % confidence intervals (CI) or as medians and interquartile ranges (IQR). A P value lower than 0.05 was considered significant. All analyses were performed using SPSS software (Graduate Pack 11.0) (SPSS, Chicago, IL, USA) in consultation with a statistician.
Results Of the 17 randomized participants, 14 (5 T and 9 C) completed their final evaluations. The characteristics of the participants
123
Surg Endosc Table 1 Comparison of simulator-trained and control groups at enrollment and during the study period Group
PGY (2/3/4/5)
Control group (n = 9)
Training group (n = 5)
P value
5/1/2/1
0/5/0/0
0.52
Table 2 Change is GOALS-GH itemized scores between baseline and final assessment Control group
P value
(n = 9)
Training group (n = 5)
1. Trocar placement
-0.1 (-0.7 to 0.5)
0.4 (-0.1 to 0.9)
0.14
2. Creation of preperitoneal space
0.4 (-0.5 to 1.3)
0.8 (0.3–1.3)
0.05
3. Hernia sac reduction
0.1 (-1.0 to 1.2)
1.2 (0.4–2.0)
0.06
Gender (male/female)
3/2
7/2
0.55
Dominant hand (right/left)
9/0
5/0
1
LIHR experience before baseline evaluation
1 (1–5)
1 (0–3)
0.84
LIHR performed during study
1 (1–5)
1 (1–2)
0.50
Other laparoscopic experience before baseline evaluation Other laparoscopic cases performed during study
39 (29–45) 13 (7–31)
23 (17–27) 4 (0–12)
0.045
4. Mesh placement and fixation
0.2 (-0.6 to 1.0)
0.6 (0–1.2)
0.32
0.083
5. Knowledge of anatomy and flow
0.6 (-0.5 to 1.7)
0.4 (-0.2 to 1.0)
0.74
Total score
1.2 (-1.1 to 3.6)
3.4 (2.0–4.8)
0.08
Median case number (interquartile range)
Simulation-based training costs are provided in Table 1. The C group had slightly more overall laparoscopic experience at baseline. The two groups did not differ in terms of baseline GOALS-GH scores (T 14.8; range 12.8–16.8 vs. C 13.6; range 12.3–14.8; P = 0.20). The baseline GOALS-GH scores for the two residents that dropped out of the study were both 14, similar to the scores obtained by those who completed the study. After training, OR performance improved in the T group by 3.4 points (range 2.0–4.8 points; P = 0.03), whereas no significant change was seen in the C group (change of 1.2 points; range -1.1 to 3.6 points; P = 0.27). The final GOALS-GH scores were higher in the T group (18.2; range 14.9–21.5) than in the C group (14.8; range 12.4–17.1), but this difference did not achieve statistical significance (P = 0.06). The change in the score for each of the GOALS items is provided in Table 2. Survey of the training group The T group learners evaluated the simulation training for each step according to the GOALS-GH items on a scale of 1–5 (5 denoting the highest value) as follows: (step 1) trocar placement (4.8 ± 0.45), (step 2) creation of preperitoneal space (4 ± 0.71), (step 3) hernia sac reduction (3 ± 1.0), (step 4) mesh placement and fixation (4 ± 1.0), and (step 5) knowledge of anatomy and flow of procedure (4.4 ± 0.55). Training time and end points The mean number of simulation training sessions required to achieve proficiency was 4.8 (range 4.4–5.2), and the mean total training time was 109 min (range 61.9–149.1 min).
123
The training of five learners required 24 sessions. We used two MLIHS inserts ($40 9 2) and 24 sets of skin and peritoneum materials ($2 9 24), for a total cost of $128 (Canadian dollars). This did not include the cost of the laparoscopic tower, instruments, trocars, tackers, and mesh, which were obtained either in kind from industry or as unused or expired materials from the OR.
Discussion We demonstrated significant improvements in TEP performance after proficiency-based simulation training using a procedure-specific scale (GOALS-GH) and a low-cost but anatomically accurate simulator (MLIHS). The improvement in the final scores achieved by the training group compared with the control group, however, did not achieve statistical significance. This may have been related to the small sample size. Our power calculation suggested a need for seven participants in each group, and despite adequate recruitment, only five participants were able to complete the study. The reasons for this were not related to poor performance because the residents who did not complete the study had baseline scores similar to those of their colleagues, and both achieved proficiency in the simulated environment. This was a logistical issue of being able to arrange an intraoperative final assessment within the limits of the study design given busy schedules and a limited number of procedures available. The residents rotated at several hospitals, and they did not have opportunities to participate in TEP procedures at all sites. Moreover, the simulator was available at only one site, thus
Surg Endosc
increasing the challenge for residents to practice. The MLIHS requires another person to hold the camera, which added another logistical hurdle. The use of a camera holder in the future may render the simulation more practical. The power analysis also was based on residents achieving a mean GOALS-GH score of 20 ± 3 in the OR, whereas they achieved a score of only 18.2 (range 14.9–21.5). The proficiency curriculum required residents in the training group to achieve a GOALS-GH score of 24, which was met by all of the residents. This highlights the gap that still remains between the simulation environment and the real clinical environment. The simulator focuses mostly on technical skills and steps of the procedure and helps residents to automate some of this basic knowledge and skill. The curriculum did not address much of the judgment and decision making related to LIHR, which may explain this gap. The concept of transferring part of the learning curve out of the OR, thus allowing learners to automate some of the technical, procedural, and organizational components of the procedure still is very appealing. The described model recognizes the limitations of working memory and cognitive capacity. If training is designed to help residents learn whatever can be learned outside the OR, the assumption is that they will be better prepared for their clinical experiences. If they have more working memory and attention available, they may be more able to prevent errors and learn judgment and decision making, which are challenging to simulate. Our study did not investigate patient outcomes, the ultimate barometer of simulation training effectiveness. In 2011, Zendejas et al. [11] examined not only the influence of TEP simulation training on performance but also the clinical impact on patients. In their large randomized trial, the training group participated in a mastery learning program consisting of both an online course and skills training using a commercially available TEP simulator (Guildford MATTU TEP task trainer; Limbs & Things, Ltd., Bristol, UK). In their study, PGY 1–5 residents practiced hernia sac reduction and mesh fixation until they could complete it within 2 min at a preestablished mastery level in the simulated environment. The training group demonstrated improvements compared with the control group in terms of shorter operative times (6 min faster), better TEP performance as measured by a laparoscopy-specific general global rating scale (GOALS), and fewer complications. Our study demonstrated the transfer of skills required for LIHR using a low-cost procedure-specific simulator of the OR. In 2001, Hamilton et al. [10] were among the first to show a link between simulation training and performance in the OR. After baseline assessments in the OR, they randomized surgical residents to a training program using a
locally created rubber TEP simulator, videos, and an interactive CD-ROM and compared them with a control group (standard residency training). The TEP simulation training was primarily focused on mesh fixation. The training group demonstrated greater improvements in performance of clinical TEP as measured using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale initially developed as a measure of open technical skills on a benchtop model [14]. Although the aforementioned study was very innovative and forward thinking, it is difficult to appreciate how much of the learning was related to the simulation because the training group also used videos and a CD-ROM. In addition, any focused educational intervention that encourages students to think about a procedure is likely to have some benefit over no training at all. We tried to control for this by ensuring that all the residents in our study participated in the teaching session on hernias, which included a review of a step-by-step TEP video. We did not formally assess this, but it is probable that the residents in the training group invested more time and focused more effort on learning TEP in preparation for the simulations and their OR assessments than did the control group. This may be an added bonus of simulation-based education, and in our opinion, it is a phenomenon likely to be observed outside the context of a study as well. This study adds to the evidence supporting the use of procedure-specific competency-based simulation programs in the surgical curriculum. Ongoing unanswered questions remain regarding how to integrate the programs, at what stage, and with what regularity. Problems of scheduling, of being able to provide useful and accurate feedback, and of motivating learners to engage with the process also are ubiquitous and very difficult to solve with limited resources. The culture is slowly changing, however, and as we accumulate increasing evidence, teachers, learners, and patients will recognize the value of these educational interventions and the role they can play. The MLIHS is low cost in terms of materials, at $128 for the entire study. This does not include, however, the laparoscopic tower, instruments, and tackers used, which may not be readily available to all training programs. Moreover, the value of having access to a trained proctor who can provide specific high-quality feedback is certainly the cornerstone of effective deliberate practice and probably is one of the biggest logistical hurdles [15]. The costs and feasibility of this also need to be taken into account when the use of a simulation-based curriculum for procedural skills is considered. Commitment of faculty to train proctors when appropriate or to train residents to be proctors in their more senior years may eventually result in a culture change that will have lasting, sustainable benefits, but this will require years of targeted and well-structured efforts.
123
Surg Endosc
In the meantime, simulation is certainly a useful adjunct to clinical training for TEP hernia repair, and our residents found it to be valuable. Interestingly, the residents found the simulator to be the least useful for hernia sac reduction, with a mean reported educational value of 3 compared with a value greater than 4 for all the other steps. The greatest increase in performance compared with baseline, however, was observed in hernia sac reduction (Table 2). This may be explained perhaps by the ‘‘they don’t know what they don’t know’’ phenomenon, referred to in the education and sociology literature as the Dunning-Kruger effect [16]. Alternatively, this may show the complexity of trying to simulate procedures versus using deconstructed tasks, which is a topic for future research in the area of procedural simulation. Skills acquired in the simulated environment using a low-cost procedure-specific simulator do transfer to the OR. Residents who trained to proficiency on the MLIHS showed greater skill improvement than their colleagues who did not. These results provide evidence supporting the use of simulation to teach and assess LIHR. Acknowledgments The authors thank the McGill general surgery residents for their participation in this study. We could not have completed the study without their diligence and support. Disclosures The salary of Yo Kurashima was funded by an unrestricted Covidien grant during the time this research was conducted, but this is not believed to have influenced the research in any way. Liane S. Feldman is a consultant for Covidien and had a centre of excellence Grant from Covidien as well as an investigator-initiated research grant from Ethicon. Pepa Kaneva had a Covidien unrestricted grant to fund research. Gerald M. Fried had an Covidien unrestricted grant to fund research, and his son works for CAE Healthcare. Melina C. Vassiliou is a Covidien consultant and had an unrestricted Grant to fund research. Simon Bergman, Sebastian V. Demyttenaere, and Chao Li have no conflicts of interest or financial ties to disclose.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
References 15. 1. Reznick RK, MacRae H (2006) Teaching surgical skills: changes in the wind. N Engl J Med 355:2664–2669 2. Simons MP, Aufenacker T, Bay-Nielsen M, Bouillot JL, Campanelli G, Conze J, de Lange D, Fortelny R, Heikkinen T, Kingsnorth A, Kukleta J, Morales-Conde S, Nordin P, Schumpelick V, Smedberg S, Smietanski M, Weber G, Miserez
123
16.
M (2009) European Hernia Society guidelines on the treatment of inguinal hernia in adult patients. Hernia 13:343–403. doi:10. 1007/s10029-009-0529-7 Neumayer L, Giobbie-Hurder A, Jonasson O, Fitzgibbons R, Dunlop D, Gibbs J, Reda D, Henderson W (2004) Open mesh versus laparoscopic mesh repair of inguinal hernia. N Engl J Med 350:1819–1827 The MRC Laparoscopic Groin Hernia Trial Group (1999) Laparoscopic versus open repair of groin hernia: a randomised comparison. Lancet 354:185–190 Rutkow IM (2003) Demographic and socioeconomic aspects of hernia repair in the United States in 2003. Surg Clin North Am 83:1045–1051 Jacob BP, Tong W, Reiner M, Vine A, Katz LB (2009) Singleincision total extraperitoneal (one SITE) laparoscopic inguinal hernia repair using a single-access port device. Hernia 13:571–572 Qureshi A, Vergis A, Jimenez C, Green J, Pryor A, Schlachta CM, Okrainec A (2011) MIS training in Canada: a national survey of general surgery residents. Surg Endosc 25:3057–3065 Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–528 Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM (2010) Fundamentals of Laparoscopic Surgery simulator training to proficiency improves laparoscopic performance in the operating room: a randomized controlled trial. Am J Surg 199:115–120 Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB (2001) Improving operative performance using a laparoscopic hernia simulator. Am J Surg 182:725–728 Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG, Farley DR (2011) Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg 254:502–511 Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A novel low-cost simulator for laparoscopic inguinal hernia repair. Surg Innov 18:171–175 Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg 201:54–61 Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective Structured Assessment of Technical Skill (OSATS) for surgical residents. Br J Surg 84:273–278 Ericsson KA (2008) Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 15:988–994. doi:10.1111/j.1553-2712.2008.00227.x Kruger J, Dunning D (1999) Unskilled and unaware of it: how difficulties of recognizing one’s own incompetence lead to inflated self-assessments. J Per Soc Psychol 77:1121–1134
and Other Interventional Techniques
Simulation-based training improves the operative performance of totally extraperitoneal (TEP) laparoscopic inguinal hernia repair: a prospective randomized controlled trial Yo Kurashima • Liane S. Feldman • Pepa A. Kaneva • Gerald M. Fried • Simon Bergman • Sebastian V. Demyttenaere Chao Li • Melina C. Vassiliou
•
Received: 15 July 2013 / Accepted: 26 September 2013 Ó Springer Science+Business Media New York 2013
Abstract Background Laparoscopic surgery has an important role to play in the care of patients with inguinal hernias, but the procedure is difficult to learn. This study aimed to assess whether training to proficiency using a novel laparoscopic inguinal hernia repair (LIHR) simulation curriculum improved operating room (OR) performance. Methods For this study, 17 surgical residents [postgraduate years (PGYs) 2–5] participated in a didactic LIHR course and then were randomized to a training (T) or a control (C, standard residency) group. Performance of totally extraperitoneal (TEP) LIHR in the OR at baseline and after the study was measured using the Global Operative Assessment of Laparoscopic Skills–Groin Hernia (GOALS-GH). Results Of the 17 residents, 14 (5 T and 9 C) completed their final evaluations. The two groups showed no differences in terms of LIHR experience. The baseline GOALSGH scores in the OR were similar (T 14.8; range 12.8–16.8 Presented at the 2013 SAGES 2013 Annual meeting, April 17–20, 2013, Baltimore, MD, USA. Y. Kurashima L. S. Feldman P. A. Kaneva G. M. Fried C. Li M. C. Vassiliou Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University, Montreal, QC, Canada e-mail: [email protected] Y. Kurashima L. S. Feldman P. A. Kaneva G. M. Fried C. Li M. C. Vassiliou Arnold and Blema Steinberg Medical Simulation Centre, Montreal, QC, Canada
vs. C 13.6; range 12.3–14.8; P = 0.20). The mean number of training sessions needed to achieve proficiency was 4.8 (range 4.4–5.2), and the mean total training time was 109 min (range 61.9–149.1 min). After training, OR performance improved in the T group by 3.4 points (range 2.0–4.8 points; P = 0.002), whereas no significant change was seen in the C group [1.2; (range -1.1 to 3.6; P = 0.27)]. The final total GOALS-GH scores showed a trend toward better performance in the T group than in the C group [18.2; (range 14.9–21.5) vs. 14.8; (range 12.4–17.1); P = 0.06). Conclusions This study demonstrated the skills required for transfer of LIHR to the OR using a low-cost procedurespecific simulator. Residents who trained to proficiency on the McGill Laparoscopic Inguinal Hernia Simulator (MLIHS) showed greater skill improvement than their colleagues who did not. These results provide evidence supporting the use of simulation to teach and assess LIHR. Keywords Inguinal hernia Simulation training Laparoscopy
S. Bergman Department of Surgery, Jewish General Hospital, McGill University, Montreal, QC, Canada S. V. Demyttenaere Department of Surgery, Saint Mary’s Hospital, McGill University, Montreal, QC, Canada
Y. Kurashima (&) Department of Gastroenterological Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan e-mail: [email protected]
123
Surg Endosc
Inguinal hernia repair is one of the most common procedures performed by general surgeons, and minimally invasive techniques are becoming increasingly important in the management of hernia patients [1, 2]. The procedures, however, are challenging to teach and learn, and reports describe increased complications and recurrences during the learning curve, which is reported to require as many as 250 procedures [3]. When performed by experienced surgeons, the laparoscopic approach is beneficial for the repair of bilateral or recurrent inguinal hernias, and some studies have shown that laparoscopic inguinal hernia repair (LIHR) is associated with less postoperative pain and earlier return to normal activities than the open technique [3, 4]. Despite compelling data, only 14–17.5 % of inguinal hernia repairs are performed laparoscopically [5, 6]. Furthermore, a gap still exists in the training of residents to perform LIHR. In a recent survey, 79.5 % of Canadian residents logged fewer than five LIHR cases as primary operators during their entire residency program [7]. Clear evidence supports the role of simulation in the acquisition of fundamental surgical skills, particularly laparoscopic skills [8, 9]. The use of simulation to teach procedures, however, is less well established. Surgical procedures that require complex integration of different types of knowledge and skill by the surgeon while interacting with the environment can be challenging to simulate effectively. Previous studies have demonstrated improved performance of totally extraperitoneal (TEP) hernia repair in the OR after training with different TEP simulators, and one study also demonstrated improvements in patient outcomes [10, 11]. However, the simulators used in these studies focused only on parts of the procedure such as mesh placement and sac reduction. Moreover, they used intraoperative measures of TEP performance not specific to TEP and not validated specifically for that purpose. We developed and established validity evidence for a simple, low-cost model, the McGill Laparoscopic Inguinal Hernia Simulator (MLIHS), which allows learners to practice the entire TEP procedure [12]. In addition, we created and also obtained validity evidence for a procedure-specific assessment tool that can be used in the simulator and in the operating room (OR): the Global Operative Assessment of Laparoscopic Skills–Groin Hernia (GOALS-GH) [13]. Both of these tools were used to design a proficiency curriculum for LIHR. This randomized controlled trial aimed to compare the performance of TEP in the OR after a proficiency-based LIHR simulation curriculum with that of standard clinical training.
A07-M71-10B). General surgery residents in postgraduate years (PGYs) 2–5 at McGill University participated in the study from September 2010 to May 2012. The residents all participated in a didactic teaching session about inguinal hernia repair and anatomy during their usual protected teaching time. They then were assessed at baseline in the OR during a TEP repair using the GOALS-GH. Incarcerated or recurrent inguinal hernias were excluded from this study. After baseline testing, the residents were randomly assigned to the simulation training or control group. The MLIHS training was proctored by a surgeon experienced in minimally invasive surgery (MIS). Proficiency metrics for the simulator were based on expert performance established in a previous study [12]. Both groups continued their regular residency training, and the participants were asked to record their clinical laparoscopic experience including all inguinal hernia repairs throughout the study period. Both groups were tested again in the OR during a TEP hernia repair within 3 months after baseline testing, and the training group was tested additionally within 2 weeks after achievement of proficiency. The evaluators were blinded to the training status of the participants. The flow of the participants in the study is depicted in Fig. 1. Randomization Randomization was performed by having the participants draw assignments from a box with a 50 % chance of allotment to the T or C group. The participants were asked to keep their randomization status confidential for the study period. The investigator who performed the randomization was not involved in the evaluation of the TEP performances in the OR. The MLIHS The TEP training for this study was done using the MLIHS, described in detail elsewhere [12]. The MLIHS is a physical simulator that models the inguinal anatomy. Created from low-cost materials, the MLIHS can be used to practice both transabdominal preperitoneal (TAPP) and TEP hernia repairs. The MLIHS uses the same laparoscopic tower and instruments used in the OR. Learners can practice each step of the operative procedure including trocar insertion and mesh placement. Assessment of TEP performance
Methods The institutional review board (IRB) of the Faculty of Medicine at McGill University approved this study (IRB:
123
Intraoperative TEP performance was evaluated using the GOALS-GH, which evaluates the operative judgment and technical skills specific to LIHR. Validity evidence for use
Surg Endosc Fig. 1 Flow of study participants
of the GOALS-GH as a measure of surgical skills in the OR and the simulator has been published [13]. The scale can be used to assess the performance of both the TAPP and TEP procedures using the following five items: (1) trocar location and placement, (2) creation of a peritoneal flap (TAPP) or creation of preperitoneal space (TEP), (3) hernia sac identification and reduction, (4) mesh placement and fixation, and (5) knowledge of anatomy and flow of procedure. Each domain is scored on a Likert scale from 1 to 5, with anchoring descriptors at 1, 3, and 5. The highest score is 25. All the participants were evaluated in the OR by the attending surgeon blinded to their randomization status. Some residents were evaluated by the same surgeons at both baseline and final assessment, whereas others were evaluated by different surgeons. The surgeon evaluators were trained to use the GOALS-GH. Simulation-based training The participants randomized to the simulation-training group practiced bilateral TEP procedures using the MLIHS. Experienced MIS surgeons (fellows and researchers) assisted the residents by holding the laparoscope and providing instruction or feedback at the resident’s request. The proficiency goal was to achieve a GOALS-GH score of 24 within 15 min. The training schedule was determined by the learners, who contacted the proctors independently. The learners were asked to repeat practice sessions until they achieved proficiency as assessed by the proctor. The number and duration of the training sessions were recorded for each participant. The
interval between the last MLIHS practice and the final evaluation in the OR was set at a maximum of 15 days. At the end of the training, the learners were asked about their impressions of the TEP simulator and its value as a training tool. Statistical analysis The primary outcome of the study was the difference between baseline and final performance in the OR measured by the GOALS-GH. A power analysis was conducted before the study based on pilot and previous data. The mean total GOALS-GH score in the OR for novices was 13.8 ± 3. A posttraining increase in performance to a GOALS-GH score of 20 ± 3 was considered significant compared with an estimated score of 15 for the control group. To demonstrate significant differences between the two groups with an alpha of 0.05 and a beta error of 0.20, the analysis indicated a need for a minimum of seven participants in each group. The GOALS-GH scores were compared between the T and C groups using Student’s t test. The results are reported as means and 95 % confidence intervals (CI) or as medians and interquartile ranges (IQR). A P value lower than 0.05 was considered significant. All analyses were performed using SPSS software (Graduate Pack 11.0) (SPSS, Chicago, IL, USA) in consultation with a statistician.
Results Of the 17 randomized participants, 14 (5 T and 9 C) completed their final evaluations. The characteristics of the participants
123
Surg Endosc Table 1 Comparison of simulator-trained and control groups at enrollment and during the study period Group
PGY (2/3/4/5)
Control group (n = 9)
Training group (n = 5)
P value
5/1/2/1
0/5/0/0
0.52
Table 2 Change is GOALS-GH itemized scores between baseline and final assessment Control group
P value
(n = 9)
Training group (n = 5)
1. Trocar placement
-0.1 (-0.7 to 0.5)
0.4 (-0.1 to 0.9)
0.14
2. Creation of preperitoneal space
0.4 (-0.5 to 1.3)
0.8 (0.3–1.3)
0.05
3. Hernia sac reduction
0.1 (-1.0 to 1.2)
1.2 (0.4–2.0)
0.06
Gender (male/female)
3/2
7/2
0.55
Dominant hand (right/left)
9/0
5/0
1
LIHR experience before baseline evaluation
1 (1–5)
1 (0–3)
0.84
LIHR performed during study
1 (1–5)
1 (1–2)
0.50
Other laparoscopic experience before baseline evaluation Other laparoscopic cases performed during study
39 (29–45) 13 (7–31)
23 (17–27) 4 (0–12)
0.045
4. Mesh placement and fixation
0.2 (-0.6 to 1.0)
0.6 (0–1.2)
0.32
0.083
5. Knowledge of anatomy and flow
0.6 (-0.5 to 1.7)
0.4 (-0.2 to 1.0)
0.74
Total score
1.2 (-1.1 to 3.6)
3.4 (2.0–4.8)
0.08
Median case number (interquartile range)
Simulation-based training costs are provided in Table 1. The C group had slightly more overall laparoscopic experience at baseline. The two groups did not differ in terms of baseline GOALS-GH scores (T 14.8; range 12.8–16.8 vs. C 13.6; range 12.3–14.8; P = 0.20). The baseline GOALS-GH scores for the two residents that dropped out of the study were both 14, similar to the scores obtained by those who completed the study. After training, OR performance improved in the T group by 3.4 points (range 2.0–4.8 points; P = 0.03), whereas no significant change was seen in the C group (change of 1.2 points; range -1.1 to 3.6 points; P = 0.27). The final GOALS-GH scores were higher in the T group (18.2; range 14.9–21.5) than in the C group (14.8; range 12.4–17.1), but this difference did not achieve statistical significance (P = 0.06). The change in the score for each of the GOALS items is provided in Table 2. Survey of the training group The T group learners evaluated the simulation training for each step according to the GOALS-GH items on a scale of 1–5 (5 denoting the highest value) as follows: (step 1) trocar placement (4.8 ± 0.45), (step 2) creation of preperitoneal space (4 ± 0.71), (step 3) hernia sac reduction (3 ± 1.0), (step 4) mesh placement and fixation (4 ± 1.0), and (step 5) knowledge of anatomy and flow of procedure (4.4 ± 0.55). Training time and end points The mean number of simulation training sessions required to achieve proficiency was 4.8 (range 4.4–5.2), and the mean total training time was 109 min (range 61.9–149.1 min).
123
The training of five learners required 24 sessions. We used two MLIHS inserts ($40 9 2) and 24 sets of skin and peritoneum materials ($2 9 24), for a total cost of $128 (Canadian dollars). This did not include the cost of the laparoscopic tower, instruments, trocars, tackers, and mesh, which were obtained either in kind from industry or as unused or expired materials from the OR.
Discussion We demonstrated significant improvements in TEP performance after proficiency-based simulation training using a procedure-specific scale (GOALS-GH) and a low-cost but anatomically accurate simulator (MLIHS). The improvement in the final scores achieved by the training group compared with the control group, however, did not achieve statistical significance. This may have been related to the small sample size. Our power calculation suggested a need for seven participants in each group, and despite adequate recruitment, only five participants were able to complete the study. The reasons for this were not related to poor performance because the residents who did not complete the study had baseline scores similar to those of their colleagues, and both achieved proficiency in the simulated environment. This was a logistical issue of being able to arrange an intraoperative final assessment within the limits of the study design given busy schedules and a limited number of procedures available. The residents rotated at several hospitals, and they did not have opportunities to participate in TEP procedures at all sites. Moreover, the simulator was available at only one site, thus
Surg Endosc
increasing the challenge for residents to practice. The MLIHS requires another person to hold the camera, which added another logistical hurdle. The use of a camera holder in the future may render the simulation more practical. The power analysis also was based on residents achieving a mean GOALS-GH score of 20 ± 3 in the OR, whereas they achieved a score of only 18.2 (range 14.9–21.5). The proficiency curriculum required residents in the training group to achieve a GOALS-GH score of 24, which was met by all of the residents. This highlights the gap that still remains between the simulation environment and the real clinical environment. The simulator focuses mostly on technical skills and steps of the procedure and helps residents to automate some of this basic knowledge and skill. The curriculum did not address much of the judgment and decision making related to LIHR, which may explain this gap. The concept of transferring part of the learning curve out of the OR, thus allowing learners to automate some of the technical, procedural, and organizational components of the procedure still is very appealing. The described model recognizes the limitations of working memory and cognitive capacity. If training is designed to help residents learn whatever can be learned outside the OR, the assumption is that they will be better prepared for their clinical experiences. If they have more working memory and attention available, they may be more able to prevent errors and learn judgment and decision making, which are challenging to simulate. Our study did not investigate patient outcomes, the ultimate barometer of simulation training effectiveness. In 2011, Zendejas et al. [11] examined not only the influence of TEP simulation training on performance but also the clinical impact on patients. In their large randomized trial, the training group participated in a mastery learning program consisting of both an online course and skills training using a commercially available TEP simulator (Guildford MATTU TEP task trainer; Limbs & Things, Ltd., Bristol, UK). In their study, PGY 1–5 residents practiced hernia sac reduction and mesh fixation until they could complete it within 2 min at a preestablished mastery level in the simulated environment. The training group demonstrated improvements compared with the control group in terms of shorter operative times (6 min faster), better TEP performance as measured by a laparoscopy-specific general global rating scale (GOALS), and fewer complications. Our study demonstrated the transfer of skills required for LIHR using a low-cost procedure-specific simulator of the OR. In 2001, Hamilton et al. [10] were among the first to show a link between simulation training and performance in the OR. After baseline assessments in the OR, they randomized surgical residents to a training program using a
locally created rubber TEP simulator, videos, and an interactive CD-ROM and compared them with a control group (standard residency training). The TEP simulation training was primarily focused on mesh fixation. The training group demonstrated greater improvements in performance of clinical TEP as measured using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale initially developed as a measure of open technical skills on a benchtop model [14]. Although the aforementioned study was very innovative and forward thinking, it is difficult to appreciate how much of the learning was related to the simulation because the training group also used videos and a CD-ROM. In addition, any focused educational intervention that encourages students to think about a procedure is likely to have some benefit over no training at all. We tried to control for this by ensuring that all the residents in our study participated in the teaching session on hernias, which included a review of a step-by-step TEP video. We did not formally assess this, but it is probable that the residents in the training group invested more time and focused more effort on learning TEP in preparation for the simulations and their OR assessments than did the control group. This may be an added bonus of simulation-based education, and in our opinion, it is a phenomenon likely to be observed outside the context of a study as well. This study adds to the evidence supporting the use of procedure-specific competency-based simulation programs in the surgical curriculum. Ongoing unanswered questions remain regarding how to integrate the programs, at what stage, and with what regularity. Problems of scheduling, of being able to provide useful and accurate feedback, and of motivating learners to engage with the process also are ubiquitous and very difficult to solve with limited resources. The culture is slowly changing, however, and as we accumulate increasing evidence, teachers, learners, and patients will recognize the value of these educational interventions and the role they can play. The MLIHS is low cost in terms of materials, at $128 for the entire study. This does not include, however, the laparoscopic tower, instruments, and tackers used, which may not be readily available to all training programs. Moreover, the value of having access to a trained proctor who can provide specific high-quality feedback is certainly the cornerstone of effective deliberate practice and probably is one of the biggest logistical hurdles [15]. The costs and feasibility of this also need to be taken into account when the use of a simulation-based curriculum for procedural skills is considered. Commitment of faculty to train proctors when appropriate or to train residents to be proctors in their more senior years may eventually result in a culture change that will have lasting, sustainable benefits, but this will require years of targeted and well-structured efforts.
123
Surg Endosc
In the meantime, simulation is certainly a useful adjunct to clinical training for TEP hernia repair, and our residents found it to be valuable. Interestingly, the residents found the simulator to be the least useful for hernia sac reduction, with a mean reported educational value of 3 compared with a value greater than 4 for all the other steps. The greatest increase in performance compared with baseline, however, was observed in hernia sac reduction (Table 2). This may be explained perhaps by the ‘‘they don’t know what they don’t know’’ phenomenon, referred to in the education and sociology literature as the Dunning-Kruger effect [16]. Alternatively, this may show the complexity of trying to simulate procedures versus using deconstructed tasks, which is a topic for future research in the area of procedural simulation. Skills acquired in the simulated environment using a low-cost procedure-specific simulator do transfer to the OR. Residents who trained to proficiency on the MLIHS showed greater skill improvement than their colleagues who did not. These results provide evidence supporting the use of simulation to teach and assess LIHR. Acknowledgments The authors thank the McGill general surgery residents for their participation in this study. We could not have completed the study without their diligence and support. Disclosures The salary of Yo Kurashima was funded by an unrestricted Covidien grant during the time this research was conducted, but this is not believed to have influenced the research in any way. Liane S. Feldman is a consultant for Covidien and had a centre of excellence Grant from Covidien as well as an investigator-initiated research grant from Ethicon. Pepa Kaneva had a Covidien unrestricted grant to fund research. Gerald M. Fried had an Covidien unrestricted grant to fund research, and his son works for CAE Healthcare. Melina C. Vassiliou is a Covidien consultant and had an unrestricted Grant to fund research. Simon Bergman, Sebastian V. Demyttenaere, and Chao Li have no conflicts of interest or financial ties to disclose.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
References 15. 1. Reznick RK, MacRae H (2006) Teaching surgical skills: changes in the wind. N Engl J Med 355:2664–2669 2. Simons MP, Aufenacker T, Bay-Nielsen M, Bouillot JL, Campanelli G, Conze J, de Lange D, Fortelny R, Heikkinen T, Kingsnorth A, Kukleta J, Morales-Conde S, Nordin P, Schumpelick V, Smedberg S, Smietanski M, Weber G, Miserez
123
16.
M (2009) European Hernia Society guidelines on the treatment of inguinal hernia in adult patients. Hernia 13:343–403. doi:10. 1007/s10029-009-0529-7 Neumayer L, Giobbie-Hurder A, Jonasson O, Fitzgibbons R, Dunlop D, Gibbs J, Reda D, Henderson W (2004) Open mesh versus laparoscopic mesh repair of inguinal hernia. N Engl J Med 350:1819–1827 The MRC Laparoscopic Groin Hernia Trial Group (1999) Laparoscopic versus open repair of groin hernia: a randomised comparison. Lancet 354:185–190 Rutkow IM (2003) Demographic and socioeconomic aspects of hernia repair in the United States in 2003. Surg Clin North Am 83:1045–1051 Jacob BP, Tong W, Reiner M, Vine A, Katz LB (2009) Singleincision total extraperitoneal (one SITE) laparoscopic inguinal hernia repair using a single-access port device. Hernia 13:571–572 Qureshi A, Vergis A, Jimenez C, Green J, Pryor A, Schlachta CM, Okrainec A (2011) MIS training in Canada: a national survey of general surgery residents. Surg Endosc 25:3057–3065 Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–528 Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM (2010) Fundamentals of Laparoscopic Surgery simulator training to proficiency improves laparoscopic performance in the operating room: a randomized controlled trial. Am J Surg 199:115–120 Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB (2001) Improving operative performance using a laparoscopic hernia simulator. Am J Surg 182:725–728 Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG, Farley DR (2011) Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg 254:502–511 Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A novel low-cost simulator for laparoscopic inguinal hernia repair. Surg Innov 18:171–175 Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg 201:54–61 Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective Structured Assessment of Technical Skill (OSATS) for surgical residents. Br J Surg 84:273–278 Ericsson KA (2008) Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 15:988–994. doi:10.1111/j.1553-2712.2008.00227.x Kruger J, Dunning D (1999) Unskilled and unaware of it: how difficulties of recognizing one’s own incompetence lead to inflated self-assessments. J Per Soc Psychol 77:1121–1134
