Hernia DOI 10.1007/s10029-015-1357-6
ORIGINAL ARTICLE
Module based training improves and sustains surgical skills: a randomised controlled trial C. G. Carlsen • K. Lindorff-Larsen • P. Funch-Jensen • L. Lund • L. Konge P. Charles
•
Received: 4 May 2014 / Accepted: 20 February 2015 Ó Springer-Verlag France 2015
Abstract Purpose Traditional surgical training is challenged by factors such as patient safety issues, economic considerations and lack of exposure to surgical procedures due to short working hours. A module-based clinical training model promotes rapidly acquired and persistent surgical skills. Methods A randomised controlled trial concerning supervised hernia repair in eight training hospitals in Denmark was performed. The participants were 18 registrars [Post graduate year (PGY) 3 or more] in their first year of surgical specialist training. The intervention consisted of
Parts of the results have formerly been presented at Surgicon 2013 and AMEE 2013. ClinicalTrials.gov identifier: NCT01877824. C. G. Carlsen (&) P. Charles Centre of Medical Education, Aarhus University, Incuba Science Park, Brendstrupgaardsvej 102, 8200 Aarhus N, Denmark e-mail: [email protected] K. Lindorff-Larsen NordSim, Centre for Simulation and Skills Training, Aalborg University Hospital, Hobrovej 18, 9000 Aalborg, Denmark P. Funch-Jensen Clinical Institute, Skejby Hospital, Aarhus University, Brendstrupgaardsvej 100, 8200 Aarhus N, Denmark L. Lund Department of Urology, Odense University Hospital, Sdr. Boulevard 29, 5000 Odense, Denmark L. Konge Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100 Copenhagen, Denmark
different modules with a skills-lab course followed by 20 supervised Lichtenstein hernia repairs. Operative performance was video recorded and blindly rated by two consultants using a previously validated skills rating scale (8–40 points). Outcome measures were change in the ratings of operative skills and operative time. Results In the intervention group (n = 10) the average rating of operative skills before intervention was 22.5 (20.6–24.3) and after 26.2 (23.5–28.8), p = 0.044. At follow-up after 1 year, rating was 26.9 (23.4–30.4), p = 0.019. In the conventionally trained group average rating was 23.4 (19.4–27.3) at start and 21.7 (17.3–26.1) at end, p = 0.51. At start no difference was detected between the two groups, p = 0.59; by 1 year the difference was statistically significant favouring intervention, p = 0.044. Operative time showed similar results in favour of the intervention. Conclusions A module-based training model in Lichtenstein hernia repair was preferable in both short and longterm compared with standard clinical training. The model will probably be applicable to other surgical training procedures. Keywords Lichtenstein hernia repair Surgical training Workplace based assessment
Purpose Apprenticeship and supervision by experienced surgeons remain the cornerstones of surgical training. In recent years, however, surgical skills laboratory training has evolved providing opportunities for deliberate practice of specific tasks as preparation for training in the clinical operative setting [1]. The work-hour directives challenge
123
Hernia
the apprenticeship model, and patient safety issues promote the concept, that initial training should take place in a simulated setting without risk for patients. Simulation is undoubtedly helpful in basic training, but most surgical procedures are complicated tasks that must finally be trained in patients to gain full competency in complete procedures as performance is holistic in nature. The nontechnical conditions of the real theatre are not easily simulated and have great impact on the surgical performance due to tacit knowledge [2]. Furthermore, continuous and deliberate training is important to avoid an arrested development in skills performance as pointed out by Ericsson [3]. Future training models need to adapt to new circumstances including work-hour restrictions. To our knowledge, no former study has described the usefulness of a module-based clinical training programme in a specific surgical hernia repair procedure. This study focused on Lichtenstein hernia repair as a common and important surgical procedure encountered by trainees early in training [4]. The trainees were randomised to either module-based training with performance of 20 procedures within a condensed time period or a conventional training regimen. The conventional programme did not focus training in specific procedures, but was only determined by work shift schedules repeating every 8th to 12th week. Consequently, time spent in surgical day units might be every 8th to 12th week and entail different kinds of surgical procedures at each session, not hernia repairs only. Learning objectives in performing Lichtenstein hernia repair were not assessed at any fixed time, by any specific method or by any required number. No theoretical courses in hernia surgery were offered in the conventional programme. The conventional model was suited for a traditional apprenticeship. The concept of one master surgeon training a number of apprentices and following them over a period of time has been abandoned as impracticable due to new work schedules. In Denmark a competency-based educational model was introduced 10 years ago, but in clinical practice the training has remained unchanged. The arbitrary choice of 20 procedures in the intervention training module was based on the authors’ experience with expected skills exposure among Danish surgical trainees in first year of training. Literature was sparse in this field but indicated a wide range in trainees’ number of performed procedures [4, 5]. The study did not intend to test full autonomy in performing Lichtenstein hernia repair after only 20 procedures. All procedures in the study were supervised by senior surgeons. Instead, the aim of the study was to test the effects of a module-based training model in Lichtenstein hernia repair technical skills in a randomised study of first-year surgical trainees (registrars). A more focused approach may benefit training outcome and suit the aims of competency-based surgical training. We aimed to
123
explore the changes in trainees’ technical skills by videotape-rated assessment. We expected the intervention model to reduce operative time and increase the surgical proficiency. Furthermore, we explored trainees’ experience and self-assessed confidence short interviews.
Methods Design The study was randomised in two parallel groups: intervention and conventionally trained. Randomisation was made with consecutively numbered envelopes when trainees started their first year of formal surgical training. If two trainees started employment in the same department simultaneously, they were block randomised to avoid grudges among their registrar colleagues. We included results from the northern educational region of Denmark. Exclusion criterion was prior operative experience to the extent of permission to perform Lichtenstein hernia repairs unsupervised. Trainees at non-participating hospitals were not included. Setting The study was conducted in eight Danish surgical departments in Jutland. Two departments, however, withdrew their consent shortly after study start. Surgery was performed in day units and supervised by local senior staff identical to any other clinical surgical training session in the hospitals. The supervisors knew the participant’s randomization group, but were not involved in the final blinded ratings. The intervention differed from the usual training programme only in the demand for 20 planned Lichtenstein hernia repair procedures within a set time (four to eight weeks) and the video recordings. The recordings were taped directly in the operative theatre from skin incision to wound closure. For most trainees the programme consisted of four to six days in the surgical day unit during the time period. The study did not intend that other learning opportunities were missed during the intervention. The days in the day unit were intended to be planned condensed within weeks instead of massed throughout the first year. Scheduled training might in some cases have been cancelled if acute patient care issues demanded the registrars’ presence elsewhere. Prior to the first operative procedure a skills course was conducted in a skills lab setting. In the conventionally trained group only the video recordings differed from an ordinary training session. In all other aspects of training, both groups followed the usual
Hernia
training programme and expected learning opportunities as planned by their departments. Participants The Danish post-graduate surgical training consisted of 1 year general basic training, a minimum of 1 year of surgical introduction and finally, 5 years of specialty training. Consequently, the participants in this study equalled the US Post graduate year (PGY) 3 or more. 25 Danish surgical trainees were accepted for the 5-year surgical specialist training allocated in eight different hospitals during the years 2010–11. Eighteen trainees were randomised by the investigator just prior to employment start of their first year; one conventionally trained group participant did not contribute due to his department’s withdrawal from the study, and one intervention group participant was lost to follow-up due to employment change to a non-participating department. Seven trainees were not included in the study: Three were excluded because their departments refused to participate and two trainees did not meet the inclusion criteria. Two trainees did not respond to the invitation for unknown reasons despite reminders. Figure 1 shows participant flow. Intervention group Intervention group participants were allocated to a 1-day skills lab course in Lichtenstein hernia repair within their first 2 months of first-year training. The course contained theory, skills training on a plastic phantom mimicking the human male groin (Figs. 2, 3, 4) and surgery in an anaesthetized pig with congenital hernia for dissection training purposes (Figs. 5, 6). Following the course they were assigned to perform 20 supervised primary Lichtenstein hernia repairs within 4–8 weeks. They were allocated 4–6 days in their departments’ day surgery facilities. If the trainees were removed from other duty this was covered by other members of the staff not attending the study. No learning objectives were missed by attending doctors or other members of staff due to the project. On the first, mid, and last day of the assigned procedures their performances were video recorded. The mid day recording was obtained to evaluate the sufficiency of the chosen number of procedures. A fourth video recording of a similar Lichtenstein hernia repair performed by the participants was obtained within the last months of the first training year. Besides the planned day unit assignments, the participants followed the regular portfolio training programme in their departments. At the end of intervention, a short interview (5–10 min) was conducted individually with a 9-question semi-structured guide to explore trainees’ experience and self-assessed confidence following the module based training programme.
Conventionally trained group The conventionally trained group participants were asked to notify the investigator about their first scheduled supervised Lichtenstein hernia repair, and this was video recorded in the same way as the intervention group recordings. Similarly, they notified about the last assigned procedure in the final months of their first training year and then the second recording was obtained. During the study period the trainees followed the regular portfolio training programme in their departments. Both groups were asked to count their personal number of Lichtenstein hernia repairs performed before study entry and during the study (from the first video recording in the study to the last video recording by the end of the first training year). The study was conducted in seven different departments. Three departments had trainees in the intervention group. three departments had conventionally trained group participants and one department both (Fig. 1). Video recordings All video recordings were blinded regarding patient, trainee, supervisor and department. Only the operative field was recorded. The procedure was recorded from skin incision to wound closure. All recordings in this study concerned only primary inguinal hernia to ensure a standardized procedure. All ratings were blindly performed by three independent consultants, each video was rated by two raters and in order to mix the recordings to the raters, no recordings were rated before the study was finished. None of the raters supervised the hernia repairs or contributed to the video recordings. Ethics Patients accepted the recordings prior to the procedure and gave informed consent. The video recordings were anonymous and no patient data were recorded or known to the raters. Consequently, no permission from the Research Ethics Committee or the Danish Data Protection Agency was needed according to Danish legislation. The study was reported to Clinical Trials as a randomised study, ClinicalTrials.gov identifier: NCT01877824. This study did not include changes in delivered patient care as all procedures were supervised by senior doctors. The study added a possible improvement to training in the intervention group, but no changes occurred in the planned training of the conventionally trained group. Conventionally trained group participants were offered participation in the hernia course at end of study (their first training year). For the hernia courses, a porcine model was used for
123
Hernia
Fig. 1 Flow diagram of participants in intervention group and conventionally trained group
training purposes, and all ethical regulations were observed. All pigs were anaesthetized by authorized personnel at a specially designed operating room at the Aarhus University, Faculty of Agricultural Science in Foulum, and euthanized at the end of the course. The pigs had congenital hernias which provided the opportunity for dissection training.
123
Main outcome measures All video recordings were blindly assessed by two surgically experienced raters using a previously validated assessment tool for Lichtenstein hernia repair with 8 items designed to objectively assess the procedure quality [6]. This prior study showed a good inter-rater reliability.
Hernia
Fig. 2 Open inguinal hernia repair trainer—incision
Fig. 5 Dissection of the pig’s groin
Fig. 3 Open inguinal hernia repair trainer—spermatic cord
Fig. 6 Pig training course
Minimum score was 8 and maximum score was 40. Results are presented using an average of the raters’ score of each recording. Time to complete the procedure was also noted. Trainees’ experience and confidence were recorded in the interviews. Statistics
Fig. 4 Open inguinal hernia repair trainer—suturing mesh
Rating scores are presented as average score of two observers. The level of statistical significance was defined as p \ 0.05. Normal distribution was assumed on behalf of study design and further visualised by qq-plots. Data are shown as mean (95 % Confidence interval) and compared using either paired or independent samples t test as appropriate. Sample size calculations (power analysis) were
123
Hernia
not conducted, as all trainees possible were included. Statistical analyses were performed using a statistical software package: STATA ver. 11.1, 2009, StataCorp, 4905 Lakeway Drive, College Station, Texas 77845, USA.
Results Table 1 shows the demographics of the participants. Time in intervention and numbers of performed procedures are displayed in Table 2. Results of the ratings and operative time in both the intervention and conventionally trained group are shown in Table 3. At study start no difference was detected between the two groups, p = 0.59; by 1 year the difference in rating was 5.2 and statistically significant favouring intervention, p = 0.044. The intervention group improved, whereas the conventionally trained group did not improve their technical skills performance during the first year of specialty training. Results are also displayed graphically in Fig. 7a, b. Operative time showed results similar to the ratings. At start no difference was detected between the two groups, p = 0.87. At follow-up by 1 year, difference in mean operative time in the two groups was 21.2 min., though this difference failed to reach a statistically significant difference, p = 0.059.
We obtained a video recording during the intervention period approximately mid-way. By the time of this measurement mean number of procedures was 9. These recordings were rated similarly, and no statistically significant difference in rating score or time was found compared to either the first or the last recording of the intervention period. The short interviews conducted at end of intervention all showed a high degree of participant satisfaction with the programme and supplemented the findings from the blinded ratings. All 10 trainees concluded that the programme had been favourable and provided a high degree of safety in performing the procedure. Nine out of ten felt that they were ready to perform the procedure without supervision at the end of the intervention. Table 4 displays some quotes from the interviews.
Discussion This prospective randomised study has shown that surgical skills are improved and retained after a module-based training programme whereas no improvement in technical skills or operative time is seen in the conventionally trained group. In other words our current practice, where operative
Table 1 Participant background data
Intervention Mean (95 % CI)
Conventionally trained Mean (95 % CI)
p
No. of participants
10
8
NS
Age (years)
32.4 (30.7–34.1)
33.8 (31.8–35.8)
NS
Male/female (No)
7/3
4/4
NS
Prior Lichtenstein hernia repair experience (No)
12.9 (8.0–17.8)
20.4 (9.7–31.0)
NS
Prior surgical employment (months)
23.8 (18.9–28.7)
21.0 (17.0–25.0)
NS
Conventionally trained Mean (95 % CI)
p
Table 2 Time in intervention and number of performed procedures
Intervention Mean (95 % CI) No. of Lichtenstein rep. during intervention
16.8 (14.0–19.6)
Time in intervention (days)
37.9 (29.5–46.3)
No. of Lichtenstein rep. during the first year Days to follow-up
Table 3 Results of the modulebased training study in hernia repair in average score and time to complete procedure
29.3 (18.4–40.2) 300.2 (271.3–329.1)
Initially
End of intervention
p
17.6 (4.4–30.7)
NS
282.4 (252.3–312.5)
Follow-up
NS
p
Intervention Average score
22.5 (20.6–24.3)
26.2 (23.5–28.8)
0.044
26.9 (23.4–30.4)
0.019
Time (min)
53.3 (45.7–60.9)
38.5 (32.4–44.6)
0.009
40.1 (33.0–47.2)
\0.001
Conventionally trained
123
Average score
23.4 (19.4–27.3)
21.7 (17.3–26.1)
0.51
Time (min)
52.3 (39.3–65.3)
61.3 (39.3–83.3)
0.40
Hernia Fig. 7 a Box plot comparing average rating score in intervention group and conventionally trained group. b Box plot comparing operative time in intervention group and conventionally trained group
Table 4 Selected trainee answers (translated) ‘‘I think this has been a good way to be trained—by performing this type of standard operations, and several in a row to get some volume. I definitely feel more comfortable about dealing with it now’’. Trainee no. 15 ‘‘ This is exactly the way to learn to operate. You are scheduled for a lot of operations close together, instead of doing one operation and then another after a month. It gives a completely different flow and you become more sure of your technique and the way to do it’’. Trainee no. 8 ‘‘I feel that I am ready to deal with hernias on my own now. Of course you may encounter something you cannot manage and need to have help, but I feel I can deal with the vast majority now’’. Trainee no. 6
training is often scheduled when it is convenient rather than as part of a structured plan, is inferior to a well-planned programme with a specified number of procedures carried out over a short period of time. In recent years surgical training in the clinical setting has been discussed in the light of patient safety issues [7]. There is an increasing use of simulation and skills training in laboratories, but the main part of surgical training still necessarily takes place in the operative theatre in real patients. The intervention in the present study consisted of both a short skills lab course and the operative procedures and it is not possible to separate the effect. The combination is probably optimal to assure both correct operative technique and operative experience. Dispersed skills training is superior to massed training [8, 9], though training dispersed throughout a very long time period does not seem to be advantageous. The effect of the training model was not only a statistically significantly higher rated technical performance but also lower operative time. Lower operative time as result of training is well described and indeed naturally expected [10–12] and profitable for productivity as well as for the patients. The study results suggest the extent of a necessary training effort. Due to the study size and design patient outcome was not measured here. Supervised novices ought to have the same low rates of complications as their supervisors and
unsupervised training was considered unethical in this study concerning relatively inexperienced trainees. Complication rates in Lichtenstein hernia repair are low and the needed number of patients to detect a difference would have outnumbered the trainees available for this study. However, it has been shown that higher assessment scores are correlated with better patient outcomes [13], and the improved assessment scores in our study indicate improved surgical technique with the potential of reducing operative complications and recurrences. The average number of supervisors was 3.6. This reflected daily clinical work conditions in Denmark and probably elsewhere. We did not intervene in the supervisor allocation as this was impossible with operations taking place in several different hospital settings. Supervisors were not involved in the ratings. Randomising was expected to account for this bias. Furthermore, the information of numbers of different supervisors and of their experience leads us to conclude that the supervisor issue did not influence the study in favour of the intervention. The number of performed procedures differed in the two groups by end of study. However, the difference was statistically non-significant. The trainees in the intervention group performed approximately 17 repairs during the intervention period equal to the conventionally trained group’s performance during the whole year. After the
123
Hernia
intervention period the intervention group did not improve their average score in spite of the added number of procedures. This adds weight to our conclusion that condensed training is superior to the more dispersed training. In the above mentioned Danish context of a 37-h work week, it was not surprising that the trainees in this programme performed a limited number of operations. Time constraints in the education of surgeons have also become relevant for other countries. In line with the European Working Time Directive a 48-h work week for doctors in training was implemented in August 2009 and the possible detrimental effect of this on training is now being explored [14–16]. In the United States the current 80-h work week has made experienced surgeons wonder whether the new duty hour restrictions provide sufficient time to reach the level of proficiency necessary to support the safe, independent practice of medicine [17]. This study provided a possible model of surgical clinical training with tight workhour restrictions. From an advantage point of view, the study is prospective and randomised. Few educational studies in a clinical setting are randomised. This may be due to problems with blinding as the awareness of a study being conducted may cause changes in work-planning. Therefore, the study was focused on one specific procedure, limited in time, and trainees with same employment start date were randomised in identical groups. Another advantage of the study was the blinded rating of the video-recorded performance. The most experienced trainees were not enrolled, as they would not receive supervision. Supervision was provided by the local consultants as it would have been otherwise. Using specially trained supervisors instead of local consultants may have further improved results though impairing the generalizability of the study to the clinical training context. Regarding disadvantages, the study is small in participant number, although all new trainees were considered for participation. However, results were statistically significant despite the small number indicating large effect sizes. The decrease of operative time is clear, though not statistically significant. This may be due to Type 2 error. One possible confounder was the number of performed procedures prior to intervention. We tested this in a regression analysis, which showed no statistically significant difference between the groups at study entry.
Conclusions This module-based training model in Lichtenstein hernia repair in a daily clinical setting showed significant and sustained improvement in technical skills as compared to current practice. The study showed high participant
123
contentment, and as the study was conducted in a clinical setting it would be easy to apply. The model will, moreover, probably be applicable in other surgical training procedures. Acknowledgments We thank the doctors and the staff involved in skills lab course, participating trainees, supervisors and patients accepting the video recording of the hernia repairs. Furthermore, this study owes a special thanks to the huge work of the consultants involved in the rating of the recordings. A special thanks to the Tryg foundation that sponsored the skills lab courses. Thank you to the public funders of the research: Aarhus University, Aalborg University Hospital and The Central Denmark Region. Conflict of interest Dr. C Carlsen declares no conflict of interest.Dr. K Lindorff-Larsen declares no conflict of interest.Dr. P FunchJensen declares no conflict of interest.Dr. L Lund declares no conflict of interest.Dr. L Konge declares no conflict of interest.Dr. P Charles declares no conflict of interest.
References 1. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR (2009) Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev (1):CD006575. doi:10.1002/ 14651858.CD006575 2. Eraut M (2004) Informal learning in the workplace. Stud Cont Edu 26:274–293 3. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79:S70–S81 4. Bell RH Jr, Biester TW, Tabuenca A, Rhodes RS, Cofer JB, Britt LD, Lewis FR Jr (2009) Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg 249:719–724 5. Davies BW, Campbell WB (1995) Inguinal hernia repair: see one, do one, teach one? Ann R Coll Surg Engl 77:299–301 6. Carlsen CG, Lindoff-Larsen K, Funch-Jensen P, Lund L, Charles P, Konge L (2013) Reliable and valid assessment of Lichtenstein hernia repair skills. Hernia 18(4):543–548 7. Department of Health U (2006) Good doctors, safer patients. A report by the Chief Medical Officer. No. 276071. http://www.dh. gov.uk/publications 8. Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R (2006) Teaching surgical skills: what kind of practice makes perfect: a randomized, controlled trial. Ann Surg 244:400–409 9. Verdaasdonk EG, Stassen LP, van Wijk RP, Dankelman J (2007) The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc 21:214–219 10. Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, Stone M (2004) The, ‘‘cost’’ of operative training for surgical residents. Arch Surg 139:366–369 11. Koperna T (2004) How long do we need teaching in the operating room? The true costs of achieving surgical routine. Langenbecks Arch Surg 389:204–208 12. Wilkiemeyer M, Pappas TN, Giobbie-Hurder A, Itani KM, Jonasson O, Neumayer LA (2005) Does resident post graduate year influence the outcomes of inguinal hernia repair? Ann Surg 241:879–882 13. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical
Hernia skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442 14. Blencowe NS, Parsons BA, Hollowood AD (2011) Effects of changing work patterns on general surgical training over the last decade. Postgrad Med J 87:795–799 15. Canter R (2011) Impact of reduced working time on surgical training in the United Kingdom and Ireland. Surgeon 9(Suppl 1):S6–S7
16. Eaton L (2010) Working time directive puts patients’ lives at risk, surgeons claim. BMJ 341:c4212 17. Pellegrini VD Jr (2012) Perspective: ten thousand hours to patient safety, sooner or later. Acad Med 87:164–167
123
ORIGINAL ARTICLE
Module based training improves and sustains surgical skills: a randomised controlled trial C. G. Carlsen • K. Lindorff-Larsen • P. Funch-Jensen • L. Lund • L. Konge P. Charles
•
Received: 4 May 2014 / Accepted: 20 February 2015 Ó Springer-Verlag France 2015
Abstract Purpose Traditional surgical training is challenged by factors such as patient safety issues, economic considerations and lack of exposure to surgical procedures due to short working hours. A module-based clinical training model promotes rapidly acquired and persistent surgical skills. Methods A randomised controlled trial concerning supervised hernia repair in eight training hospitals in Denmark was performed. The participants were 18 registrars [Post graduate year (PGY) 3 or more] in their first year of surgical specialist training. The intervention consisted of
Parts of the results have formerly been presented at Surgicon 2013 and AMEE 2013. ClinicalTrials.gov identifier: NCT01877824. C. G. Carlsen (&) P. Charles Centre of Medical Education, Aarhus University, Incuba Science Park, Brendstrupgaardsvej 102, 8200 Aarhus N, Denmark e-mail: [email protected] K. Lindorff-Larsen NordSim, Centre for Simulation and Skills Training, Aalborg University Hospital, Hobrovej 18, 9000 Aalborg, Denmark P. Funch-Jensen Clinical Institute, Skejby Hospital, Aarhus University, Brendstrupgaardsvej 100, 8200 Aarhus N, Denmark L. Lund Department of Urology, Odense University Hospital, Sdr. Boulevard 29, 5000 Odense, Denmark L. Konge Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100 Copenhagen, Denmark
different modules with a skills-lab course followed by 20 supervised Lichtenstein hernia repairs. Operative performance was video recorded and blindly rated by two consultants using a previously validated skills rating scale (8–40 points). Outcome measures were change in the ratings of operative skills and operative time. Results In the intervention group (n = 10) the average rating of operative skills before intervention was 22.5 (20.6–24.3) and after 26.2 (23.5–28.8), p = 0.044. At follow-up after 1 year, rating was 26.9 (23.4–30.4), p = 0.019. In the conventionally trained group average rating was 23.4 (19.4–27.3) at start and 21.7 (17.3–26.1) at end, p = 0.51. At start no difference was detected between the two groups, p = 0.59; by 1 year the difference was statistically significant favouring intervention, p = 0.044. Operative time showed similar results in favour of the intervention. Conclusions A module-based training model in Lichtenstein hernia repair was preferable in both short and longterm compared with standard clinical training. The model will probably be applicable to other surgical training procedures. Keywords Lichtenstein hernia repair Surgical training Workplace based assessment
Purpose Apprenticeship and supervision by experienced surgeons remain the cornerstones of surgical training. In recent years, however, surgical skills laboratory training has evolved providing opportunities for deliberate practice of specific tasks as preparation for training in the clinical operative setting [1]. The work-hour directives challenge
123
Hernia
the apprenticeship model, and patient safety issues promote the concept, that initial training should take place in a simulated setting without risk for patients. Simulation is undoubtedly helpful in basic training, but most surgical procedures are complicated tasks that must finally be trained in patients to gain full competency in complete procedures as performance is holistic in nature. The nontechnical conditions of the real theatre are not easily simulated and have great impact on the surgical performance due to tacit knowledge [2]. Furthermore, continuous and deliberate training is important to avoid an arrested development in skills performance as pointed out by Ericsson [3]. Future training models need to adapt to new circumstances including work-hour restrictions. To our knowledge, no former study has described the usefulness of a module-based clinical training programme in a specific surgical hernia repair procedure. This study focused on Lichtenstein hernia repair as a common and important surgical procedure encountered by trainees early in training [4]. The trainees were randomised to either module-based training with performance of 20 procedures within a condensed time period or a conventional training regimen. The conventional programme did not focus training in specific procedures, but was only determined by work shift schedules repeating every 8th to 12th week. Consequently, time spent in surgical day units might be every 8th to 12th week and entail different kinds of surgical procedures at each session, not hernia repairs only. Learning objectives in performing Lichtenstein hernia repair were not assessed at any fixed time, by any specific method or by any required number. No theoretical courses in hernia surgery were offered in the conventional programme. The conventional model was suited for a traditional apprenticeship. The concept of one master surgeon training a number of apprentices and following them over a period of time has been abandoned as impracticable due to new work schedules. In Denmark a competency-based educational model was introduced 10 years ago, but in clinical practice the training has remained unchanged. The arbitrary choice of 20 procedures in the intervention training module was based on the authors’ experience with expected skills exposure among Danish surgical trainees in first year of training. Literature was sparse in this field but indicated a wide range in trainees’ number of performed procedures [4, 5]. The study did not intend to test full autonomy in performing Lichtenstein hernia repair after only 20 procedures. All procedures in the study were supervised by senior surgeons. Instead, the aim of the study was to test the effects of a module-based training model in Lichtenstein hernia repair technical skills in a randomised study of first-year surgical trainees (registrars). A more focused approach may benefit training outcome and suit the aims of competency-based surgical training. We aimed to
123
explore the changes in trainees’ technical skills by videotape-rated assessment. We expected the intervention model to reduce operative time and increase the surgical proficiency. Furthermore, we explored trainees’ experience and self-assessed confidence short interviews.
Methods Design The study was randomised in two parallel groups: intervention and conventionally trained. Randomisation was made with consecutively numbered envelopes when trainees started their first year of formal surgical training. If two trainees started employment in the same department simultaneously, they were block randomised to avoid grudges among their registrar colleagues. We included results from the northern educational region of Denmark. Exclusion criterion was prior operative experience to the extent of permission to perform Lichtenstein hernia repairs unsupervised. Trainees at non-participating hospitals were not included. Setting The study was conducted in eight Danish surgical departments in Jutland. Two departments, however, withdrew their consent shortly after study start. Surgery was performed in day units and supervised by local senior staff identical to any other clinical surgical training session in the hospitals. The supervisors knew the participant’s randomization group, but were not involved in the final blinded ratings. The intervention differed from the usual training programme only in the demand for 20 planned Lichtenstein hernia repair procedures within a set time (four to eight weeks) and the video recordings. The recordings were taped directly in the operative theatre from skin incision to wound closure. For most trainees the programme consisted of four to six days in the surgical day unit during the time period. The study did not intend that other learning opportunities were missed during the intervention. The days in the day unit were intended to be planned condensed within weeks instead of massed throughout the first year. Scheduled training might in some cases have been cancelled if acute patient care issues demanded the registrars’ presence elsewhere. Prior to the first operative procedure a skills course was conducted in a skills lab setting. In the conventionally trained group only the video recordings differed from an ordinary training session. In all other aspects of training, both groups followed the usual
Hernia
training programme and expected learning opportunities as planned by their departments. Participants The Danish post-graduate surgical training consisted of 1 year general basic training, a minimum of 1 year of surgical introduction and finally, 5 years of specialty training. Consequently, the participants in this study equalled the US Post graduate year (PGY) 3 or more. 25 Danish surgical trainees were accepted for the 5-year surgical specialist training allocated in eight different hospitals during the years 2010–11. Eighteen trainees were randomised by the investigator just prior to employment start of their first year; one conventionally trained group participant did not contribute due to his department’s withdrawal from the study, and one intervention group participant was lost to follow-up due to employment change to a non-participating department. Seven trainees were not included in the study: Three were excluded because their departments refused to participate and two trainees did not meet the inclusion criteria. Two trainees did not respond to the invitation for unknown reasons despite reminders. Figure 1 shows participant flow. Intervention group Intervention group participants were allocated to a 1-day skills lab course in Lichtenstein hernia repair within their first 2 months of first-year training. The course contained theory, skills training on a plastic phantom mimicking the human male groin (Figs. 2, 3, 4) and surgery in an anaesthetized pig with congenital hernia for dissection training purposes (Figs. 5, 6). Following the course they were assigned to perform 20 supervised primary Lichtenstein hernia repairs within 4–8 weeks. They were allocated 4–6 days in their departments’ day surgery facilities. If the trainees were removed from other duty this was covered by other members of the staff not attending the study. No learning objectives were missed by attending doctors or other members of staff due to the project. On the first, mid, and last day of the assigned procedures their performances were video recorded. The mid day recording was obtained to evaluate the sufficiency of the chosen number of procedures. A fourth video recording of a similar Lichtenstein hernia repair performed by the participants was obtained within the last months of the first training year. Besides the planned day unit assignments, the participants followed the regular portfolio training programme in their departments. At the end of intervention, a short interview (5–10 min) was conducted individually with a 9-question semi-structured guide to explore trainees’ experience and self-assessed confidence following the module based training programme.
Conventionally trained group The conventionally trained group participants were asked to notify the investigator about their first scheduled supervised Lichtenstein hernia repair, and this was video recorded in the same way as the intervention group recordings. Similarly, they notified about the last assigned procedure in the final months of their first training year and then the second recording was obtained. During the study period the trainees followed the regular portfolio training programme in their departments. Both groups were asked to count their personal number of Lichtenstein hernia repairs performed before study entry and during the study (from the first video recording in the study to the last video recording by the end of the first training year). The study was conducted in seven different departments. Three departments had trainees in the intervention group. three departments had conventionally trained group participants and one department both (Fig. 1). Video recordings All video recordings were blinded regarding patient, trainee, supervisor and department. Only the operative field was recorded. The procedure was recorded from skin incision to wound closure. All recordings in this study concerned only primary inguinal hernia to ensure a standardized procedure. All ratings were blindly performed by three independent consultants, each video was rated by two raters and in order to mix the recordings to the raters, no recordings were rated before the study was finished. None of the raters supervised the hernia repairs or contributed to the video recordings. Ethics Patients accepted the recordings prior to the procedure and gave informed consent. The video recordings were anonymous and no patient data were recorded or known to the raters. Consequently, no permission from the Research Ethics Committee or the Danish Data Protection Agency was needed according to Danish legislation. The study was reported to Clinical Trials as a randomised study, ClinicalTrials.gov identifier: NCT01877824. This study did not include changes in delivered patient care as all procedures were supervised by senior doctors. The study added a possible improvement to training in the intervention group, but no changes occurred in the planned training of the conventionally trained group. Conventionally trained group participants were offered participation in the hernia course at end of study (their first training year). For the hernia courses, a porcine model was used for
123
Hernia
Fig. 1 Flow diagram of participants in intervention group and conventionally trained group
training purposes, and all ethical regulations were observed. All pigs were anaesthetized by authorized personnel at a specially designed operating room at the Aarhus University, Faculty of Agricultural Science in Foulum, and euthanized at the end of the course. The pigs had congenital hernias which provided the opportunity for dissection training.
123
Main outcome measures All video recordings were blindly assessed by two surgically experienced raters using a previously validated assessment tool for Lichtenstein hernia repair with 8 items designed to objectively assess the procedure quality [6]. This prior study showed a good inter-rater reliability.
Hernia
Fig. 2 Open inguinal hernia repair trainer—incision
Fig. 5 Dissection of the pig’s groin
Fig. 3 Open inguinal hernia repair trainer—spermatic cord
Fig. 6 Pig training course
Minimum score was 8 and maximum score was 40. Results are presented using an average of the raters’ score of each recording. Time to complete the procedure was also noted. Trainees’ experience and confidence were recorded in the interviews. Statistics
Fig. 4 Open inguinal hernia repair trainer—suturing mesh
Rating scores are presented as average score of two observers. The level of statistical significance was defined as p \ 0.05. Normal distribution was assumed on behalf of study design and further visualised by qq-plots. Data are shown as mean (95 % Confidence interval) and compared using either paired or independent samples t test as appropriate. Sample size calculations (power analysis) were
123
Hernia
not conducted, as all trainees possible were included. Statistical analyses were performed using a statistical software package: STATA ver. 11.1, 2009, StataCorp, 4905 Lakeway Drive, College Station, Texas 77845, USA.
Results Table 1 shows the demographics of the participants. Time in intervention and numbers of performed procedures are displayed in Table 2. Results of the ratings and operative time in both the intervention and conventionally trained group are shown in Table 3. At study start no difference was detected between the two groups, p = 0.59; by 1 year the difference in rating was 5.2 and statistically significant favouring intervention, p = 0.044. The intervention group improved, whereas the conventionally trained group did not improve their technical skills performance during the first year of specialty training. Results are also displayed graphically in Fig. 7a, b. Operative time showed results similar to the ratings. At start no difference was detected between the two groups, p = 0.87. At follow-up by 1 year, difference in mean operative time in the two groups was 21.2 min., though this difference failed to reach a statistically significant difference, p = 0.059.
We obtained a video recording during the intervention period approximately mid-way. By the time of this measurement mean number of procedures was 9. These recordings were rated similarly, and no statistically significant difference in rating score or time was found compared to either the first or the last recording of the intervention period. The short interviews conducted at end of intervention all showed a high degree of participant satisfaction with the programme and supplemented the findings from the blinded ratings. All 10 trainees concluded that the programme had been favourable and provided a high degree of safety in performing the procedure. Nine out of ten felt that they were ready to perform the procedure without supervision at the end of the intervention. Table 4 displays some quotes from the interviews.
Discussion This prospective randomised study has shown that surgical skills are improved and retained after a module-based training programme whereas no improvement in technical skills or operative time is seen in the conventionally trained group. In other words our current practice, where operative
Table 1 Participant background data
Intervention Mean (95 % CI)
Conventionally trained Mean (95 % CI)
p
No. of participants
10
8
NS
Age (years)
32.4 (30.7–34.1)
33.8 (31.8–35.8)
NS
Male/female (No)
7/3
4/4
NS
Prior Lichtenstein hernia repair experience (No)
12.9 (8.0–17.8)
20.4 (9.7–31.0)
NS
Prior surgical employment (months)
23.8 (18.9–28.7)
21.0 (17.0–25.0)
NS
Conventionally trained Mean (95 % CI)
p
Table 2 Time in intervention and number of performed procedures
Intervention Mean (95 % CI) No. of Lichtenstein rep. during intervention
16.8 (14.0–19.6)
Time in intervention (days)
37.9 (29.5–46.3)
No. of Lichtenstein rep. during the first year Days to follow-up
Table 3 Results of the modulebased training study in hernia repair in average score and time to complete procedure
29.3 (18.4–40.2) 300.2 (271.3–329.1)
Initially
End of intervention
p
17.6 (4.4–30.7)
NS
282.4 (252.3–312.5)
Follow-up
NS
p
Intervention Average score
22.5 (20.6–24.3)
26.2 (23.5–28.8)
0.044
26.9 (23.4–30.4)
0.019
Time (min)
53.3 (45.7–60.9)
38.5 (32.4–44.6)
0.009
40.1 (33.0–47.2)
\0.001
Conventionally trained
123
Average score
23.4 (19.4–27.3)
21.7 (17.3–26.1)
0.51
Time (min)
52.3 (39.3–65.3)
61.3 (39.3–83.3)
0.40
Hernia Fig. 7 a Box plot comparing average rating score in intervention group and conventionally trained group. b Box plot comparing operative time in intervention group and conventionally trained group
Table 4 Selected trainee answers (translated) ‘‘I think this has been a good way to be trained—by performing this type of standard operations, and several in a row to get some volume. I definitely feel more comfortable about dealing with it now’’. Trainee no. 15 ‘‘ This is exactly the way to learn to operate. You are scheduled for a lot of operations close together, instead of doing one operation and then another after a month. It gives a completely different flow and you become more sure of your technique and the way to do it’’. Trainee no. 8 ‘‘I feel that I am ready to deal with hernias on my own now. Of course you may encounter something you cannot manage and need to have help, but I feel I can deal with the vast majority now’’. Trainee no. 6
training is often scheduled when it is convenient rather than as part of a structured plan, is inferior to a well-planned programme with a specified number of procedures carried out over a short period of time. In recent years surgical training in the clinical setting has been discussed in the light of patient safety issues [7]. There is an increasing use of simulation and skills training in laboratories, but the main part of surgical training still necessarily takes place in the operative theatre in real patients. The intervention in the present study consisted of both a short skills lab course and the operative procedures and it is not possible to separate the effect. The combination is probably optimal to assure both correct operative technique and operative experience. Dispersed skills training is superior to massed training [8, 9], though training dispersed throughout a very long time period does not seem to be advantageous. The effect of the training model was not only a statistically significantly higher rated technical performance but also lower operative time. Lower operative time as result of training is well described and indeed naturally expected [10–12] and profitable for productivity as well as for the patients. The study results suggest the extent of a necessary training effort. Due to the study size and design patient outcome was not measured here. Supervised novices ought to have the same low rates of complications as their supervisors and
unsupervised training was considered unethical in this study concerning relatively inexperienced trainees. Complication rates in Lichtenstein hernia repair are low and the needed number of patients to detect a difference would have outnumbered the trainees available for this study. However, it has been shown that higher assessment scores are correlated with better patient outcomes [13], and the improved assessment scores in our study indicate improved surgical technique with the potential of reducing operative complications and recurrences. The average number of supervisors was 3.6. This reflected daily clinical work conditions in Denmark and probably elsewhere. We did not intervene in the supervisor allocation as this was impossible with operations taking place in several different hospital settings. Supervisors were not involved in the ratings. Randomising was expected to account for this bias. Furthermore, the information of numbers of different supervisors and of their experience leads us to conclude that the supervisor issue did not influence the study in favour of the intervention. The number of performed procedures differed in the two groups by end of study. However, the difference was statistically non-significant. The trainees in the intervention group performed approximately 17 repairs during the intervention period equal to the conventionally trained group’s performance during the whole year. After the
123
Hernia
intervention period the intervention group did not improve their average score in spite of the added number of procedures. This adds weight to our conclusion that condensed training is superior to the more dispersed training. In the above mentioned Danish context of a 37-h work week, it was not surprising that the trainees in this programme performed a limited number of operations. Time constraints in the education of surgeons have also become relevant for other countries. In line with the European Working Time Directive a 48-h work week for doctors in training was implemented in August 2009 and the possible detrimental effect of this on training is now being explored [14–16]. In the United States the current 80-h work week has made experienced surgeons wonder whether the new duty hour restrictions provide sufficient time to reach the level of proficiency necessary to support the safe, independent practice of medicine [17]. This study provided a possible model of surgical clinical training with tight workhour restrictions. From an advantage point of view, the study is prospective and randomised. Few educational studies in a clinical setting are randomised. This may be due to problems with blinding as the awareness of a study being conducted may cause changes in work-planning. Therefore, the study was focused on one specific procedure, limited in time, and trainees with same employment start date were randomised in identical groups. Another advantage of the study was the blinded rating of the video-recorded performance. The most experienced trainees were not enrolled, as they would not receive supervision. Supervision was provided by the local consultants as it would have been otherwise. Using specially trained supervisors instead of local consultants may have further improved results though impairing the generalizability of the study to the clinical training context. Regarding disadvantages, the study is small in participant number, although all new trainees were considered for participation. However, results were statistically significant despite the small number indicating large effect sizes. The decrease of operative time is clear, though not statistically significant. This may be due to Type 2 error. One possible confounder was the number of performed procedures prior to intervention. We tested this in a regression analysis, which showed no statistically significant difference between the groups at study entry.
Conclusions This module-based training model in Lichtenstein hernia repair in a daily clinical setting showed significant and sustained improvement in technical skills as compared to current practice. The study showed high participant
123
contentment, and as the study was conducted in a clinical setting it would be easy to apply. The model will, moreover, probably be applicable in other surgical training procedures. Acknowledgments We thank the doctors and the staff involved in skills lab course, participating trainees, supervisors and patients accepting the video recording of the hernia repairs. Furthermore, this study owes a special thanks to the huge work of the consultants involved in the rating of the recordings. A special thanks to the Tryg foundation that sponsored the skills lab courses. Thank you to the public funders of the research: Aarhus University, Aalborg University Hospital and The Central Denmark Region. Conflict of interest Dr. C Carlsen declares no conflict of interest.Dr. K Lindorff-Larsen declares no conflict of interest.Dr. P FunchJensen declares no conflict of interest.Dr. L Lund declares no conflict of interest.Dr. L Konge declares no conflict of interest.Dr. P Charles declares no conflict of interest.
References 1. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR (2009) Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev (1):CD006575. doi:10.1002/ 14651858.CD006575 2. Eraut M (2004) Informal learning in the workplace. Stud Cont Edu 26:274–293 3. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79:S70–S81 4. Bell RH Jr, Biester TW, Tabuenca A, Rhodes RS, Cofer JB, Britt LD, Lewis FR Jr (2009) Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg 249:719–724 5. Davies BW, Campbell WB (1995) Inguinal hernia repair: see one, do one, teach one? Ann R Coll Surg Engl 77:299–301 6. Carlsen CG, Lindoff-Larsen K, Funch-Jensen P, Lund L, Charles P, Konge L (2013) Reliable and valid assessment of Lichtenstein hernia repair skills. Hernia 18(4):543–548 7. Department of Health U (2006) Good doctors, safer patients. A report by the Chief Medical Officer. No. 276071. http://www.dh. gov.uk/publications 8. Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R (2006) Teaching surgical skills: what kind of practice makes perfect: a randomized, controlled trial. Ann Surg 244:400–409 9. Verdaasdonk EG, Stassen LP, van Wijk RP, Dankelman J (2007) The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc 21:214–219 10. Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, Stone M (2004) The, ‘‘cost’’ of operative training for surgical residents. Arch Surg 139:366–369 11. Koperna T (2004) How long do we need teaching in the operating room? The true costs of achieving surgical routine. Langenbecks Arch Surg 389:204–208 12. Wilkiemeyer M, Pappas TN, Giobbie-Hurder A, Itani KM, Jonasson O, Neumayer LA (2005) Does resident post graduate year influence the outcomes of inguinal hernia repair? Ann Surg 241:879–882 13. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical
Hernia skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442 14. Blencowe NS, Parsons BA, Hollowood AD (2011) Effects of changing work patterns on general surgical training over the last decade. Postgrad Med J 87:795–799 15. Canter R (2011) Impact of reduced working time on surgical training in the United Kingdom and Ireland. Surgeon 9(Suppl 1):S6–S7
16. Eaton L (2010) Working time directive puts patients’ lives at risk, surgeons claim. BMJ 341:c4212 17. Pellegrini VD Jr (2012) Perspective: ten thousand hours to patient safety, sooner or later. Acad Med 87:164–167
123
