ORIGINAL REPORTS
Virtual Reality Robotic Surgical Simulation: An Analysis of Gynecology Trainees Sangini S. Sheth, MD,* Amanda N. Fader, MD,*,† Ana I. Tergas, MD,*,† Christina L. Kushnir, MD,*,† and Isabel C. Green, MD* *
Department of Gynecology and Obstetrics, Johns Hopkins Medical Institutions, Baltimore, Maryland; and Department of Obstetrics and Gynecology, Section of Gynecologic Oncology, Greater Baltimore Medical Center, Baltimore, Maryland †
STUDY OBJECTIVE: To analyze the learning curves of
gynecology trainees on several virtual reality da Vinci Skills Simulator exercises.
the 10th repetition. The mean number of repetitions required to achieve performance plateau ranged from 6.4 to 9.3.
DESIGN: Prospective cohort pilot study.
CONCLUSION: Virtual reality robotic simulation improves
SETTING: Academic hospital-based gynecology training
program. PARTICIPANTS: Novice robotic surgeons from a gynecology training program. METHODS: Novice robotic surgeons from an academic
gynecology training program completed 10 repetitions of 4 exercises on the da Vinci Skills Simulator: matchboard, ring and rail, suture sponge, and energy switching. Performance metrics measured included time to completion, economy of instrument movement, excessive force, collisions, master workspace range, missed targets, misapplied energy, critical errors, and overall score. Statistical analyses were conducted to define the learning curve for trainees and the optimal number of repetitions for each exercise.
ability through repetition at all levels of training. Further, a performance plateau may exist during a single training session. Larger studies are needed to further define the most highyield simulator exercises, the ideal number of repetitions, and recommended intervals between training sessions to improve C 2014 Associoperative performance. ( J Surg 71:125-132. J ation of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: robotics, computer simulation, surgical
procedures, surgery
minimally
invasive/education,
gynecologic
COMPETENCIES: Practice-Based Learning and Improve-
ment, Patient Care
RESULTS: A total of 34 participants were enrolled, of
which 9 were medical students, 22 were residents, and 3 were fellows. There was a significant improvement in performance between the 1st and 10th repetitions across multiple metrics for all exercises. Senior trainees performed the suture exercise significantly faster than the junior trainees during the first and last repetitions (p ¼ 0.004 and p ¼ 0.003, respectively). However, the performance gap between seniors and juniors narrowed significantly by Correspondence: Inquiries to Isabel C. Green, MD, Department of Gynecology and Obstetrics, The Johns Hopkins Medical Institutions, 600 North Wolfe Street, Baltimore, MD 21287; e-mail: [email protected] Presentation information: Accepted for an oral plenary presentation at the 60th Annual Clinical Meeting of the American College of Obstetricians and Gynecologists, San Diego, CA; May 7, 2012. Disclosure: Dr. Isabel Green received a travel honorarium from Intuitive Surgical to participate in consensus meetings for the development of a robotics curriculum in gynecology residency training programs (unrelated to this study) in 2010 and 2011.
INTRODUCTION Minimally invasive surgery (MIS) approaches in gynecology are becoming more common and include vaginal, laparoscopic, and robotic surgical methods.1-3 Robotic-assisted hysterectomies are one of the fastest growing and most commonly performed robotic procedures in the United States. In 2010, 110,000 robotic-assisted hysterectomies were performed.4 With the growing application of MIS, MIS techniques are becoming a more common part of gynecology resident surgical exposure.5-8 Based on national statistics, the mean number of cases of laparoscopic hysterectomy among residents grew from 23 in 2008-2009 to 34 in 2010-2011, whereas the mean number of cases of abdominal hysterectomy dropped from 74 to 65 during those years.9,10 Survey results published by Smith et al.11 indicate that 70% of the U.S. obstetrics and gynecology
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2013.06.009
125
residents had participated in a robotics procedure and 44% planned to incorporate robotic surgery into their practice; however, only 3.6% felt equipped to do so without additional training. A separate survey of obstetrics and gynecology residency program directors indicated that the most common method to assess residents’ robotic surgical competency was through operating room performance, but only 28% of respondents believed their current robotic training methods were “effective” or “very effective.”12 Studies in laparoscopic training and simulation have demonstrated that a structured curriculum can lead to improved performance and patient safety in the operating room.13,14 Similarly, in residency programs in which robotic surgery is a component of the resident surgical experience, a structured curriculum is likely to ensure adequate training of the resident surgeons, as well as patient safety and quality health care delivery.15 Robotic surgical education must fit a training environment constrained by restricted resident work hours, finances, and decreasing hysterectomy surgical volume.16,17 The recent advent and refinement of surgical simulators, and specifically virtual reality simulators, contemporize gynecologic surgical education by serving as a sophisticated and efficacious adjunct to traditional surgical training in an increasingly challenged training environment. Although the use of laparoscopic surgical simulators in training has been validated, the incorporation of virtual reality robotic simulation into resident training is in its infancy.18-20 The da Vinci Skills Simulator (Intuitive Surgical Inc., Sunnyvale, CA), which uses Mimic virtual reality training software (Mimic Technologies, Seattle, WA), is a surgical simulation platform for the robotic surgical system. A number of studies in the urologic literature have previously demonstrated the face, content, and construct validity of this virtual reality simulator.15,21,22 Given the similarities in robotic surgical principles between urology and gynecology, these validation studies are generally considered to apply to gynecologic training as well.15 Recently, Tergas et al. reported on the efficacy of both the da Vinci Skills Simulator and a standard da Vinci robotic simulation platform with regard to resident performance on a suture exercise. Additionally, the authors postulated that the da Vinci Skills Simulator would allow for greater efficiency, trainee autonomy, and improved measurement of proficiency assessments compared with a standard surgical simulator.23 We conducted this study to further understand and describe the learning curve associated with virtual reality robotic simulation training to inform future robotic surgical curricula. The study objectives were to describe the learning curve among junior and senior level novice surgeons utilizing the da Vinci Skills Simulator with Mimic virtual reality training software (MdVT), to compare performance metrics by training level and to define a performance plateau on each of the 4 MdVT exercises. 126
METHODS Participants This was a multi-institution, institutional review board– approved study conducted at 2 affiliated academic institutions: Greater Baltimore Medical Center and Johns Hopkins Hospital, both in Baltimore, MD. Medical students and gynecology trainees (postgraduate years 1-6) who had participated in 10 or fewer robotic-assisted surgeries, and therefore considered novice robotic surgeons, were invited to attend 2 one-week robotic surgery training symposia held in 2011, which utilized the da Vinci Skills Simulator. Medical students and residents of postgraduate years 1-2 were considered to be junior level trainees whereas fellows and residents of postgraduate years 3-4 were considered as senior level trainees.
Training Platform The da Vinci Skills Simulator is a virtual reality simulation system developed as a collaborative effort by Mimic Technologies and Intuitive Surgical, Inc. The MdVT uses Mimic’s virtual reality surgical simulation platform and contains a variety of exercises specifically designed to give users the opportunity to improve their proficiency with the da Vinci surgeon console controls and basic surgical skills. Previously published da Vinci Skills Simulator validation studies have included exercises utilized in this study: matchboard, ring and rail, suture sponge, and energy switching.21,24,25 These exercises are organized into system training and skills training modules: EndoWrist manipulation, camera and clutching, fourth arm integration, system settings and console overview, needle control and driving, and energy and dissection. The portable case, or “backpack,” which measures 57.2 cm 60.3 cm, attaches directly on the back of the da Vinci Surgical System surgeon console, so that the console can be used for virtual reality training without the need for the patient-side cart or instruments (Fig. 1). No additional system components are required.
Training Sessions The MdVT training session started with a brief orientation to the robotic system provided by the study team to familiarize novice surgical trainees to the robotic console and its operation. The trainees were thereafter able to proceed autonomously during the exercises. The MdVT training symposium consisted of 4 exercises: matchboard 1 (MB), ring and rail 1 (RR), suture sponge 1 (SS), and energy switching 1 (ES) (Fig. 2). All participants were asked to complete 10 repetitions of at least 1 exercise in the available time. Participants performing fewer than 10 repetitions of an exercise were excluded from analysis of that
Journal of Surgical Education Volume 71/Number 1 January/February 2014
After completion of each repetition, the trainee was given a detailed performance report generated by the MdVT (Fig. 3). Measured metrics included the following: time to complete exercise, economy of instrument motion (EOM) and collisions, excessive instrument force, instruments out of view, master workspace range, missed targets, critical errors, and overall score. These metrics were used to provide feedback to the participants by the proctoring surgeon between repetitions. Performance plateau on each exercise was defined as the number of repetitions performed before having a ≤3 percentage point change in overall score between each of 3 serial repetitions. If a plateau was not reached, the trainee was assigned a plateau point of 11 repetitions, indicating that at least 11 repetitions would have been required to reach a plateau. Statistics Performance metrics comparing each trainee’s first and last repetitions were analyzed with the Wilcoxon signed-rank test. The Student t test was used to compare performance metrics between novice and experienced trainees. A p value of less than 0.05 was considered statistically significant for all analyses. Statistical analysis was performed using Stata 12.1 statistical software (StataCorp, College Station, TX). FIGURE 1. da Vinci skills simulator shown attached to da Vinci surgical Si console.26
exercise so as to not skew interpretation of mean performance metrics. These 4 exercises represent the diverse set of skills required for the performance of robotic surgery. The MB exercise involves placement of various 3-dimensional wooden letter and number blocks onto a board with corresponding pattern cutouts. This exercise reinforces surgical multitasking, use of both hands, EndoWrist manipulation, and camera and clutching skills. In the RR exercise, the trainee moves a ring along a twisted rod and utilizes skills in camera and clutching as well as EndoWrist dexterity of both hands. The SS exercise involves driving a virtual needle through precise entrance and exit targets on a virtual sponge. The exercise requires the participant to drive the needle at many different angles and with their left and right hands. Finally, the ES exercise requires the trainee to accurately grasp vessel targets and appropriately apply monopolar and bipolar energy.
RESULTS A total of 34 trainees participated in the symposia (Table 1). Among the trainees, there were 9 medical students, 22 residents, and 3 fellows; 20 participants were considered to be junior level trainees and 14 senior level trainees. After excluding those that did not complete 10 repetitions of an exercise, the RR, SS, and ES exercises were completed by 20 trainees with an equal balance of junior and senior level participants. The MB exercise was completed by 18 participants, 10 were senior level trainees. Participants completed less than 10 repetitions because of personal time constraints or limited uninterrupted availability of the da Vinci robotic system. Table 2 shows the difference in mean performance metrics between the first and last repetitions among trainees for each exercise. Time to completion significantly improved between the first and last repetitions across all exercises, with the completion time decreasing by at least
FIGURE 2. Examples of the MdVT training symposium exercises.27 Journal of Surgical Education Volume 71/Number 1 January/February 2014
127
FIGURE 3. Mimic MScore performance report.27
50% for RR, SS, and ES. Similarly, economy of motion, critical errors, and the overall score significantly improved for all exercises. Missed targets in the suturing exercise and misapplied energy in the energy switching exercise were significantly lower by the 10th repetition. Additionally, trainees showed significant improvement in reducing collisions and master workspace range in the suturing exercise, although similar improvements were not seen with the other exercises. Trainees did not show improvement in use of excessive force. The mean repetitions for performance plateau ranged from 6.4 repetitions for the RR exercise to 9.3 repetitions for the MB exercise (Table 3). There was no significant difference in mean repetitions to reach performance plateau between junior and senior level trainees (Table 4). Table 5 compares mean performance parameters at the 1st and 10th repetitions by training level for the suture sponge exercise. A significant difference among training level was seen only in the suturing exercise. Senior level trainees completed the first repetition of the suturing exercise significantly faster than junior trainees. Although the difference remained statistically significant by the 10th repetition, it had narrowed considerably (167 s difference vs 37 s difference). A similar performance pattern for the 1st and 10th repetitions was seen in the EOM and overall score parameters, although it had no statistical significance. For the remaining exercises (data not shown), there was no significant difference between junior and senior level trainees in the first or last repetition. For RR, senior novice surgeons performed the exercise faster, more efficiently, and with higher overall scores. Notably, the difference between the training levels narrowed between the 1st and 10th 128
repetitions. In the MB exercise, junior novice surgeons completed the exercise more quickly, but with a lower overall score on the 1st repetition, and ultimately outperformed the senior novice surgeons by the 10th repetition. In the ES exercise, senior novice surgeons performed the exercise faster, more efficiently and with higher overall scores. Similar to RR, the difference in performance in ES narrowed from the first to last repetition. Senior trainees misapplied energy more often than their junior colleagues in the first repetition, but their accuracy improved more steeply over the additional repetitions (PGY o 2: 6.6 vs 3.8 s, p ¼ 0.01; and PGY 4 3: 10 vs 3.5 s between first and last trials, p ¼ 0.006).
TABLE 1. Participant Characteristics
Number of participants Gender, female Age, mean (y) Training level Medical student PGY 1 PGY 2 PGY 3 PGY 4 Fellow (PGY 5) Fellow (PGY 6) Exercises completed Matchboard (MB) Ring and rail (RR) Suture sponge (SS) Energy switching (ES)
Junior
Senior
20 (58%) 12 26.9
14 (42%) 11 30.7
9 8 3 – – – –
– – – 6 5 2 1
8 10 10 10
10 10 10 10
Journal of Surgical Education Volume 71/Number 1 January/February 2014
o0.001 84 (86, [44-99]) 86 (87, [6999]) 72 (72, [56-90])
o0.001
96 (96, [92-100])
– – – – – –
In the absence of postgraduate minimally invasive fellowship education, the upcoming robotically assisted laparoscopic surgeons will require high case volumes and exposure to a comprehensive residency curriculum that complements existing conventional laparoscopic surgical training and ensures proficiency in these surgical modalities. The incorporation of surgical simulation into residency programs has become particularly important in training for conventional laparoscopic and robotic-assisted procedures, which are often complex and leave little room for error. A curriculum utilizing a validated virtual reality robotic surgery simulator may minimize the steep learning curve associated with many of these complex procedures and thus enable more efficient learning and better patient outcomes. In this pilot study, we describe the learning curve of gynecologic surgical trainees on a virtual reality robotic simulator platform, the da Vinci Skills Simulator (MdVT), for a variety of robotic console exercises. Training on the MdVT was associated with improvement in multiple objective console performance metrics that are essential to the successful and safe performance of live surgery, including economy of motion, missed targets, and time to exercise completion. The da Vinci Skills Simulator has demonstrated face, content, construct, and concurrent validity in nongynecologic studies.15,21,24,25 However, the learning curve and ideal simulation exercises for gynecology residents have not been well defined. There are 40 console simulation modules available on the MdVT, and 4 exercises were specifically chosen in the current study that emphasized multitasking and represented a range of console skills that are critical for the performance of gynecologic procedures (clutching, camera movement, energy capabilities, and suturing). Although this is a pilot study, the learning curve data and selection of appropriate modules for gynecology residents would help inform the development of a formal, comprehensive robotics surgical curriculum for this cohort of trainees. An analysis of the learning curve data revealed that mean performance plateaus occurred with fewer than 10 repetitions for all console exercises regardless of trainee level. There was a significant improvement in performance between the 1st and 10th repetitions across multiple metrics for all training modules. As the performance plateau for the various exercises was reached by 6-9 repetitions, it is unlikely that an additional benefit would be gained from
TABLE 3. Mean Repetitions For Performance Plateau Overall score (%)
– Missed targets
10.4 (10.1, [8.2-13.9]) 2.0 (2, [1-3]) Master workspace range (cm) Critical errors
0 Excess force
Misapplied energy (s)
– –
0.009
0.459
0.071
175.4 (180.3, o0.001 148.9 (122.3, 75.6 (69.7, o0.001 [115.7-229]) [78.6-319.6]) [56.1-110.7]) 196.8 (198.7, o0.001 192.5 (185, 154.6 (152.5, o0.001 [158.2-229.7]) [138.2-298]) [131.1-192.4]) 0.95 (1, [0-4]) o0.001 0.1 (0, [0-1]) 0.05 (0, 0.564 [0-1]) 0.63 0.28 (0, [0-4]) 0.289 2.0 (0, [00.35 (0, 0.341 (0, [0-6.5]) 25.5]) [0-5.5]) 4.7 (4.5, 4.1 (4.2, o0.001 15.0 (15.6, 14.1 (13.4, 0.179 [4.1-6.3]) [3.4-4.8]) [6.8-21.2]) [7.6-22.5]) 2.45 (3, [1-4]) 0.25 (0, [0-1]) o0.001 2.45 (2, [0-5]) 0.9 (1, o0.001 [0-3]) 11.4 (11, 4.5 (4, [0-14]) o0.001 – – – [0-21]) – – – 8.4 (5.2, [3.33.6 (3.7, o0.001 30]) [1.2-5.8]) 62 (62, [3394 (94, [85- o0.001 69 (73, [2889 (91, o0.001 88]) 100]) 83]) [64-97]) o0.001 387.1 (397.2, [171.9-598.7] 0.001 339.4 (318.7, [198.2-543.4]) 0.094 7.1 (7, [0-22])
24.0 (21.8, [14.1-45]) 40.8 (42.1, [25.5-59.9]) 0.15 (0, [0-1]) 0.6 (0, [0-4]) 7.7 (7.6, [5.7-11.3]) 0
p p
55.3 (48.2, [22.9-178]) 58.5 (57.9, [27.9-98.7]) 0.5 (0, [0-3]) 0 0 2.1 (0.3, [0-11]) 10.3 (10.5, [70.845 7.8 13.8]) (7.8, [0-10.7]) 0.78 (1, [0-2]) o0.001 0.55 (0, [0-3]) – – –
p Last Trial (Median, [Range]) First Trial (Median, [Range]) Last Trial (Median, [Range]) First Trial (Median, [Range]) Last Trial (Median, [Range]) First Trial (Median, [Range])
Time to comple193.5 (189.7, 137.1 (116.1, 0.004 tion (s) [122.4-368.7]) [70.5-387.5]) Economy of motion (s) 339.6 (330, 238.1 (238.9, o0.001 [224.8-534.1]) [174.5-344.8]) Collisions 0.6 (0, [0-3]) 0.7 (1.0, [0-2]) 0.784
p Last Trial (Median, [Range]) First Trial (Median, [Range])
Energy Switching (N ¼ 20) Suture Sponge (N ¼ 20) Ring and Rail (N ¼ 20) Matchboard (N ¼ 18)
TABLE 2. Difference in Mean Performance Metrics Between 1st and 10th Repetitions Across 4 MdVT Exercises
DISCUSSION
Plateau (Median [Range]) MB (N ¼ 18) RR (N ¼ 20) SS (N ¼ 20) ES (N ¼ 20)
Journal of Surgical Education Volume 71/Number 1 January/February 2014
9.3 6.4 9.1 8.2
(11, [5-11]) (6, [1-11]) (11, [2-11]) (8, [2-11])
SD 2.4 3.6 3 2.7 129
TABLE 4. Mean Repetitions For Performance Plateau by Training Level
MB RR SS ES
PGY ≤ 2 (Median, [Range])
PGY ≥ 3 (Median, [Range])
p
9.6 (11, [5-11]) 6.8 (6, [1-11]) 8.8 (11, [4-11]) 7.8 (7.5, [2-11])
9.1 (11, [6-11]) 6 (5.5, [1-11]) 9.3 (11, [2-11]) 8.5 (8.5, [5-11])
0.665 0.630 0.718 0.579
having trainees perform more than 10 repetitions of a given exercise in a training session. Further, Tergas et al. suggest that uninterrupted training on the MdVT for an hour or more may lead to temporary eye and neck strain as well as headaches by users of the MdVT simulator.23 This supports the need to define the most safe and efficient use of the MdVT in training sessions. Prior learning curve studies for minimally invasive gynecologic procedures have involved performance on live surgical procedures,28,29 laparoscopic virtual reality simulators,30 and the da Vinci Surgical System dry laboratory platform.31 To our knowledge, our study is one of the first to define a learning curve amongst gynecology trainees for the specific MdVT modules studied. Unlike other surgical learning curve studies that focus primarily on time to completion of a surgical procedure or task, the current study analyzed trainee performance on several simulation metrics, including economy of motion, occurrence of critical errors, missed targets, misapplied energy, and overall performance over the course of 10 repetitions per exercise. This level of evaluation may help surgical faculty tailor their instruction and selection of console exercises to the needs of individual trainees by providing them with more specific, objective performance feedback than was previously possible. Performance improved in all parameters with the exception of workspace range (except in SS) and use of excessive force. These parameters may require direct proctoring by an experienced surgeon during the exercise to yield improvement. Additionally, our study identified areas that may serve as patient safety parameters. Performance metrics, such as missed targets for the suturing exercise and misapplied energy for the energy switching exercise, in which 10 repetitions resulted in significant improvement, may deserve special attention in the development of a surgical curriculum. Such parameters could be considered patient safety parameters as
inaccuracy in suture placement or bipolar/monopolar energy application could lead to surgical complications. Trainees should be required to gain competence in these parameters before performing live surgery. A limitation of the study was the small number of trainees that performed each exercise. Comparisons between junior and senior level trainees may also have been significantly affected by having only 10 trainees of each level for each exercise. Several participants were excluded from analysis of an exercise because they did not complete 10 repetitions. Future studies would benefit from enrolling a larger group of trainees and blocking out time in their schedule dedicated to simulation training. Additionally, voluntary recruitment may have affected the results as those trainees that agreed to participate may have been more motivated to improve their performance on the MdVT. The inclusion of a single gynecology training program in this study also limits generalizability of the results. Study strengths include assessment of novice robotic surgeons at a range of training levels and that this is one of the first studies to define a learning curve and potential curriculum for the MdVT amongst gynecology residents. As training programs develop robotic surgical skills curricula, it is important to consider inclusion of trainees at all levels and to identify skill-level appropriate exercises and proficiency goals. This study helps identify how senior and junior level trainees may differ in their baseline skills and the general slope of the learning curves that may be expected for both groups. Senior trainees performed the suturing exercise significantly faster than the junior trainees on the 1st repetition, and although this finding remained by the 10th repetition, the narrowing of the performance gap may be an indication of a steeper learning curve for junior level trainees. The significant improvement observed in several of the exercises studies indicates that they may serve as high-yield exercises that can help establish a baseline threshold that must be attained before trainees can participate in live robotic surgical procedures.
CONCLUSION This study demonstrates that the virtual reality robotic da Vinci Skills Simulator improves technical performance for
TABLE 5. Mean Performance Parameters by Training Level, Suture Sponge PGY ≤ 2 (N ¼ 10) (Median, [Range])
PGY ≥ 3 (N ¼ 10) (Median, [Range])
First trial time (s) First trial EOM (s) First trial overall score (%)
470.5 (499.9, [264.6-598.7]) 377.3 (353.2, [268.7543.4]) 55.9 (56, [33.0-78.0])
303.7 (286.5, [171.9-467.4]) 301.4 (304.05, [198.2-478.7]) 68.4 (65.0, [43.0-88.0])
Last trial time (s) Last trial EOM (s) Last trial overall score (%)
193.9 (194.7, [161.2-229.0]) 203.3 (201.2, [180.7-229.7]) 94.7 (96.0, [85.0-100.0])
156.8 (162.5, [115.7-187.6]) 190.2 (193.7, [158.2-217.4]) 92.8 (92.5, [86.0-100.0])
130
Difference
p
166.8 75.9 12.5
0.004 0.073 0.100
37.1 13.1 1.9
0.003 0.092 0.420
Journal of Surgical Education Volume 71/Number 1 January/February 2014
gynecology trainees at all levels across multiple surgical exercises. Junior level trainees may experience a steeper learning curve on certain robotic simulation exercises, whereas in other, possibly more surgically sophisticated, exercises, senior trainees demonstrated a steeper learning curve. In most cases, performance plateau is reached after 6 to 9 repetitions of a particular MdVT exercise. Such data can be used to help determine proficiency before live surgery, as well as to guide training and curriculum design. Future studies are needed to further inform robotic surgical curriculum including randomized trials investigating the utility of the da Vinci Skills Simulator in preparation for live surgery and to assess the effect of virtual reality simulation exercises on patient safety and outcomes. Larger studies involving multiple training institutions are also needed to identify high-yield simulator exercises, effect of direct proctoring vs. autonomous practice, and recommended intervals between training sessions to establish skills retention.
8. Weinberg L, Rao S, Escobar PF. Robotic surgery in
gynecology: an updated systematic review. Obstet Gynecol Int. 2011;2011:852061. 9. Accreditation council of graduate medical education.
Obstetrics and gynecology case logs: National data report 2008-2009. Available at: 〈http://www.acgme. org/residentdatacollection/documentation/statistical_ reports.asp〉. Accessed 14.04.12. 10. Accreditation council of graduate medical education.
Obstetrics and gynecology case logs: National data report 2010-2011. Available at: 〈http://www.acgme. org/residentdatacollection/documentation/statistical_ reports.asp〉. Accessed 14.04.12. 11. Smith AL, Schneider KM, Berens PD. Survey of
obstetrics and gynecology residents’ training and opinions on robotic surgery. J Robot Surg. 2010;4:23-27. 12. Gobern JM, Novak CM, Lockrow EG. Survey of
robotic surgery training in obstetrics and gynecology residency. J Minim Invasive Gynecol. 2011;18:755-760.
REFERENCES 1. Johnson N, Barlow D, Lethaby A, Tavender E, Curr
E, Garry R. Surgical approach to hysterectomy for benign gynaecological disease. Cochrane Database Syst Rev. 2006; CD003677. 2. Garry R, Fountain J, Mason S, et al. The evaluate study:
two parallel randomised trials, one comparing laparoscopic with abdominal hysterectomy, the other comparing laparoscopic with vaginal hysterectomy. Br Med J. 2004;328:129. 3. Choosing the route of hysterectomy for benign disease.
ACOG committee opinion no. 444. American College of Obstetricians and Gynecologists. Obstet Gynecol. 2009;114:1156-1158. 4. Intuitive surgical. Annual report 2010. Available at: 〈http://
investor.intuitivesurgical.com/phoenix.zhtml?c=122359&p =irol-reportsAnnual〉. Accessed on 14.04.12. 5. Tu FF, Beaumont JL, Senapati S, Gordon TEJ. Route
of hysterectomy influence and teaching hospital status. Obstet Gynecol. 2009;114(1):73-78. 6. Matthews CA, Reid N, Ramakrishnan V, Hull K,
Cohen S. Evaluation of the introduction of robotic technology on route of hysterectomy and complications in the first year of use. Am J Obstet Gynecol. 2010;203(5):[499 e1-499 e5]. 7. Paley PJ, Veljovich DS, Shah CA, et al. Surgical
outcomes in gynecologic oncology in the era of robotics: analysis of first 1000 cases. Am J Obstet Gynecol. 2011;204(6):[551.e1-551.e9].
13. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463. 14. Hyltander A, Liljegren E, Rhodin PH, Lonroth H.
The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16:1324-1328. 15. Schreuder HWR, Wolswjik R, Zweemer RP, Schijven
MP, Verheijen RHM. Training and learning robotic surgery, time for a more structured approach: a systematic review. BJOG. 2012;119:137-149. 16. Lepine LA, Hillis SD, Marchbanks PA, et al. Hyster-
ectomy surveillance—United States, 1980-1993. MMWR CDC Surveill Summ. 1997;46:1-15. 17. Whiteman MK, Hillis SD, Jamieson DJ, Morrow B,
Podgornik MN, Brett KM, Marchbanks PA. Inpatient hysterectomy surveillance in the United States, 2000– 2004. Am J Obstet Gynecol. 2008;198:34.e1–7. 18. Burden C, Oestergaard J, Larsen CR. Integration of
laparoscopic virtual-reality simulation into gynaecology training. BJOG. 2011;118:5-10. 19. Crochet P, Aggarwal R, Dubb SS, et al. Deliberate
practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg. 2011;253:1216-1222. 20. Loukas C, Nikiteas N, Kanakis M, Georgiou E. The
contribution of simulation training in enhancing key
Journal of Surgical Education Volume 71/Number 1 January/February 2014
131
components of laparoscopic competence. Am Surg. 2011;77:708-715. 21. Hung AJ, Zehnder P, Patil MB, et al. Content and
26. da Vinci surgical simulator brochure. Available at:
〈http://www.intuitivesurgical.com/products/skills_si mulator/〉. Accessed 29.04.12.
construct validity of a novel robotic surgery simulator. J Urol. 2011;186:1019-1024.
27. Mimic technologies. Available at: 〈http://www.mimicsimu
22. Lendvay TS, Casale P, Sweet R, Peters C. Initial
28. Song T, Kim TJ, Lee YY, et al. What is the learning
validation of a virtual-reality robotic simulator. J Robot Surg. 2008;2:145-149. 23. Tergas AI, Sheth SS, Green IC, Giuntoli RL, Winder
AD, Fader AN. A pilot study of surgical training utilizing a virtual robotic surgery simulator. JSLS, In press.
lation.com/products/〉. Accessed 29.04.12. curve for the single-port access laparoscopic-assisted vaginal hysterectomy? Eur J Obstet Gynecol Reprod Biol. 2011;158:93-96. 29. Ghomi A, Littman P, Prasad A, Einarsson JI. Assessing
24. Kelly DC, Margules AC, Kundavaram CR, et al. Face,
the learning curve for laparoscopic supracervical hysterectomy. JSLS. 2007;11:190-194.
content, and construct validation of the da Vinci skills simulator. Urology. 2012;79:1068-1072.
30. Gor M, McCloy R, Stone R, Smith A. Virtual reality
25. Finnegan KT, Meraney AM, Staff I, Shichman SJ. da
Vinci skills simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance. Urology. 2012;80:330-335.
132
laparoscopic simulator for assessment in gynaecology. BJOG. 2003;110:181-187. 31. Geller EJ, Schuler KM, Boggess JF. Robotic surgical
training program in gynecology: how to train residents and fellows. J Minim Invasive Gynecol. 2011;18:224-229.
Journal of Surgical Education Volume 71/Number 1 January/February 2014
Virtual Reality Robotic Surgical Simulation: An Analysis of Gynecology Trainees Sangini S. Sheth, MD,* Amanda N. Fader, MD,*,† Ana I. Tergas, MD,*,† Christina L. Kushnir, MD,*,† and Isabel C. Green, MD* *
Department of Gynecology and Obstetrics, Johns Hopkins Medical Institutions, Baltimore, Maryland; and Department of Obstetrics and Gynecology, Section of Gynecologic Oncology, Greater Baltimore Medical Center, Baltimore, Maryland †
STUDY OBJECTIVE: To analyze the learning curves of
gynecology trainees on several virtual reality da Vinci Skills Simulator exercises.
the 10th repetition. The mean number of repetitions required to achieve performance plateau ranged from 6.4 to 9.3.
DESIGN: Prospective cohort pilot study.
CONCLUSION: Virtual reality robotic simulation improves
SETTING: Academic hospital-based gynecology training
program. PARTICIPANTS: Novice robotic surgeons from a gynecology training program. METHODS: Novice robotic surgeons from an academic
gynecology training program completed 10 repetitions of 4 exercises on the da Vinci Skills Simulator: matchboard, ring and rail, suture sponge, and energy switching. Performance metrics measured included time to completion, economy of instrument movement, excessive force, collisions, master workspace range, missed targets, misapplied energy, critical errors, and overall score. Statistical analyses were conducted to define the learning curve for trainees and the optimal number of repetitions for each exercise.
ability through repetition at all levels of training. Further, a performance plateau may exist during a single training session. Larger studies are needed to further define the most highyield simulator exercises, the ideal number of repetitions, and recommended intervals between training sessions to improve C 2014 Associoperative performance. ( J Surg 71:125-132. J ation of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: robotics, computer simulation, surgical
procedures, surgery
minimally
invasive/education,
gynecologic
COMPETENCIES: Practice-Based Learning and Improve-
ment, Patient Care
RESULTS: A total of 34 participants were enrolled, of
which 9 were medical students, 22 were residents, and 3 were fellows. There was a significant improvement in performance between the 1st and 10th repetitions across multiple metrics for all exercises. Senior trainees performed the suture exercise significantly faster than the junior trainees during the first and last repetitions (p ¼ 0.004 and p ¼ 0.003, respectively). However, the performance gap between seniors and juniors narrowed significantly by Correspondence: Inquiries to Isabel C. Green, MD, Department of Gynecology and Obstetrics, The Johns Hopkins Medical Institutions, 600 North Wolfe Street, Baltimore, MD 21287; e-mail: [email protected] Presentation information: Accepted for an oral plenary presentation at the 60th Annual Clinical Meeting of the American College of Obstetricians and Gynecologists, San Diego, CA; May 7, 2012. Disclosure: Dr. Isabel Green received a travel honorarium from Intuitive Surgical to participate in consensus meetings for the development of a robotics curriculum in gynecology residency training programs (unrelated to this study) in 2010 and 2011.
INTRODUCTION Minimally invasive surgery (MIS) approaches in gynecology are becoming more common and include vaginal, laparoscopic, and robotic surgical methods.1-3 Robotic-assisted hysterectomies are one of the fastest growing and most commonly performed robotic procedures in the United States. In 2010, 110,000 robotic-assisted hysterectomies were performed.4 With the growing application of MIS, MIS techniques are becoming a more common part of gynecology resident surgical exposure.5-8 Based on national statistics, the mean number of cases of laparoscopic hysterectomy among residents grew from 23 in 2008-2009 to 34 in 2010-2011, whereas the mean number of cases of abdominal hysterectomy dropped from 74 to 65 during those years.9,10 Survey results published by Smith et al.11 indicate that 70% of the U.S. obstetrics and gynecology
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2013.06.009
125
residents had participated in a robotics procedure and 44% planned to incorporate robotic surgery into their practice; however, only 3.6% felt equipped to do so without additional training. A separate survey of obstetrics and gynecology residency program directors indicated that the most common method to assess residents’ robotic surgical competency was through operating room performance, but only 28% of respondents believed their current robotic training methods were “effective” or “very effective.”12 Studies in laparoscopic training and simulation have demonstrated that a structured curriculum can lead to improved performance and patient safety in the operating room.13,14 Similarly, in residency programs in which robotic surgery is a component of the resident surgical experience, a structured curriculum is likely to ensure adequate training of the resident surgeons, as well as patient safety and quality health care delivery.15 Robotic surgical education must fit a training environment constrained by restricted resident work hours, finances, and decreasing hysterectomy surgical volume.16,17 The recent advent and refinement of surgical simulators, and specifically virtual reality simulators, contemporize gynecologic surgical education by serving as a sophisticated and efficacious adjunct to traditional surgical training in an increasingly challenged training environment. Although the use of laparoscopic surgical simulators in training has been validated, the incorporation of virtual reality robotic simulation into resident training is in its infancy.18-20 The da Vinci Skills Simulator (Intuitive Surgical Inc., Sunnyvale, CA), which uses Mimic virtual reality training software (Mimic Technologies, Seattle, WA), is a surgical simulation platform for the robotic surgical system. A number of studies in the urologic literature have previously demonstrated the face, content, and construct validity of this virtual reality simulator.15,21,22 Given the similarities in robotic surgical principles between urology and gynecology, these validation studies are generally considered to apply to gynecologic training as well.15 Recently, Tergas et al. reported on the efficacy of both the da Vinci Skills Simulator and a standard da Vinci robotic simulation platform with regard to resident performance on a suture exercise. Additionally, the authors postulated that the da Vinci Skills Simulator would allow for greater efficiency, trainee autonomy, and improved measurement of proficiency assessments compared with a standard surgical simulator.23 We conducted this study to further understand and describe the learning curve associated with virtual reality robotic simulation training to inform future robotic surgical curricula. The study objectives were to describe the learning curve among junior and senior level novice surgeons utilizing the da Vinci Skills Simulator with Mimic virtual reality training software (MdVT), to compare performance metrics by training level and to define a performance plateau on each of the 4 MdVT exercises. 126
METHODS Participants This was a multi-institution, institutional review board– approved study conducted at 2 affiliated academic institutions: Greater Baltimore Medical Center and Johns Hopkins Hospital, both in Baltimore, MD. Medical students and gynecology trainees (postgraduate years 1-6) who had participated in 10 or fewer robotic-assisted surgeries, and therefore considered novice robotic surgeons, were invited to attend 2 one-week robotic surgery training symposia held in 2011, which utilized the da Vinci Skills Simulator. Medical students and residents of postgraduate years 1-2 were considered to be junior level trainees whereas fellows and residents of postgraduate years 3-4 were considered as senior level trainees.
Training Platform The da Vinci Skills Simulator is a virtual reality simulation system developed as a collaborative effort by Mimic Technologies and Intuitive Surgical, Inc. The MdVT uses Mimic’s virtual reality surgical simulation platform and contains a variety of exercises specifically designed to give users the opportunity to improve their proficiency with the da Vinci surgeon console controls and basic surgical skills. Previously published da Vinci Skills Simulator validation studies have included exercises utilized in this study: matchboard, ring and rail, suture sponge, and energy switching.21,24,25 These exercises are organized into system training and skills training modules: EndoWrist manipulation, camera and clutching, fourth arm integration, system settings and console overview, needle control and driving, and energy and dissection. The portable case, or “backpack,” which measures 57.2 cm 60.3 cm, attaches directly on the back of the da Vinci Surgical System surgeon console, so that the console can be used for virtual reality training without the need for the patient-side cart or instruments (Fig. 1). No additional system components are required.
Training Sessions The MdVT training session started with a brief orientation to the robotic system provided by the study team to familiarize novice surgical trainees to the robotic console and its operation. The trainees were thereafter able to proceed autonomously during the exercises. The MdVT training symposium consisted of 4 exercises: matchboard 1 (MB), ring and rail 1 (RR), suture sponge 1 (SS), and energy switching 1 (ES) (Fig. 2). All participants were asked to complete 10 repetitions of at least 1 exercise in the available time. Participants performing fewer than 10 repetitions of an exercise were excluded from analysis of that
Journal of Surgical Education Volume 71/Number 1 January/February 2014
After completion of each repetition, the trainee was given a detailed performance report generated by the MdVT (Fig. 3). Measured metrics included the following: time to complete exercise, economy of instrument motion (EOM) and collisions, excessive instrument force, instruments out of view, master workspace range, missed targets, critical errors, and overall score. These metrics were used to provide feedback to the participants by the proctoring surgeon between repetitions. Performance plateau on each exercise was defined as the number of repetitions performed before having a ≤3 percentage point change in overall score between each of 3 serial repetitions. If a plateau was not reached, the trainee was assigned a plateau point of 11 repetitions, indicating that at least 11 repetitions would have been required to reach a plateau. Statistics Performance metrics comparing each trainee’s first and last repetitions were analyzed with the Wilcoxon signed-rank test. The Student t test was used to compare performance metrics between novice and experienced trainees. A p value of less than 0.05 was considered statistically significant for all analyses. Statistical analysis was performed using Stata 12.1 statistical software (StataCorp, College Station, TX). FIGURE 1. da Vinci skills simulator shown attached to da Vinci surgical Si console.26
exercise so as to not skew interpretation of mean performance metrics. These 4 exercises represent the diverse set of skills required for the performance of robotic surgery. The MB exercise involves placement of various 3-dimensional wooden letter and number blocks onto a board with corresponding pattern cutouts. This exercise reinforces surgical multitasking, use of both hands, EndoWrist manipulation, and camera and clutching skills. In the RR exercise, the trainee moves a ring along a twisted rod and utilizes skills in camera and clutching as well as EndoWrist dexterity of both hands. The SS exercise involves driving a virtual needle through precise entrance and exit targets on a virtual sponge. The exercise requires the participant to drive the needle at many different angles and with their left and right hands. Finally, the ES exercise requires the trainee to accurately grasp vessel targets and appropriately apply monopolar and bipolar energy.
RESULTS A total of 34 trainees participated in the symposia (Table 1). Among the trainees, there were 9 medical students, 22 residents, and 3 fellows; 20 participants were considered to be junior level trainees and 14 senior level trainees. After excluding those that did not complete 10 repetitions of an exercise, the RR, SS, and ES exercises were completed by 20 trainees with an equal balance of junior and senior level participants. The MB exercise was completed by 18 participants, 10 were senior level trainees. Participants completed less than 10 repetitions because of personal time constraints or limited uninterrupted availability of the da Vinci robotic system. Table 2 shows the difference in mean performance metrics between the first and last repetitions among trainees for each exercise. Time to completion significantly improved between the first and last repetitions across all exercises, with the completion time decreasing by at least
FIGURE 2. Examples of the MdVT training symposium exercises.27 Journal of Surgical Education Volume 71/Number 1 January/February 2014
127
FIGURE 3. Mimic MScore performance report.27
50% for RR, SS, and ES. Similarly, economy of motion, critical errors, and the overall score significantly improved for all exercises. Missed targets in the suturing exercise and misapplied energy in the energy switching exercise were significantly lower by the 10th repetition. Additionally, trainees showed significant improvement in reducing collisions and master workspace range in the suturing exercise, although similar improvements were not seen with the other exercises. Trainees did not show improvement in use of excessive force. The mean repetitions for performance plateau ranged from 6.4 repetitions for the RR exercise to 9.3 repetitions for the MB exercise (Table 3). There was no significant difference in mean repetitions to reach performance plateau between junior and senior level trainees (Table 4). Table 5 compares mean performance parameters at the 1st and 10th repetitions by training level for the suture sponge exercise. A significant difference among training level was seen only in the suturing exercise. Senior level trainees completed the first repetition of the suturing exercise significantly faster than junior trainees. Although the difference remained statistically significant by the 10th repetition, it had narrowed considerably (167 s difference vs 37 s difference). A similar performance pattern for the 1st and 10th repetitions was seen in the EOM and overall score parameters, although it had no statistical significance. For the remaining exercises (data not shown), there was no significant difference between junior and senior level trainees in the first or last repetition. For RR, senior novice surgeons performed the exercise faster, more efficiently, and with higher overall scores. Notably, the difference between the training levels narrowed between the 1st and 10th 128
repetitions. In the MB exercise, junior novice surgeons completed the exercise more quickly, but with a lower overall score on the 1st repetition, and ultimately outperformed the senior novice surgeons by the 10th repetition. In the ES exercise, senior novice surgeons performed the exercise faster, more efficiently and with higher overall scores. Similar to RR, the difference in performance in ES narrowed from the first to last repetition. Senior trainees misapplied energy more often than their junior colleagues in the first repetition, but their accuracy improved more steeply over the additional repetitions (PGY o 2: 6.6 vs 3.8 s, p ¼ 0.01; and PGY 4 3: 10 vs 3.5 s between first and last trials, p ¼ 0.006).
TABLE 1. Participant Characteristics
Number of participants Gender, female Age, mean (y) Training level Medical student PGY 1 PGY 2 PGY 3 PGY 4 Fellow (PGY 5) Fellow (PGY 6) Exercises completed Matchboard (MB) Ring and rail (RR) Suture sponge (SS) Energy switching (ES)
Junior
Senior
20 (58%) 12 26.9
14 (42%) 11 30.7
9 8 3 – – – –
– – – 6 5 2 1
8 10 10 10
10 10 10 10
Journal of Surgical Education Volume 71/Number 1 January/February 2014
o0.001 84 (86, [44-99]) 86 (87, [6999]) 72 (72, [56-90])
o0.001
96 (96, [92-100])
– – – – – –
In the absence of postgraduate minimally invasive fellowship education, the upcoming robotically assisted laparoscopic surgeons will require high case volumes and exposure to a comprehensive residency curriculum that complements existing conventional laparoscopic surgical training and ensures proficiency in these surgical modalities. The incorporation of surgical simulation into residency programs has become particularly important in training for conventional laparoscopic and robotic-assisted procedures, which are often complex and leave little room for error. A curriculum utilizing a validated virtual reality robotic surgery simulator may minimize the steep learning curve associated with many of these complex procedures and thus enable more efficient learning and better patient outcomes. In this pilot study, we describe the learning curve of gynecologic surgical trainees on a virtual reality robotic simulator platform, the da Vinci Skills Simulator (MdVT), for a variety of robotic console exercises. Training on the MdVT was associated with improvement in multiple objective console performance metrics that are essential to the successful and safe performance of live surgery, including economy of motion, missed targets, and time to exercise completion. The da Vinci Skills Simulator has demonstrated face, content, construct, and concurrent validity in nongynecologic studies.15,21,24,25 However, the learning curve and ideal simulation exercises for gynecology residents have not been well defined. There are 40 console simulation modules available on the MdVT, and 4 exercises were specifically chosen in the current study that emphasized multitasking and represented a range of console skills that are critical for the performance of gynecologic procedures (clutching, camera movement, energy capabilities, and suturing). Although this is a pilot study, the learning curve data and selection of appropriate modules for gynecology residents would help inform the development of a formal, comprehensive robotics surgical curriculum for this cohort of trainees. An analysis of the learning curve data revealed that mean performance plateaus occurred with fewer than 10 repetitions for all console exercises regardless of trainee level. There was a significant improvement in performance between the 1st and 10th repetitions across multiple metrics for all training modules. As the performance plateau for the various exercises was reached by 6-9 repetitions, it is unlikely that an additional benefit would be gained from
TABLE 3. Mean Repetitions For Performance Plateau Overall score (%)
– Missed targets
10.4 (10.1, [8.2-13.9]) 2.0 (2, [1-3]) Master workspace range (cm) Critical errors
0 Excess force
Misapplied energy (s)
– –
0.009
0.459
0.071
175.4 (180.3, o0.001 148.9 (122.3, 75.6 (69.7, o0.001 [115.7-229]) [78.6-319.6]) [56.1-110.7]) 196.8 (198.7, o0.001 192.5 (185, 154.6 (152.5, o0.001 [158.2-229.7]) [138.2-298]) [131.1-192.4]) 0.95 (1, [0-4]) o0.001 0.1 (0, [0-1]) 0.05 (0, 0.564 [0-1]) 0.63 0.28 (0, [0-4]) 0.289 2.0 (0, [00.35 (0, 0.341 (0, [0-6.5]) 25.5]) [0-5.5]) 4.7 (4.5, 4.1 (4.2, o0.001 15.0 (15.6, 14.1 (13.4, 0.179 [4.1-6.3]) [3.4-4.8]) [6.8-21.2]) [7.6-22.5]) 2.45 (3, [1-4]) 0.25 (0, [0-1]) o0.001 2.45 (2, [0-5]) 0.9 (1, o0.001 [0-3]) 11.4 (11, 4.5 (4, [0-14]) o0.001 – – – [0-21]) – – – 8.4 (5.2, [3.33.6 (3.7, o0.001 30]) [1.2-5.8]) 62 (62, [3394 (94, [85- o0.001 69 (73, [2889 (91, o0.001 88]) 100]) 83]) [64-97]) o0.001 387.1 (397.2, [171.9-598.7] 0.001 339.4 (318.7, [198.2-543.4]) 0.094 7.1 (7, [0-22])
24.0 (21.8, [14.1-45]) 40.8 (42.1, [25.5-59.9]) 0.15 (0, [0-1]) 0.6 (0, [0-4]) 7.7 (7.6, [5.7-11.3]) 0
p p
55.3 (48.2, [22.9-178]) 58.5 (57.9, [27.9-98.7]) 0.5 (0, [0-3]) 0 0 2.1 (0.3, [0-11]) 10.3 (10.5, [70.845 7.8 13.8]) (7.8, [0-10.7]) 0.78 (1, [0-2]) o0.001 0.55 (0, [0-3]) – – –
p Last Trial (Median, [Range]) First Trial (Median, [Range]) Last Trial (Median, [Range]) First Trial (Median, [Range]) Last Trial (Median, [Range]) First Trial (Median, [Range])
Time to comple193.5 (189.7, 137.1 (116.1, 0.004 tion (s) [122.4-368.7]) [70.5-387.5]) Economy of motion (s) 339.6 (330, 238.1 (238.9, o0.001 [224.8-534.1]) [174.5-344.8]) Collisions 0.6 (0, [0-3]) 0.7 (1.0, [0-2]) 0.784
p Last Trial (Median, [Range]) First Trial (Median, [Range])
Energy Switching (N ¼ 20) Suture Sponge (N ¼ 20) Ring and Rail (N ¼ 20) Matchboard (N ¼ 18)
TABLE 2. Difference in Mean Performance Metrics Between 1st and 10th Repetitions Across 4 MdVT Exercises
DISCUSSION
Plateau (Median [Range]) MB (N ¼ 18) RR (N ¼ 20) SS (N ¼ 20) ES (N ¼ 20)
Journal of Surgical Education Volume 71/Number 1 January/February 2014
9.3 6.4 9.1 8.2
(11, [5-11]) (6, [1-11]) (11, [2-11]) (8, [2-11])
SD 2.4 3.6 3 2.7 129
TABLE 4. Mean Repetitions For Performance Plateau by Training Level
MB RR SS ES
PGY ≤ 2 (Median, [Range])
PGY ≥ 3 (Median, [Range])
p
9.6 (11, [5-11]) 6.8 (6, [1-11]) 8.8 (11, [4-11]) 7.8 (7.5, [2-11])
9.1 (11, [6-11]) 6 (5.5, [1-11]) 9.3 (11, [2-11]) 8.5 (8.5, [5-11])
0.665 0.630 0.718 0.579
having trainees perform more than 10 repetitions of a given exercise in a training session. Further, Tergas et al. suggest that uninterrupted training on the MdVT for an hour or more may lead to temporary eye and neck strain as well as headaches by users of the MdVT simulator.23 This supports the need to define the most safe and efficient use of the MdVT in training sessions. Prior learning curve studies for minimally invasive gynecologic procedures have involved performance on live surgical procedures,28,29 laparoscopic virtual reality simulators,30 and the da Vinci Surgical System dry laboratory platform.31 To our knowledge, our study is one of the first to define a learning curve amongst gynecology trainees for the specific MdVT modules studied. Unlike other surgical learning curve studies that focus primarily on time to completion of a surgical procedure or task, the current study analyzed trainee performance on several simulation metrics, including economy of motion, occurrence of critical errors, missed targets, misapplied energy, and overall performance over the course of 10 repetitions per exercise. This level of evaluation may help surgical faculty tailor their instruction and selection of console exercises to the needs of individual trainees by providing them with more specific, objective performance feedback than was previously possible. Performance improved in all parameters with the exception of workspace range (except in SS) and use of excessive force. These parameters may require direct proctoring by an experienced surgeon during the exercise to yield improvement. Additionally, our study identified areas that may serve as patient safety parameters. Performance metrics, such as missed targets for the suturing exercise and misapplied energy for the energy switching exercise, in which 10 repetitions resulted in significant improvement, may deserve special attention in the development of a surgical curriculum. Such parameters could be considered patient safety parameters as
inaccuracy in suture placement or bipolar/monopolar energy application could lead to surgical complications. Trainees should be required to gain competence in these parameters before performing live surgery. A limitation of the study was the small number of trainees that performed each exercise. Comparisons between junior and senior level trainees may also have been significantly affected by having only 10 trainees of each level for each exercise. Several participants were excluded from analysis of an exercise because they did not complete 10 repetitions. Future studies would benefit from enrolling a larger group of trainees and blocking out time in their schedule dedicated to simulation training. Additionally, voluntary recruitment may have affected the results as those trainees that agreed to participate may have been more motivated to improve their performance on the MdVT. The inclusion of a single gynecology training program in this study also limits generalizability of the results. Study strengths include assessment of novice robotic surgeons at a range of training levels and that this is one of the first studies to define a learning curve and potential curriculum for the MdVT amongst gynecology residents. As training programs develop robotic surgical skills curricula, it is important to consider inclusion of trainees at all levels and to identify skill-level appropriate exercises and proficiency goals. This study helps identify how senior and junior level trainees may differ in their baseline skills and the general slope of the learning curves that may be expected for both groups. Senior trainees performed the suturing exercise significantly faster than the junior trainees on the 1st repetition, and although this finding remained by the 10th repetition, the narrowing of the performance gap may be an indication of a steeper learning curve for junior level trainees. The significant improvement observed in several of the exercises studies indicates that they may serve as high-yield exercises that can help establish a baseline threshold that must be attained before trainees can participate in live robotic surgical procedures.
CONCLUSION This study demonstrates that the virtual reality robotic da Vinci Skills Simulator improves technical performance for
TABLE 5. Mean Performance Parameters by Training Level, Suture Sponge PGY ≤ 2 (N ¼ 10) (Median, [Range])
PGY ≥ 3 (N ¼ 10) (Median, [Range])
First trial time (s) First trial EOM (s) First trial overall score (%)
470.5 (499.9, [264.6-598.7]) 377.3 (353.2, [268.7543.4]) 55.9 (56, [33.0-78.0])
303.7 (286.5, [171.9-467.4]) 301.4 (304.05, [198.2-478.7]) 68.4 (65.0, [43.0-88.0])
Last trial time (s) Last trial EOM (s) Last trial overall score (%)
193.9 (194.7, [161.2-229.0]) 203.3 (201.2, [180.7-229.7]) 94.7 (96.0, [85.0-100.0])
156.8 (162.5, [115.7-187.6]) 190.2 (193.7, [158.2-217.4]) 92.8 (92.5, [86.0-100.0])
130
Difference
p
166.8 75.9 12.5
0.004 0.073 0.100
37.1 13.1 1.9
0.003 0.092 0.420
Journal of Surgical Education Volume 71/Number 1 January/February 2014
gynecology trainees at all levels across multiple surgical exercises. Junior level trainees may experience a steeper learning curve on certain robotic simulation exercises, whereas in other, possibly more surgically sophisticated, exercises, senior trainees demonstrated a steeper learning curve. In most cases, performance plateau is reached after 6 to 9 repetitions of a particular MdVT exercise. Such data can be used to help determine proficiency before live surgery, as well as to guide training and curriculum design. Future studies are needed to further inform robotic surgical curriculum including randomized trials investigating the utility of the da Vinci Skills Simulator in preparation for live surgery and to assess the effect of virtual reality simulation exercises on patient safety and outcomes. Larger studies involving multiple training institutions are also needed to identify high-yield simulator exercises, effect of direct proctoring vs. autonomous practice, and recommended intervals between training sessions to establish skills retention.
8. Weinberg L, Rao S, Escobar PF. Robotic surgery in
gynecology: an updated systematic review. Obstet Gynecol Int. 2011;2011:852061. 9. Accreditation council of graduate medical education.
Obstetrics and gynecology case logs: National data report 2008-2009. Available at: 〈http://www.acgme. org/residentdatacollection/documentation/statistical_ reports.asp〉. Accessed 14.04.12. 10. Accreditation council of graduate medical education.
Obstetrics and gynecology case logs: National data report 2010-2011. Available at: 〈http://www.acgme. org/residentdatacollection/documentation/statistical_ reports.asp〉. Accessed 14.04.12. 11. Smith AL, Schneider KM, Berens PD. Survey of
obstetrics and gynecology residents’ training and opinions on robotic surgery. J Robot Surg. 2010;4:23-27. 12. Gobern JM, Novak CM, Lockrow EG. Survey of
robotic surgery training in obstetrics and gynecology residency. J Minim Invasive Gynecol. 2011;18:755-760.
REFERENCES 1. Johnson N, Barlow D, Lethaby A, Tavender E, Curr
E, Garry R. Surgical approach to hysterectomy for benign gynaecological disease. Cochrane Database Syst Rev. 2006; CD003677. 2. Garry R, Fountain J, Mason S, et al. The evaluate study:
two parallel randomised trials, one comparing laparoscopic with abdominal hysterectomy, the other comparing laparoscopic with vaginal hysterectomy. Br Med J. 2004;328:129. 3. Choosing the route of hysterectomy for benign disease.
ACOG committee opinion no. 444. American College of Obstetricians and Gynecologists. Obstet Gynecol. 2009;114:1156-1158. 4. Intuitive surgical. Annual report 2010. Available at: 〈http://
investor.intuitivesurgical.com/phoenix.zhtml?c=122359&p =irol-reportsAnnual〉. Accessed on 14.04.12. 5. Tu FF, Beaumont JL, Senapati S, Gordon TEJ. Route
of hysterectomy influence and teaching hospital status. Obstet Gynecol. 2009;114(1):73-78. 6. Matthews CA, Reid N, Ramakrishnan V, Hull K,
Cohen S. Evaluation of the introduction of robotic technology on route of hysterectomy and complications in the first year of use. Am J Obstet Gynecol. 2010;203(5):[499 e1-499 e5]. 7. Paley PJ, Veljovich DS, Shah CA, et al. Surgical
outcomes in gynecologic oncology in the era of robotics: analysis of first 1000 cases. Am J Obstet Gynecol. 2011;204(6):[551.e1-551.e9].
13. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463. 14. Hyltander A, Liljegren E, Rhodin PH, Lonroth H.
The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16:1324-1328. 15. Schreuder HWR, Wolswjik R, Zweemer RP, Schijven
MP, Verheijen RHM. Training and learning robotic surgery, time for a more structured approach: a systematic review. BJOG. 2012;119:137-149. 16. Lepine LA, Hillis SD, Marchbanks PA, et al. Hyster-
ectomy surveillance—United States, 1980-1993. MMWR CDC Surveill Summ. 1997;46:1-15. 17. Whiteman MK, Hillis SD, Jamieson DJ, Morrow B,
Podgornik MN, Brett KM, Marchbanks PA. Inpatient hysterectomy surveillance in the United States, 2000– 2004. Am J Obstet Gynecol. 2008;198:34.e1–7. 18. Burden C, Oestergaard J, Larsen CR. Integration of
laparoscopic virtual-reality simulation into gynaecology training. BJOG. 2011;118:5-10. 19. Crochet P, Aggarwal R, Dubb SS, et al. Deliberate
practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg. 2011;253:1216-1222. 20. Loukas C, Nikiteas N, Kanakis M, Georgiou E. The
contribution of simulation training in enhancing key
Journal of Surgical Education Volume 71/Number 1 January/February 2014
131
components of laparoscopic competence. Am Surg. 2011;77:708-715. 21. Hung AJ, Zehnder P, Patil MB, et al. Content and
26. da Vinci surgical simulator brochure. Available at:
〈http://www.intuitivesurgical.com/products/skills_si mulator/〉. Accessed 29.04.12.
construct validity of a novel robotic surgery simulator. J Urol. 2011;186:1019-1024.
27. Mimic technologies. Available at: 〈http://www.mimicsimu
22. Lendvay TS, Casale P, Sweet R, Peters C. Initial
28. Song T, Kim TJ, Lee YY, et al. What is the learning
validation of a virtual-reality robotic simulator. J Robot Surg. 2008;2:145-149. 23. Tergas AI, Sheth SS, Green IC, Giuntoli RL, Winder
AD, Fader AN. A pilot study of surgical training utilizing a virtual robotic surgery simulator. JSLS, In press.
lation.com/products/〉. Accessed 29.04.12. curve for the single-port access laparoscopic-assisted vaginal hysterectomy? Eur J Obstet Gynecol Reprod Biol. 2011;158:93-96. 29. Ghomi A, Littman P, Prasad A, Einarsson JI. Assessing
24. Kelly DC, Margules AC, Kundavaram CR, et al. Face,
the learning curve for laparoscopic supracervical hysterectomy. JSLS. 2007;11:190-194.
content, and construct validation of the da Vinci skills simulator. Urology. 2012;79:1068-1072.
30. Gor M, McCloy R, Stone R, Smith A. Virtual reality
25. Finnegan KT, Meraney AM, Staff I, Shichman SJ. da
Vinci skills simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance. Urology. 2012;80:330-335.
132
laparoscopic simulator for assessment in gynaecology. BJOG. 2003;110:181-187. 31. Geller EJ, Schuler KM, Boggess JF. Robotic surgical
training program in gynecology: how to train residents and fellows. J Minim Invasive Gynecol. 2011;18:224-229.
Journal of Surgical Education Volume 71/Number 1 January/February 2014
