j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 7 ( 2 0 1 4 ) 5 3 e5 8
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
Do laparoscopic skills transfer to robotic surgery? Lucian Panait, MD,a,b Shohan Shetty, MD,a,* Patricia A. Shewokis, PhD,c,d and Juan A. Sanchez, MD, FACS, MPAa,e a
The Stanley J. Dudrick Department of Surgery, Saint Mary’s Hospital, Waterbury, Connecticut Drexel University College of Medicine, Philadelphia, Pennsylvania c Drexel University, Department of Nutrition Sciences, College of Nursing and Health Professions, Philadelphia, Pennsylvania d Drexel University, School of Biomedical Engineering, Science and Health Systems, Philadelphia, Pennsylvania e St. Agnes Healthcare, Baltimore, Maryland b
article info
abstract
Article history:
Background: Identifying the set of skills that can transfer from laparoscopic to robotic
Received 17 April 2013
surgery is an important consideration in designing optimal training curricula. We tested
Received in revised form
the degree to which laparoscopic skills transfer to a robotic platform.
9 September 2013
Methods: Fourteen medical students and 14 surgery residents with no previous robotic but
Accepted 8 October 2013
varying degrees of laparoscopic experience were studied. Three fundamentals of laparo-
Available online 12 October 2013
scopic surgery tasks were used on the laparoscopic box trainer and then the da Vinci robot: peg transfer (PT), circle cutting (CC), and intracorporeal suturing (IS). A questionnaire was
Keywords:
administered for assessing subjects’ comfort level with each task.
Surgical education
Results: Standard fundamentals of laparoscopic surgery scoring metric were used and higher
Simulation training
scores indicate a superior performance. For the group, PT and CC scores were similar between
Robotic surgery
robotic and laparoscopic modalities (90 versus 90 and 52 versus 47; P > 0.05). However, for the
FLS
advanced IS task, robotic-IS scores were significantly higher than laparoscopic-IS (80 versus
Laparoscopy
53; P < 0.001). Subgroup analysis of senior residents revealed a lower robotic-PT score when compared with laparoscopic-PT (92 versus 105; P < 0.05). Scores for CC and IS were similar in this subgroup (64 9 versus 69 15 and 95 3 versus 92 10; P > 0.05). The robot was favored over laparoscopy for all drills (PT, 66.7%; CC, 88.9%; IS, 94.4%). Conclusions: For simple tasks, participants with preexisting skills perform worse with the robot. However, with increasing task difficulty, robotic performance is equal or better than laparoscopy. Laparoscopic skills appear to readily transfer to a robotic platform, and difficult tasks such as IS are actually enhanced, even in subjects naive to the technology. ª 2014 Elsevier Inc. All rights reserved.
1.
Introduction
Robotic surgery is an innovative technology that is still evolving and has the potential to overcome the limitations of
laparoscopic techniques, while expanding the benefits of minimally invasive surgery. Its advantages include an improved degree of motion, better visualization and depth perception with three-dimensional monitors, and ergonomic
Presented at the American College of Surgeons 97th Annual Congress, San Francisco, October 2011. * Corresponding author. Saint Mary’s Hospital, The Stanley J. Dudrick Department of Surgery, 56 Franklin Street, Waterbury, CT 06706. Tel.: þ1 203 709 6315; fax: þ1 203 709 6089. E-mail address: [email protected] (S. Shetty). 0022-4804/$ e see front matter ª 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jss.2013.10.014
54
j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 7 ( 2 0 1 4 ) 5 3 e5 8
and comfortable seating positions for the surgeons. In addition, robotic surgery allows the surgeon to overcome the fulcrum effect and tremor noted in laparoscopic surgery, thereby improving the capacity to perform delicate dissections [1]. Some of the limitations of robotic surgery include a high start-up cost, requirement for additional operating room personnel, and the loss of haptic feedback [2]. The clinical efficacy and feasibility of robotic surgery is still being evaluated, while the applications in general surgery are emerging rapidly with advances in technology [3]. Several studies have demonstrated improved learning curves among inexperienced laparoscopic surgeons for advanced procedures on the robotic console [4e7]. Furthermore, robotic technology significantly enhanced performance in novices for complex tasks, such as intracorporeal suturing (IS) [8]. However, these learning curve benefits have not readily transferred to experienced laparoscopic surgeons. Stefanidis et al. [9] compared the performance of a difficult task (IS) on the robotic console with standard laparoscopy, in advanced users. Suturing scores were higher using standard laparoscopy when compared with robotic suturing, likely as a result of the participants’ greater experience with complex laparoscopic tasks, such as suturing. In the present study, we compare laparoscopic with robotic skill performance among subjects with different degrees of laparoscopic experience, including medical students and surgical residents at different levels in their training. The objective is to assess the degree to which laparoscopic skills are transferable to a robotic platform among different users. The hypothesis of our study was that a robot-assisted platform enhances user performance when performing basic laparoscopic maneuvers.
2.
Methods
Fourteen third-year medical students and 14 general surgery residents with no previous robotic but varying degrees of laparoscopic experience volunteered to participate in this Institutional Review Boardeapproved study. The resident cohort included eight junior residents (postgraduate year, 1e2) and six senior residents (postgraduate year, 3e5). On the basis of the Accreditation Council for Graduate Medical Education case logs, the senior residents had more than 100 laparoscopic cases as primary surgeons at the time the study was completed. All participants watched a video tutorial of three fundamentals of laparoscopic surgery (FLS) examination tasks of increasing level of difficulty, represented by peg transfer (PT), circle cutting (CC), and IS. Subsequently, the study subjects underwent skill evaluation on a laparoscopic box trainer using standard laparoscopic instruments. Before each exercise, participants were allowed 1 min to familiarize themselves with the laparoscopic instruments and the task to be performed. After at least a 24-h interval, the trainees repeated the same three FLS exercises on a da Vinci robotic console (Intuitive Surgical, Sunnyvale, CA). Similarly, they were allowed 1 min for each task, to familiarize themselves with the robotic console, instruments, and visualization. The 24-h interval was enforced to minimize the potential effect of the laparoscopic learning curve on the robotic performance. Because the FLS
tasks are time-limited, the time to perform all three tasks either robotically or laparoscopically was maximum 20 min per subject. Thus, we considered that a 24-h interval between sessions is sufficient to eliminate the effect of a potential learning curve. At the beginning of each trial the robotic instruments were set up by the instructor to easily reach all directions of the viewing field, thereby not requiring the use of the clutch during the performance of the task. The positions of the laparoscopic camera and the robotic camera were kept constant and did not require manipulation. Participant performance was assessed with a standardized, objective method based on task completion time and errors. Each exercise was scored for efficiency (time) and precision according to established methods. A cutoff time was assigned for each task. Time score (efficiency) was calculated by subtracting the actual time taken to complete the task from the cutoff time. A penalty score was applied for errors or lack of precision, and the penalty score was subtracted from the efficiency score to yield a final score for each task. Consequently, higher scores indicate superior performance [10]. Robotic scores were compared with laparoscopic scores for the whole group and for subgroups based on previous levels of laparoscopic expertise. The National Aeronautics and Space Administrationetask load index (NASA-TLX) questionnaire (Fig. 1) was used to evaluate task workload [11]. The NASA-TLX is a popular technique for measuring subjective mental workload. It relies on a multidimensional construct to derive an overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. With the aid of this validated 20-point visual analog scale, we documented the subjects’ self-reported performance, effort, and frustration as well as mental, physical, and temporal demands of each robotic and laparoscopic task. The total score obtained by adding individual scores in the six categories has previously been shown to reliably reflect workload [12,13]. A study synopsis is depicted in Figure 2. To assess the hypothesis that a robot-assisted platform enhances user performance over traditional training of basic laparoscopic maneuvers, separate paired t-tests were performed to calculate the objective performance score for each task (PT, CC, and IS). Secondary analyses were also conducted to determine if there were any differences in user performance across the training platforms and tasks based on previous laparoscopic training. Separate paired t-tests were conducted for the senior residents (n ¼ 5) on the objective performance score for each task. In addition to performance, we also assessed task workload with the NASA-TLX. Paired ttests were used to evaluate the six subscales comparing laparoscopic versus robotic scores. Significance criterion was set at a ¼ 0.05 for all tests. The Cohen dz effect size index was calculated to aid in the interpretation of the results. A post hoc power analysis was calculated to estimate the lowest meaningful population effect size [14] using Gpower software ver. 3.1.6 (Kiel, Germany). The population effect is best estimated by determining a 95% confidence interval of the effect size index [15]. To arrive at this estimate, we calculated the effect size for the mean difference performance score on the FLS tasks separately for the laparoscopic training compared with
55
j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 7 ( 2 0 1 4 ) 5 3 e5 8
Fig. 2 e Study synopsis.
Fig. 1 e NASA-TLX questionnaire. the robotic training. The overall IS task (highest difficulty level) resulted in a large effect (dz ¼ 0.97 with a correlation estimated to be r ¼ 0.50) and the calculated 95% confidence interval of the effect size was (0.38, 1.57). This confidence interval suggests that the true population effect ranges from a small to moderate robotic training advantage (d ¼ 0.38) to a large robotic training advantage (d ¼ 1.57). Given that we do not have estimates of variance and effect size for testing from a pilot study and the fact that the effect sizes vary across the FLS tasks, we estimate that a moderate effect size (dz ¼ 0.50) represents the lowest meaningful population effect size for selected FLS tasks of medium difficulty level and higher. Using a one-tailed mean difference matched pairs t-test with a significance level of 0.05, post hoc power is estimated to be 0.78 [14]. In this pilot study, only the IS task had sufficient statistical power.
3.
Results
All 28 participants completed the study. None of them had any robotic experience. However, the degree of laparoscopic
experience varied: the students had minimal exposure to laparoscopic instrumentation, whereas surgery residents had experience according to their postgraduate year and most of them had previously performed FLS tasks. For the group as a whole, FLS scores for PT and CC were similar between robotic and laparoscopic modalities (88 16 versus 87 19 and 53 15 versus 44 28, respectively; t(23) < 1.10, P > 0.05). Robotic-IS scores were significantly higher than laparoscopic-IS (80 17 versus 51 35; t(23) ¼ 4.69, P < 0.001, dz ¼ 0.97). Subgroup analysis of senior residents (n ¼ 5) revealed a lower robotic-PT score when compared with the laparoscopicPT (93 8 versus 105 5; t(4) ¼ 6.99, P < 0.05, dz ¼ 1.78). The scores for the CC and IS were similar between the robotic platform and the laparoscopic box trainer in this category of subjects (64 9 versus 69 15 and 95 3 versus 92 10, respectively; t < 2.18, P > 0.05). With the exception of physical demand, NASA-TLX questionnaire results did not reveal a significant difference between the robotic platform and the laparoscopic modality when performing PT. However, robotic-CC and robotic-IS were associated with decreased mental demand, physical demand, temporal demand, performance, effort, and frustration when compared with laparoscopic-CC and laparoscopic-IS, respectively; P < 0.05 (Table 1).
Table 1 e Comparison between the NASA-TLX task workload questionnaire scores for FLS and robotic modalities. Question
PT
CC
IS
Mental demand Physical demand Temporal demand Performance Effort Frustration
NS 0.04 NS NS NS NS
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
Do laparoscopic skills transfer to robotic surgery? Lucian Panait, MD,a,b Shohan Shetty, MD,a,* Patricia A. Shewokis, PhD,c,d and Juan A. Sanchez, MD, FACS, MPAa,e a
The Stanley J. Dudrick Department of Surgery, Saint Mary’s Hospital, Waterbury, Connecticut Drexel University College of Medicine, Philadelphia, Pennsylvania c Drexel University, Department of Nutrition Sciences, College of Nursing and Health Professions, Philadelphia, Pennsylvania d Drexel University, School of Biomedical Engineering, Science and Health Systems, Philadelphia, Pennsylvania e St. Agnes Healthcare, Baltimore, Maryland b
article info
abstract
Article history:
Background: Identifying the set of skills that can transfer from laparoscopic to robotic
Received 17 April 2013
surgery is an important consideration in designing optimal training curricula. We tested
Received in revised form
the degree to which laparoscopic skills transfer to a robotic platform.
9 September 2013
Methods: Fourteen medical students and 14 surgery residents with no previous robotic but
Accepted 8 October 2013
varying degrees of laparoscopic experience were studied. Three fundamentals of laparo-
Available online 12 October 2013
scopic surgery tasks were used on the laparoscopic box trainer and then the da Vinci robot: peg transfer (PT), circle cutting (CC), and intracorporeal suturing (IS). A questionnaire was
Keywords:
administered for assessing subjects’ comfort level with each task.
Surgical education
Results: Standard fundamentals of laparoscopic surgery scoring metric were used and higher
Simulation training
scores indicate a superior performance. For the group, PT and CC scores were similar between
Robotic surgery
robotic and laparoscopic modalities (90 versus 90 and 52 versus 47; P > 0.05). However, for the
FLS
advanced IS task, robotic-IS scores were significantly higher than laparoscopic-IS (80 versus
Laparoscopy
53; P < 0.001). Subgroup analysis of senior residents revealed a lower robotic-PT score when compared with laparoscopic-PT (92 versus 105; P < 0.05). Scores for CC and IS were similar in this subgroup (64 9 versus 69 15 and 95 3 versus 92 10; P > 0.05). The robot was favored over laparoscopy for all drills (PT, 66.7%; CC, 88.9%; IS, 94.4%). Conclusions: For simple tasks, participants with preexisting skills perform worse with the robot. However, with increasing task difficulty, robotic performance is equal or better than laparoscopy. Laparoscopic skills appear to readily transfer to a robotic platform, and difficult tasks such as IS are actually enhanced, even in subjects naive to the technology. ª 2014 Elsevier Inc. All rights reserved.
1.
Introduction
Robotic surgery is an innovative technology that is still evolving and has the potential to overcome the limitations of
laparoscopic techniques, while expanding the benefits of minimally invasive surgery. Its advantages include an improved degree of motion, better visualization and depth perception with three-dimensional monitors, and ergonomic
Presented at the American College of Surgeons 97th Annual Congress, San Francisco, October 2011. * Corresponding author. Saint Mary’s Hospital, The Stanley J. Dudrick Department of Surgery, 56 Franklin Street, Waterbury, CT 06706. Tel.: þ1 203 709 6315; fax: þ1 203 709 6089. E-mail address: [email protected] (S. Shetty). 0022-4804/$ e see front matter ª 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jss.2013.10.014
54
j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 7 ( 2 0 1 4 ) 5 3 e5 8
and comfortable seating positions for the surgeons. In addition, robotic surgery allows the surgeon to overcome the fulcrum effect and tremor noted in laparoscopic surgery, thereby improving the capacity to perform delicate dissections [1]. Some of the limitations of robotic surgery include a high start-up cost, requirement for additional operating room personnel, and the loss of haptic feedback [2]. The clinical efficacy and feasibility of robotic surgery is still being evaluated, while the applications in general surgery are emerging rapidly with advances in technology [3]. Several studies have demonstrated improved learning curves among inexperienced laparoscopic surgeons for advanced procedures on the robotic console [4e7]. Furthermore, robotic technology significantly enhanced performance in novices for complex tasks, such as intracorporeal suturing (IS) [8]. However, these learning curve benefits have not readily transferred to experienced laparoscopic surgeons. Stefanidis et al. [9] compared the performance of a difficult task (IS) on the robotic console with standard laparoscopy, in advanced users. Suturing scores were higher using standard laparoscopy when compared with robotic suturing, likely as a result of the participants’ greater experience with complex laparoscopic tasks, such as suturing. In the present study, we compare laparoscopic with robotic skill performance among subjects with different degrees of laparoscopic experience, including medical students and surgical residents at different levels in their training. The objective is to assess the degree to which laparoscopic skills are transferable to a robotic platform among different users. The hypothesis of our study was that a robot-assisted platform enhances user performance when performing basic laparoscopic maneuvers.
2.
Methods
Fourteen third-year medical students and 14 general surgery residents with no previous robotic but varying degrees of laparoscopic experience volunteered to participate in this Institutional Review Boardeapproved study. The resident cohort included eight junior residents (postgraduate year, 1e2) and six senior residents (postgraduate year, 3e5). On the basis of the Accreditation Council for Graduate Medical Education case logs, the senior residents had more than 100 laparoscopic cases as primary surgeons at the time the study was completed. All participants watched a video tutorial of three fundamentals of laparoscopic surgery (FLS) examination tasks of increasing level of difficulty, represented by peg transfer (PT), circle cutting (CC), and IS. Subsequently, the study subjects underwent skill evaluation on a laparoscopic box trainer using standard laparoscopic instruments. Before each exercise, participants were allowed 1 min to familiarize themselves with the laparoscopic instruments and the task to be performed. After at least a 24-h interval, the trainees repeated the same three FLS exercises on a da Vinci robotic console (Intuitive Surgical, Sunnyvale, CA). Similarly, they were allowed 1 min for each task, to familiarize themselves with the robotic console, instruments, and visualization. The 24-h interval was enforced to minimize the potential effect of the laparoscopic learning curve on the robotic performance. Because the FLS
tasks are time-limited, the time to perform all three tasks either robotically or laparoscopically was maximum 20 min per subject. Thus, we considered that a 24-h interval between sessions is sufficient to eliminate the effect of a potential learning curve. At the beginning of each trial the robotic instruments were set up by the instructor to easily reach all directions of the viewing field, thereby not requiring the use of the clutch during the performance of the task. The positions of the laparoscopic camera and the robotic camera were kept constant and did not require manipulation. Participant performance was assessed with a standardized, objective method based on task completion time and errors. Each exercise was scored for efficiency (time) and precision according to established methods. A cutoff time was assigned for each task. Time score (efficiency) was calculated by subtracting the actual time taken to complete the task from the cutoff time. A penalty score was applied for errors or lack of precision, and the penalty score was subtracted from the efficiency score to yield a final score for each task. Consequently, higher scores indicate superior performance [10]. Robotic scores were compared with laparoscopic scores for the whole group and for subgroups based on previous levels of laparoscopic expertise. The National Aeronautics and Space Administrationetask load index (NASA-TLX) questionnaire (Fig. 1) was used to evaluate task workload [11]. The NASA-TLX is a popular technique for measuring subjective mental workload. It relies on a multidimensional construct to derive an overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. With the aid of this validated 20-point visual analog scale, we documented the subjects’ self-reported performance, effort, and frustration as well as mental, physical, and temporal demands of each robotic and laparoscopic task. The total score obtained by adding individual scores in the six categories has previously been shown to reliably reflect workload [12,13]. A study synopsis is depicted in Figure 2. To assess the hypothesis that a robot-assisted platform enhances user performance over traditional training of basic laparoscopic maneuvers, separate paired t-tests were performed to calculate the objective performance score for each task (PT, CC, and IS). Secondary analyses were also conducted to determine if there were any differences in user performance across the training platforms and tasks based on previous laparoscopic training. Separate paired t-tests were conducted for the senior residents (n ¼ 5) on the objective performance score for each task. In addition to performance, we also assessed task workload with the NASA-TLX. Paired ttests were used to evaluate the six subscales comparing laparoscopic versus robotic scores. Significance criterion was set at a ¼ 0.05 for all tests. The Cohen dz effect size index was calculated to aid in the interpretation of the results. A post hoc power analysis was calculated to estimate the lowest meaningful population effect size [14] using Gpower software ver. 3.1.6 (Kiel, Germany). The population effect is best estimated by determining a 95% confidence interval of the effect size index [15]. To arrive at this estimate, we calculated the effect size for the mean difference performance score on the FLS tasks separately for the laparoscopic training compared with
55
j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 7 ( 2 0 1 4 ) 5 3 e5 8
Fig. 2 e Study synopsis.
Fig. 1 e NASA-TLX questionnaire. the robotic training. The overall IS task (highest difficulty level) resulted in a large effect (dz ¼ 0.97 with a correlation estimated to be r ¼ 0.50) and the calculated 95% confidence interval of the effect size was (0.38, 1.57). This confidence interval suggests that the true population effect ranges from a small to moderate robotic training advantage (d ¼ 0.38) to a large robotic training advantage (d ¼ 1.57). Given that we do not have estimates of variance and effect size for testing from a pilot study and the fact that the effect sizes vary across the FLS tasks, we estimate that a moderate effect size (dz ¼ 0.50) represents the lowest meaningful population effect size for selected FLS tasks of medium difficulty level and higher. Using a one-tailed mean difference matched pairs t-test with a significance level of 0.05, post hoc power is estimated to be 0.78 [14]. In this pilot study, only the IS task had sufficient statistical power.
3.
Results
All 28 participants completed the study. None of them had any robotic experience. However, the degree of laparoscopic
experience varied: the students had minimal exposure to laparoscopic instrumentation, whereas surgery residents had experience according to their postgraduate year and most of them had previously performed FLS tasks. For the group as a whole, FLS scores for PT and CC were similar between robotic and laparoscopic modalities (88 16 versus 87 19 and 53 15 versus 44 28, respectively; t(23) < 1.10, P > 0.05). Robotic-IS scores were significantly higher than laparoscopic-IS (80 17 versus 51 35; t(23) ¼ 4.69, P < 0.001, dz ¼ 0.97). Subgroup analysis of senior residents (n ¼ 5) revealed a lower robotic-PT score when compared with the laparoscopicPT (93 8 versus 105 5; t(4) ¼ 6.99, P < 0.05, dz ¼ 1.78). The scores for the CC and IS were similar between the robotic platform and the laparoscopic box trainer in this category of subjects (64 9 versus 69 15 and 95 3 versus 92 10, respectively; t < 2.18, P > 0.05). With the exception of physical demand, NASA-TLX questionnaire results did not reveal a significant difference between the robotic platform and the laparoscopic modality when performing PT. However, robotic-CC and robotic-IS were associated with decreased mental demand, physical demand, temporal demand, performance, effort, and frustration when compared with laparoscopic-CC and laparoscopic-IS, respectively; P < 0.05 (Table 1).
Table 1 e Comparison between the NASA-TLX task workload questionnaire scores for FLS and robotic modalities. Question
PT
CC
IS
Mental demand Physical demand Temporal demand Performance Effort Frustration
NS 0.04 NS NS NS NS
