537119 research-article2014
SRIXXX10.1177/1553350614537119Surgical InnovationSurgical InnovationSingapogu et al
Surgical Education: Training for the future
Simulator-Based Assessment of Haptic Surgical Skill: A Comparative Study
Surgical Innovation 2015, Vol. 22(2) 183–188 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1553350614537119 sri.sagepub.com
Ravikiran B. Singapogu, PhD1,2, Lindsay O. Long, PhD5, Dane E. Smith, MD1,3, Timothy C. Burg, PhD1,4, Christopher C. Pagano, PhD1,5, Varun V. Prabhu, MS4, and Karen J. L. Burg, PhD1,2,4
Abstract The aim of this study was to examine if the forces applied by users of a haptic simulator could be used to distinguish expert surgeons from novices. Seven surgeons with significant operating room expertise and 9 novices with no surgical experience participated in this study. The experimental task comprised exploring 4 virtual materials with the haptic device and learning the precise forces required to compress the materials to various depths. The virtual materials differed in their stiffness and force-displacement profiles. The results revealed that for nonlinear virtual materials, surgeons applied significantly greater magnitudes of force than novices. Furthermore, for the softer nonlinear and linear materials, surgeons were significantly more accurate in reproducing forces than novices. The results of this study suggest that the magnitudes of force measured using haptic simulators may be used to objectively differentiate experts’ haptic skill from that of novices. This knowledge can inform the design of virtual reality surgical simulators and lead to the future incorporation of haptic skills training in medical school curricula. Keywords simulation, surgical education, biomedical engineering
Introduction The landscape of surgical education is rapidly changing, with increasing emphases on objective assessments of surgical skills, optimizing medical students’ training time, and ensuring patients’ safety.1 Surgical simulators are poised to play a key role in this educational paradigm shift.2 Consequently, skills training laboratories across the world are seeking to incorporate simulators in their training curricula.3,4 Within the past decade, several simulator-based training curricula have been implemented and tested, such as the Fundamentals of Laparoscopic Surgery (FLS) for teaching basic laparoscopic skills in North America.5 The FLS simulator has undergone extensive validation testing and is now mandated by the American College of Surgeons for general surgery residents to demonstrate technical skills competency.6-8 The standard FLS simulator comprises a hollow box with ports for inserting laparoscopic instruments, a webcam for relaying video, and common surgical materials (suture, gauze, etc) for simulating 5 basic laparoscopic tasks. Assessment of surgical skills is done using time and accuracy metrics for tasks performed on the simulator. As the logical next step to improve on “low-tech” box trainers such as the FLS simulator, virtual reality (VR)
trainers have been suggested.9 VR trainers feature computer-based renderings of virtual worlds, which users explore through input devices. Using VR skills trainers, motion and time profiles of users’ tools can be recorded and used for advanced skills assessment, sometimes even while a task is being performed (ie, real-time performance feedback).10 Furthermore, a host of metrics can be devised for assessing trainees’ performance, eliminating the need for an expert to constantly coach a novice trainee, as is often the case with a box trainer.11 Another promising feature of VR trainers is the ability to track skills progress over time and develop training modules custom tailored to each trainee. As VR trainers evolve in 1
Institute for Biological Interfaces of Engineering, Clemson, SC, USA Department of Bioengineering, Clemson University, Clemson, SC, USA 3 Greenville Hospital System University Medical Center, Greenville, SC, USA 4 Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA 5 Department of Psychology, Clemson University, Clemson, SC, USA 2
Corresponding Author: Ravikiran B. Singapogu, PhD, Clemson University, 401 Rhodes Engineering Research Center, Clemson, SC 29634-0119, USA. Email: [email protected]
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
184
Surgical Innovation 22(2)
Figure 1. Experimental setup with haptic interface and with visual feedback in the learning session.
their capabilities, their efficacy in training is constantly being improved.12,13 One area of particular challenge for surgeon-educators and simulator designers alike is rendering force or haptic feedback that is useful for skills training.14,15 Haptic training is desirable because surgeons need to interpret reduced touch-based cues during minimally invasive surgery, inherent to the physical setup.16 Also present are other haptic sensations, such as the friction caused by trocars, the fulcrum effect caused by the pivoting of laparoscopic tools at the abdominal wall, and so on, that interfere with the touch cues from interacting with tissues.17 However, expert surgeons rely heavily on haptic cues to proficiently perform surgical tasks such as dissection and tissue manipulation.18,19 Therefore, it is necessary to devise simulator-based methods to teach this critical skill set to novice trainees. Standard box trainers do not necessarily include training for haptic skills. Recently, the manufacturers of some VR simulators have sought to include expensive haptic feedback mechanisms in their hardware.6,20 Studies on the effect of haptic feedback with these simulators, however, have reported disappointing results and consequently have raised important research questions.21-23 The objective of this study was to answer a fundamental research question regarding the use of haptic devices in surgical simulators: can the magnitude of forces applied using a haptic device differentiate experts’ skill from that of novices?
Materials and Methods The goal of this study was to examine if a VR-based haptic simulator could be used to differentiate the haptic skills of expert surgeons from those of novices. In current simulator validation methodology, the demonstration of evidence of validity on the basis of relationships to other
variables is a first step toward amassing validity evidence for the use of a surgical simulator.24 The haptic interface used in the study is the commercially available Falcon (Novint Inc, Albuquerque, New Mexico) that was controlled with a standard personal computer. The device was programmed using MATLAB (The MathWorks Inc, Natick, Massachusetts) in conjunction with QUARC (Quanser Inc, Markham, Ontario, Canada) software for high-performance haptics rendering. Four virtual materials were rendered, which were felt using the haptic device. Two materials had force-displacement profiles that varied linearly, and the other 2 had force-displacement profiles that varied nonlinearly. It should be noted that most tissue behavior is nonlinear, while for short displacements, linear force-displacement patterns may be approximated. The rendered stiffness model (where F is in newtons and x is in meters) for the softer linear material was F = 250x and for the harder material was F = 400x. For the softer nonlinear material, the stiffness model was F = 7000x2, and for the harder nonlinear material, the stiffness model was F = 250,000x3. The experimental task comprised participants’ (expert surgeons and novices) exploring the 4 virtual materials by axially moving the haptic interface and feeling forces proportional to the penetration distance into the virtual material (Figure 1). The goal of the task as communicated to all participants was to learn the forces at specific penetration distances into the virtual material and to reproduce these forces accurately with no visual cues. To aid participants in learning the specific penetration distances, a numeric score was presented to them on a computer monitor. This metric, called “score,” was a function of the penetration distance into the virtual material and was normalized to the range of [0, 125]. Consequently, for the same penetration distance into 2 different materials, the force felt could be significantly different because of their different force-displacement profiles.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
185
Singapogu et al Table 1. Results of t Tests and P Values for Applied Scores by Virtual Material and Score Level. Virtual Materials Linear Nonlinear
10
25
t(94) = 0.58, P > .05 t(94) = 1.19, P > .05
t(94) = 0.52, P > .05 t(94) = 0.26, P > .05
50 t(94) = 0.27, P > .05 t(94) = −3.08, P < .01a
75 t(94) = −1.9, P > .05 t(94) = −3.77, P < .01a
100 t(94) = −1.83, P > .05 t(94) = −2.83, P < .01a
a
Statistically significant (P < .05).
All participants started with a 3-minute learning session, wherein they were able to see their scores as they manipulated the virtual materials. This period enabled participants to correlate the forces required to obtain each of the 5 force levels: 10, 25, 50, 75, and 100. After completing this session, they were presented with the 4 virtual materials in random order and asked to reproduce the forces at specific levels (ie, 10, 25, 50, 75, and 100, in random order). No visual feedback was provided in this session. After making their force estimates using the haptic device, participants’ scores were recorded at each level. The testing session lasted about 10 to 15 minutes. Because the central hypothesis of the study examined the differences in force application behavior between experts and novices, 2 main statistical analyses were conducted. First, exerted-score means were calculated for both groups by force levels. Furthermore, to examine if both groups differed in force application accuracy, regression models for all 4 materials were constructed, with actual score as the independent variable and exerted score as the dependent variable. All statistical analyses were conducted using Minitab (Minitab Inc, State College, Pennsylvania) software.
Figure 2. Applied versus actual scores of surgeons and novices in linear virtual models.
Results Nine novices and 7 surgeons participated in this study. Novice participants had no previous laparoscopic or open surgical experience. Surgeons were medical residents and attending surgeons who had performed >100 laparoscopic surgical procedures. All participants filled out a questionnaire detailing their gaming and athletic histories (see Singapogu et al.25 for questionnaire analysis). Data analysis revealed that for the nonlinear materials, surgeons produced significantly more force than novices at upper force levels. For an expected score of 50, the mean force for surgeons was 68.52, compared with 56.29 for novices (P < .01); for an expected score of 75, the mean force for surgeons was 90.14, compared with 75.24 for novices (P < .01); and for an expected score of 100, the mean force for surgeons was 110.6, compared with 99.33 for novices (P < .01) (see Table 1, Figures 2 and 3). For the linear materials, no significant differences were found at any force levels (for an expected score of 75, the mean force for surgeons was 90.73, compared with 115.69 for novices; and for an expected score of 100,
Figure 3. Applied versus actual scores of surgeons and novices in nonlinear virtual models.
the mean force for surgeons was 81.96, compared with 107.4 for novices). When the 4 virtual materials were analyzed individually using regression models (produced score = f[actual score]), significant differences between surgeons and novices were evident for the softer linear and nonlinear materials (see Table 2). For the softer nonlinear material (K = 7000 ), surgeons had a significantly higher r2 value
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
186
Surgical Innovation 22(2)
Table 2. Simple Linear Regression Models for Novices and Surgeons for Each Virtual Material. r2 K 250 400 2000 3000
Slope
Intercept
Novices Surgeons Novices Surgeons Novices Surgeons 0.581a 0.862 0.626a 0.712
0.861a 0.846 0.801a 0.790
0.881 0.9371 0.827 0.781
1.103 0.989 0.971 1.023
16.61 18.22 14.34 20.35
18.13 7.24 9.63 18.54
a
Significantly different (P < .05).
than novices (t[14] = −2.44, P < .05), indicating that surgeons were more accurate when reproducing force levels on this virtual material. For the harder nonlinear material (K = 250,000), there was no significant difference in accuracy between novices and surgeons (t[14] = −0.79, P = .45). Similarly, for the softer linear material (K = 250), surgeons had a significantly higher overall r2 value than novices (t[14] = –2.73, P < .05), indicating that surgeons were more accurate when estimating forces. For the harder linear material (K = 400), there was no significant difference in accuracy between novices and surgeons (t[14] = –1.8, P = .10).
Discussion The results of this study demonstrate that when surgeons and novices applied forces using the haptic skills simulator, surgeons generally applied greater magnitudes of force than novices. Furthermore, surgeons were generally more accurate than novices, as revealed by the regression models. To our knowledge, this study is among the first to demonstrate that a VR haptic surgical skills simulator may be used to objectively differentiate the haptic skills of expert surgeons from those of novices on nonlinear virtual materials. The results of this study are supported by those of several previous studies. For instance, Zhou et al26 reported that when asked to apply contact forces on a synthetic tissue model with a laparoscopic tool, surgeons applied significantly greater forces than novices and also were significantly faster at detecting contact. Similarly, Wagner et al27 used a haptic virtual environment to test the magnitudes of applied forces for groups of various clinical experience in an artery dissection task. They reported that attending surgeons—clinicians with the most surgical experience—generally applied the greatest forces. In a recent study by Forrest et al,28 expert veterinarians and novices were examined for their haptic palpation skills on the Haptic Cow bovine simulator. Their results also revealed that expert clinicians generally applied significantly greater mean maximum forces compared with
novice students. Thus, the magnitudes of force applied using haptic interfaces in VR simulators seem to be a reliable indicator of the level of surgical expertise, as demonstrated in the present study. However, there are some significant research questions that need to be answered before widespread adoption of haptic surgical skills simulators. For example, Forrest et al28 noted that although significant differences in haptic behavior on the simulator were observed, neither group was able to identify >2 levels of force. One would reasonably expect that experts should be able to identify more subtle variations in force levels. Furthermore, it was noted that clinicians produced “considerably different” exploratory patterns when interacting with the haptic simulator, depending on their conceptions of the context. When users were told to assume that the virtual material was close to the real tissue they might encounter clinically, interaction patterns were different from those seen when users were not given a clinical context. This insight concurs with our observations in using haptic interfaces for surgical simulators. In 2 recent studies, we tested a custom-built haptic simulator for evidence of validity on the basis of relationships to other variables. The simulator was constructed for the distinct purpose of haptic skills training for laparoscopic surgery and therefore features standard laparoscopic tools that are instrumented to sense and render forces.29 Our studies revealed that although experts’ skill level was clearly distinguishable from that of novices using simulator metrics, the magnitudes of force applied by novices and surgeons were reversed.15 Novices applied greater forces than surgeons, and training novices on the simulator resulted in decreased force magnitude patterns.14 In this case, the hardware of the simulator combined with the task and virtual materials rendered may have presented a different context to users, especially clinicians. This line of reasoning agrees with the results of a study by Richards et al,19 who recorded force and torque data as experts and novices performed 2 common laparoscopic procedures on porcine models. They reported that surgeons applied greater forces than novices during dissection-like tasks, whereas surgeons applied lesser forces than novices when performing tissue manipulation tasks, 2 different contexts for surgical tasks.19 On the basis of the cumulative evidence of these studies, it seems that surgeons apply greater forces than novices for dissection-like tasks because of their familiarity with tissue behavior. That is, on the basis of their previous experience with tissue handling, they apply forces within a “tight” range, the lower boundary of which is the minimal force to perceive tissue contact, and the upper boundary of which is less than forces that cause irreversible tissue damage.26 Novices, on the other hand, are
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
187
Singapogu et al much more tentative when applying forces, because of their unfamiliarity with tissue interaction forces. Further research is needed to examine some crucial questions pertaining to the hardware and software makeup of haptic surgical simulators. For example, what stiffness profiles are suitable for the virtual tissue models rendered on the simulators? Also, should other haptic attributes, such as the size and shape of the virtual materials, be taken into consideration when creating a standard testing model for skills evaluation? Another key area of research pertains to the hardware configurations of haptic interfaces. Current haptic devices differ widely in their mechanical construction and force-reflecting capabilities. In fact, several recent studies of commercial laparoscopic haptic simulators have called into question the efficacy of the haptic interfaces used in these particular studies to render truly useful force feedback for training. Optimizing haptic hardware may also help in understanding motor strategy patterns used by surgeons and novices when using haptic devices, because studies have pointed out that users differ in their exploratory patterns depending on their levels of expertise.30 Therefore, although we report promising initial results regarding the use of haptic surgical simulators in quantifying surgical skills, further work must be undertaken before the widespread adoption of force-enabled simulators in medical schools. However, one can foresee haptic simulators’ being useful in surgical warm-up, in which patient-specific nonlinear tissue models can be simulated for surgeons’ practice before entering the operating room. Haptic simulators may also find particular use in training for operations with delicate tissues and/or instruments.
Conclusion The future holds great promise for a simulator-based haptic skills training methodology. By using simulators with evidence for validity, novices’ skill levels can be objectively tracked over their learning period. Also, proficiency targets can be set during training, relative to established expert skill levels. Consequently, surgical simulators may play a key role in the future of surgery to increase patient safety, ensure surgeons’ competence, and instill confidence in novice trainees. Authors’ Note This work was presented at the Medicine Meets Virtual Reality conference, San Diego, California, February 2012.
Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) received the following financial support for the research, authorship, and/or publication of this article: Support was provided by the Clemson University Institute for Biological Interfaces of Engineering.
References 1. Roberts K, Bell R, Duffy A. Evolution of surgical skills training. World J Gastroenterol. 2006;12:3219-3224. 2. Tsuda S, Scott D, Doyle J, Jones DB. Surgical skills training and simulation. Curr Probl Surg. 2009;46:271-370. 3. Stefanidis D, Arora S, Parrack DM, et al. Research priorities in surgical simulation for the 21st century. Am J Surg. 2012;203:49-53. 4. Stefanidis D, Heniford BT. The formula for a successful laparoscopic skills curriculum. Arch Surg. 2009;144:77-82. 5. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240:518-525. 6. Panait L, Akkary E, Bell RL, Roberts KE, Dudrick SJ, Duffy AJ. The role of haptic feedback in laparoscopic simulation training. J Surg Res. 2009;156:312-316. 7. McCluney AL, Vassiliou MC, Kaneva PA, et al. FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc. 2007;21:1991-1995. 8. Fried GM. FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg. 2008;12:210-212. 9. Satava RM. Virtual reality surgical simulator. The first steps. Surg Endosc. 1993;7:203-205. 10. Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg. 2007;246:771-779. 11. Chmarra MK, Klein S, de Winter JCF, Jansen F-W, Dankelman J. Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc. 2010;24:1031-1039. 12. Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg. 2008;95:1088-1097. 13. Thijssen AS, Schijven MP. Contemporary virtual reality laparoscopy simulators: quicksand or solid grounds for assessing surgical trainees? Am J Surg. 2010;199:529-541. 14. Singapogu RB, DuBose S, Long LO, et al. Salient haptic skills trainer: initial validation of a novel simulator for training force-based laparoscopic surgical skills. Surg Endosc. 2013;27:1653-1661. 15. Singapogu RB, Smith DE, Long LO, Burg TC, Pagano CC, Burg KJL. Objective differentiation of force-based laparoscopic skills using a novel haptic simulator. J Surg Educ. 2012;69:766-773. 16. Westebring-van der Putten EP, Goossens RH, Jakimowicz JJ, Dankelman J. Haptics in minimally invasive surgery—a review. Minim Invasive Ther Allied Technol. 2008;17:3-16. 17. Perreault JO, Cao CG. Effects of vision and friction on haptic perception. Hum Factors. 2006;48:574-586.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
188
Surgical Innovation 22(2)
18. Lamata P, Gómez EJ, Sánchez-Margallo FM, Lamata F, del Pozo F, Usón J. Tissue consistency perception in laparoscopy to define the level of fidelity in virtual reality simulation. Surg Endosc. 2006;20:1368-1375. 19. Richards C, Rosen J, Hannaford B, Pellegrini C, Sinanan M. Skills evaluation in minimally invasive surgery using force/torque signatures. Surg Endosc. 2000;14:791-798. 20. Salkini MW, Doarn CR, Kiehl N, Broderick TJ, Donovan JF, Gaitonde K. The role of haptic feedback in laparoscopic training using the LapMentor II. J Endourol. 2010;24:99-102. 21. Thompson JR, Leonard AC, Doarn CR, Roesch MJ, Broderick TJ. Limited value of haptics in virtual reality laparoscopic cholecystectomy training. Surg Endosc. 2010;25:1107-1114. 22. Zhou M, Tse S, Derevianko A, Jones DB, Schwaitzberg SD, Cao CG. Effect of haptic feedback in laparoscopic surgery skill acquisition. Surg Endosc. 2012;26:1128-1134. 23. Chmarra MK, Dankelman J, van den Dobbelsteen JJ, Jansen F-W. Force feedback and basic laparoscopic skills. Surg Endosc. 2008;22:2140-2148. 24. Schout BM, Hendrikx AJ, Scheele F, Bemelmans BL, Scherpbier AJ. Validation and implementation of surgical
simulators: a critical review of present, past, and future. Surg Endosc. 2010;24:536-546. 25. Singapogu RB, Smith DE, Altenhoff BM, et al. Assessing surgeon and novice force skill on a haptic stiffness simulator for laparoscopic surgery. Stud Health Technol Inform. 2012;173:469-474. 26. Zhou M, Perreault J, Schwaitzberg S, Cao C. Effects of experience on force perception threshold in minimally invasive surgery. Surg Endosc. 2008;22:510-515. 27. Wagner CR, Stylopoulos N, Jackson PG, Howe RD. The benefit of force feedback in surgery: examination of blunt dissection. Presence Teleoperators Virtual Environ. 2007;16:252-262. 28. Forrest N, Baillie S, Kalita P, Tan HZ. A comparative study of haptic stiffness identification by veterinarians and students. IEEE Trans Haptics. 2011;4:78-87. 29. Long LO, Singapogu RB, Arcese G, et al. A haptic simulator to increase laparoscopic force application sensitivity. Stud Health Technol Inform. 2013;184:273-275. 30. Lederman SJ, Klatzky RL. Hand movements: a window into haptic object recognition. Cogn Psychol. 1987;19: 342-368.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
SRIXXX10.1177/1553350614537119Surgical InnovationSurgical InnovationSingapogu et al
Surgical Education: Training for the future
Simulator-Based Assessment of Haptic Surgical Skill: A Comparative Study
Surgical Innovation 2015, Vol. 22(2) 183–188 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1553350614537119 sri.sagepub.com
Ravikiran B. Singapogu, PhD1,2, Lindsay O. Long, PhD5, Dane E. Smith, MD1,3, Timothy C. Burg, PhD1,4, Christopher C. Pagano, PhD1,5, Varun V. Prabhu, MS4, and Karen J. L. Burg, PhD1,2,4
Abstract The aim of this study was to examine if the forces applied by users of a haptic simulator could be used to distinguish expert surgeons from novices. Seven surgeons with significant operating room expertise and 9 novices with no surgical experience participated in this study. The experimental task comprised exploring 4 virtual materials with the haptic device and learning the precise forces required to compress the materials to various depths. The virtual materials differed in their stiffness and force-displacement profiles. The results revealed that for nonlinear virtual materials, surgeons applied significantly greater magnitudes of force than novices. Furthermore, for the softer nonlinear and linear materials, surgeons were significantly more accurate in reproducing forces than novices. The results of this study suggest that the magnitudes of force measured using haptic simulators may be used to objectively differentiate experts’ haptic skill from that of novices. This knowledge can inform the design of virtual reality surgical simulators and lead to the future incorporation of haptic skills training in medical school curricula. Keywords simulation, surgical education, biomedical engineering
Introduction The landscape of surgical education is rapidly changing, with increasing emphases on objective assessments of surgical skills, optimizing medical students’ training time, and ensuring patients’ safety.1 Surgical simulators are poised to play a key role in this educational paradigm shift.2 Consequently, skills training laboratories across the world are seeking to incorporate simulators in their training curricula.3,4 Within the past decade, several simulator-based training curricula have been implemented and tested, such as the Fundamentals of Laparoscopic Surgery (FLS) for teaching basic laparoscopic skills in North America.5 The FLS simulator has undergone extensive validation testing and is now mandated by the American College of Surgeons for general surgery residents to demonstrate technical skills competency.6-8 The standard FLS simulator comprises a hollow box with ports for inserting laparoscopic instruments, a webcam for relaying video, and common surgical materials (suture, gauze, etc) for simulating 5 basic laparoscopic tasks. Assessment of surgical skills is done using time and accuracy metrics for tasks performed on the simulator. As the logical next step to improve on “low-tech” box trainers such as the FLS simulator, virtual reality (VR)
trainers have been suggested.9 VR trainers feature computer-based renderings of virtual worlds, which users explore through input devices. Using VR skills trainers, motion and time profiles of users’ tools can be recorded and used for advanced skills assessment, sometimes even while a task is being performed (ie, real-time performance feedback).10 Furthermore, a host of metrics can be devised for assessing trainees’ performance, eliminating the need for an expert to constantly coach a novice trainee, as is often the case with a box trainer.11 Another promising feature of VR trainers is the ability to track skills progress over time and develop training modules custom tailored to each trainee. As VR trainers evolve in 1
Institute for Biological Interfaces of Engineering, Clemson, SC, USA Department of Bioengineering, Clemson University, Clemson, SC, USA 3 Greenville Hospital System University Medical Center, Greenville, SC, USA 4 Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA 5 Department of Psychology, Clemson University, Clemson, SC, USA 2
Corresponding Author: Ravikiran B. Singapogu, PhD, Clemson University, 401 Rhodes Engineering Research Center, Clemson, SC 29634-0119, USA. Email: [email protected]
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
184
Surgical Innovation 22(2)
Figure 1. Experimental setup with haptic interface and with visual feedback in the learning session.
their capabilities, their efficacy in training is constantly being improved.12,13 One area of particular challenge for surgeon-educators and simulator designers alike is rendering force or haptic feedback that is useful for skills training.14,15 Haptic training is desirable because surgeons need to interpret reduced touch-based cues during minimally invasive surgery, inherent to the physical setup.16 Also present are other haptic sensations, such as the friction caused by trocars, the fulcrum effect caused by the pivoting of laparoscopic tools at the abdominal wall, and so on, that interfere with the touch cues from interacting with tissues.17 However, expert surgeons rely heavily on haptic cues to proficiently perform surgical tasks such as dissection and tissue manipulation.18,19 Therefore, it is necessary to devise simulator-based methods to teach this critical skill set to novice trainees. Standard box trainers do not necessarily include training for haptic skills. Recently, the manufacturers of some VR simulators have sought to include expensive haptic feedback mechanisms in their hardware.6,20 Studies on the effect of haptic feedback with these simulators, however, have reported disappointing results and consequently have raised important research questions.21-23 The objective of this study was to answer a fundamental research question regarding the use of haptic devices in surgical simulators: can the magnitude of forces applied using a haptic device differentiate experts’ skill from that of novices?
Materials and Methods The goal of this study was to examine if a VR-based haptic simulator could be used to differentiate the haptic skills of expert surgeons from those of novices. In current simulator validation methodology, the demonstration of evidence of validity on the basis of relationships to other
variables is a first step toward amassing validity evidence for the use of a surgical simulator.24 The haptic interface used in the study is the commercially available Falcon (Novint Inc, Albuquerque, New Mexico) that was controlled with a standard personal computer. The device was programmed using MATLAB (The MathWorks Inc, Natick, Massachusetts) in conjunction with QUARC (Quanser Inc, Markham, Ontario, Canada) software for high-performance haptics rendering. Four virtual materials were rendered, which were felt using the haptic device. Two materials had force-displacement profiles that varied linearly, and the other 2 had force-displacement profiles that varied nonlinearly. It should be noted that most tissue behavior is nonlinear, while for short displacements, linear force-displacement patterns may be approximated. The rendered stiffness model (where F is in newtons and x is in meters) for the softer linear material was F = 250x and for the harder material was F = 400x. For the softer nonlinear material, the stiffness model was F = 7000x2, and for the harder nonlinear material, the stiffness model was F = 250,000x3. The experimental task comprised participants’ (expert surgeons and novices) exploring the 4 virtual materials by axially moving the haptic interface and feeling forces proportional to the penetration distance into the virtual material (Figure 1). The goal of the task as communicated to all participants was to learn the forces at specific penetration distances into the virtual material and to reproduce these forces accurately with no visual cues. To aid participants in learning the specific penetration distances, a numeric score was presented to them on a computer monitor. This metric, called “score,” was a function of the penetration distance into the virtual material and was normalized to the range of [0, 125]. Consequently, for the same penetration distance into 2 different materials, the force felt could be significantly different because of their different force-displacement profiles.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
185
Singapogu et al Table 1. Results of t Tests and P Values for Applied Scores by Virtual Material and Score Level. Virtual Materials Linear Nonlinear
10
25
t(94) = 0.58, P > .05 t(94) = 1.19, P > .05
t(94) = 0.52, P > .05 t(94) = 0.26, P > .05
50 t(94) = 0.27, P > .05 t(94) = −3.08, P < .01a
75 t(94) = −1.9, P > .05 t(94) = −3.77, P < .01a
100 t(94) = −1.83, P > .05 t(94) = −2.83, P < .01a
a
Statistically significant (P < .05).
All participants started with a 3-minute learning session, wherein they were able to see their scores as they manipulated the virtual materials. This period enabled participants to correlate the forces required to obtain each of the 5 force levels: 10, 25, 50, 75, and 100. After completing this session, they were presented with the 4 virtual materials in random order and asked to reproduce the forces at specific levels (ie, 10, 25, 50, 75, and 100, in random order). No visual feedback was provided in this session. After making their force estimates using the haptic device, participants’ scores were recorded at each level. The testing session lasted about 10 to 15 minutes. Because the central hypothesis of the study examined the differences in force application behavior between experts and novices, 2 main statistical analyses were conducted. First, exerted-score means were calculated for both groups by force levels. Furthermore, to examine if both groups differed in force application accuracy, regression models for all 4 materials were constructed, with actual score as the independent variable and exerted score as the dependent variable. All statistical analyses were conducted using Minitab (Minitab Inc, State College, Pennsylvania) software.
Figure 2. Applied versus actual scores of surgeons and novices in linear virtual models.
Results Nine novices and 7 surgeons participated in this study. Novice participants had no previous laparoscopic or open surgical experience. Surgeons were medical residents and attending surgeons who had performed >100 laparoscopic surgical procedures. All participants filled out a questionnaire detailing their gaming and athletic histories (see Singapogu et al.25 for questionnaire analysis). Data analysis revealed that for the nonlinear materials, surgeons produced significantly more force than novices at upper force levels. For an expected score of 50, the mean force for surgeons was 68.52, compared with 56.29 for novices (P < .01); for an expected score of 75, the mean force for surgeons was 90.14, compared with 75.24 for novices (P < .01); and for an expected score of 100, the mean force for surgeons was 110.6, compared with 99.33 for novices (P < .01) (see Table 1, Figures 2 and 3). For the linear materials, no significant differences were found at any force levels (for an expected score of 75, the mean force for surgeons was 90.73, compared with 115.69 for novices; and for an expected score of 100,
Figure 3. Applied versus actual scores of surgeons and novices in nonlinear virtual models.
the mean force for surgeons was 81.96, compared with 107.4 for novices). When the 4 virtual materials were analyzed individually using regression models (produced score = f[actual score]), significant differences between surgeons and novices were evident for the softer linear and nonlinear materials (see Table 2). For the softer nonlinear material (K = 7000 ), surgeons had a significantly higher r2 value
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
186
Surgical Innovation 22(2)
Table 2. Simple Linear Regression Models for Novices and Surgeons for Each Virtual Material. r2 K 250 400 2000 3000
Slope
Intercept
Novices Surgeons Novices Surgeons Novices Surgeons 0.581a 0.862 0.626a 0.712
0.861a 0.846 0.801a 0.790
0.881 0.9371 0.827 0.781
1.103 0.989 0.971 1.023
16.61 18.22 14.34 20.35
18.13 7.24 9.63 18.54
a
Significantly different (P < .05).
than novices (t[14] = −2.44, P < .05), indicating that surgeons were more accurate when reproducing force levels on this virtual material. For the harder nonlinear material (K = 250,000), there was no significant difference in accuracy between novices and surgeons (t[14] = −0.79, P = .45). Similarly, for the softer linear material (K = 250), surgeons had a significantly higher overall r2 value than novices (t[14] = –2.73, P < .05), indicating that surgeons were more accurate when estimating forces. For the harder linear material (K = 400), there was no significant difference in accuracy between novices and surgeons (t[14] = –1.8, P = .10).
Discussion The results of this study demonstrate that when surgeons and novices applied forces using the haptic skills simulator, surgeons generally applied greater magnitudes of force than novices. Furthermore, surgeons were generally more accurate than novices, as revealed by the regression models. To our knowledge, this study is among the first to demonstrate that a VR haptic surgical skills simulator may be used to objectively differentiate the haptic skills of expert surgeons from those of novices on nonlinear virtual materials. The results of this study are supported by those of several previous studies. For instance, Zhou et al26 reported that when asked to apply contact forces on a synthetic tissue model with a laparoscopic tool, surgeons applied significantly greater forces than novices and also were significantly faster at detecting contact. Similarly, Wagner et al27 used a haptic virtual environment to test the magnitudes of applied forces for groups of various clinical experience in an artery dissection task. They reported that attending surgeons—clinicians with the most surgical experience—generally applied the greatest forces. In a recent study by Forrest et al,28 expert veterinarians and novices were examined for their haptic palpation skills on the Haptic Cow bovine simulator. Their results also revealed that expert clinicians generally applied significantly greater mean maximum forces compared with
novice students. Thus, the magnitudes of force applied using haptic interfaces in VR simulators seem to be a reliable indicator of the level of surgical expertise, as demonstrated in the present study. However, there are some significant research questions that need to be answered before widespread adoption of haptic surgical skills simulators. For example, Forrest et al28 noted that although significant differences in haptic behavior on the simulator were observed, neither group was able to identify >2 levels of force. One would reasonably expect that experts should be able to identify more subtle variations in force levels. Furthermore, it was noted that clinicians produced “considerably different” exploratory patterns when interacting with the haptic simulator, depending on their conceptions of the context. When users were told to assume that the virtual material was close to the real tissue they might encounter clinically, interaction patterns were different from those seen when users were not given a clinical context. This insight concurs with our observations in using haptic interfaces for surgical simulators. In 2 recent studies, we tested a custom-built haptic simulator for evidence of validity on the basis of relationships to other variables. The simulator was constructed for the distinct purpose of haptic skills training for laparoscopic surgery and therefore features standard laparoscopic tools that are instrumented to sense and render forces.29 Our studies revealed that although experts’ skill level was clearly distinguishable from that of novices using simulator metrics, the magnitudes of force applied by novices and surgeons were reversed.15 Novices applied greater forces than surgeons, and training novices on the simulator resulted in decreased force magnitude patterns.14 In this case, the hardware of the simulator combined with the task and virtual materials rendered may have presented a different context to users, especially clinicians. This line of reasoning agrees with the results of a study by Richards et al,19 who recorded force and torque data as experts and novices performed 2 common laparoscopic procedures on porcine models. They reported that surgeons applied greater forces than novices during dissection-like tasks, whereas surgeons applied lesser forces than novices when performing tissue manipulation tasks, 2 different contexts for surgical tasks.19 On the basis of the cumulative evidence of these studies, it seems that surgeons apply greater forces than novices for dissection-like tasks because of their familiarity with tissue behavior. That is, on the basis of their previous experience with tissue handling, they apply forces within a “tight” range, the lower boundary of which is the minimal force to perceive tissue contact, and the upper boundary of which is less than forces that cause irreversible tissue damage.26 Novices, on the other hand, are
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
187
Singapogu et al much more tentative when applying forces, because of their unfamiliarity with tissue interaction forces. Further research is needed to examine some crucial questions pertaining to the hardware and software makeup of haptic surgical simulators. For example, what stiffness profiles are suitable for the virtual tissue models rendered on the simulators? Also, should other haptic attributes, such as the size and shape of the virtual materials, be taken into consideration when creating a standard testing model for skills evaluation? Another key area of research pertains to the hardware configurations of haptic interfaces. Current haptic devices differ widely in their mechanical construction and force-reflecting capabilities. In fact, several recent studies of commercial laparoscopic haptic simulators have called into question the efficacy of the haptic interfaces used in these particular studies to render truly useful force feedback for training. Optimizing haptic hardware may also help in understanding motor strategy patterns used by surgeons and novices when using haptic devices, because studies have pointed out that users differ in their exploratory patterns depending on their levels of expertise.30 Therefore, although we report promising initial results regarding the use of haptic surgical simulators in quantifying surgical skills, further work must be undertaken before the widespread adoption of force-enabled simulators in medical schools. However, one can foresee haptic simulators’ being useful in surgical warm-up, in which patient-specific nonlinear tissue models can be simulated for surgeons’ practice before entering the operating room. Haptic simulators may also find particular use in training for operations with delicate tissues and/or instruments.
Conclusion The future holds great promise for a simulator-based haptic skills training methodology. By using simulators with evidence for validity, novices’ skill levels can be objectively tracked over their learning period. Also, proficiency targets can be set during training, relative to established expert skill levels. Consequently, surgical simulators may play a key role in the future of surgery to increase patient safety, ensure surgeons’ competence, and instill confidence in novice trainees. Authors’ Note This work was presented at the Medicine Meets Virtual Reality conference, San Diego, California, February 2012.
Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) received the following financial support for the research, authorship, and/or publication of this article: Support was provided by the Clemson University Institute for Biological Interfaces of Engineering.
References 1. Roberts K, Bell R, Duffy A. Evolution of surgical skills training. World J Gastroenterol. 2006;12:3219-3224. 2. Tsuda S, Scott D, Doyle J, Jones DB. Surgical skills training and simulation. Curr Probl Surg. 2009;46:271-370. 3. Stefanidis D, Arora S, Parrack DM, et al. Research priorities in surgical simulation for the 21st century. Am J Surg. 2012;203:49-53. 4. Stefanidis D, Heniford BT. The formula for a successful laparoscopic skills curriculum. Arch Surg. 2009;144:77-82. 5. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240:518-525. 6. Panait L, Akkary E, Bell RL, Roberts KE, Dudrick SJ, Duffy AJ. The role of haptic feedback in laparoscopic simulation training. J Surg Res. 2009;156:312-316. 7. McCluney AL, Vassiliou MC, Kaneva PA, et al. FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc. 2007;21:1991-1995. 8. Fried GM. FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg. 2008;12:210-212. 9. Satava RM. Virtual reality surgical simulator. The first steps. Surg Endosc. 1993;7:203-205. 10. Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg. 2007;246:771-779. 11. Chmarra MK, Klein S, de Winter JCF, Jansen F-W, Dankelman J. Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc. 2010;24:1031-1039. 12. Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg. 2008;95:1088-1097. 13. Thijssen AS, Schijven MP. Contemporary virtual reality laparoscopy simulators: quicksand or solid grounds for assessing surgical trainees? Am J Surg. 2010;199:529-541. 14. Singapogu RB, DuBose S, Long LO, et al. Salient haptic skills trainer: initial validation of a novel simulator for training force-based laparoscopic surgical skills. Surg Endosc. 2013;27:1653-1661. 15. Singapogu RB, Smith DE, Long LO, Burg TC, Pagano CC, Burg KJL. Objective differentiation of force-based laparoscopic skills using a novel haptic simulator. J Surg Educ. 2012;69:766-773. 16. Westebring-van der Putten EP, Goossens RH, Jakimowicz JJ, Dankelman J. Haptics in minimally invasive surgery—a review. Minim Invasive Ther Allied Technol. 2008;17:3-16. 17. Perreault JO, Cao CG. Effects of vision and friction on haptic perception. Hum Factors. 2006;48:574-586.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
188
Surgical Innovation 22(2)
18. Lamata P, Gómez EJ, Sánchez-Margallo FM, Lamata F, del Pozo F, Usón J. Tissue consistency perception in laparoscopy to define the level of fidelity in virtual reality simulation. Surg Endosc. 2006;20:1368-1375. 19. Richards C, Rosen J, Hannaford B, Pellegrini C, Sinanan M. Skills evaluation in minimally invasive surgery using force/torque signatures. Surg Endosc. 2000;14:791-798. 20. Salkini MW, Doarn CR, Kiehl N, Broderick TJ, Donovan JF, Gaitonde K. The role of haptic feedback in laparoscopic training using the LapMentor II. J Endourol. 2010;24:99-102. 21. Thompson JR, Leonard AC, Doarn CR, Roesch MJ, Broderick TJ. Limited value of haptics in virtual reality laparoscopic cholecystectomy training. Surg Endosc. 2010;25:1107-1114. 22. Zhou M, Tse S, Derevianko A, Jones DB, Schwaitzberg SD, Cao CG. Effect of haptic feedback in laparoscopic surgery skill acquisition. Surg Endosc. 2012;26:1128-1134. 23. Chmarra MK, Dankelman J, van den Dobbelsteen JJ, Jansen F-W. Force feedback and basic laparoscopic skills. Surg Endosc. 2008;22:2140-2148. 24. Schout BM, Hendrikx AJ, Scheele F, Bemelmans BL, Scherpbier AJ. Validation and implementation of surgical
simulators: a critical review of present, past, and future. Surg Endosc. 2010;24:536-546. 25. Singapogu RB, Smith DE, Altenhoff BM, et al. Assessing surgeon and novice force skill on a haptic stiffness simulator for laparoscopic surgery. Stud Health Technol Inform. 2012;173:469-474. 26. Zhou M, Perreault J, Schwaitzberg S, Cao C. Effects of experience on force perception threshold in minimally invasive surgery. Surg Endosc. 2008;22:510-515. 27. Wagner CR, Stylopoulos N, Jackson PG, Howe RD. The benefit of force feedback in surgery: examination of blunt dissection. Presence Teleoperators Virtual Environ. 2007;16:252-262. 28. Forrest N, Baillie S, Kalita P, Tan HZ. A comparative study of haptic stiffness identification by veterinarians and students. IEEE Trans Haptics. 2011;4:78-87. 29. Long LO, Singapogu RB, Arcese G, et al. A haptic simulator to increase laparoscopic force application sensitivity. Stud Health Technol Inform. 2013;184:273-275. 30. Lederman SJ, Klatzky RL. Hand movements: a window into haptic object recognition. Cogn Psychol. 1987;19: 342-368.
Downloaded from sri.sagepub.com at UNIV OF WISCONSIN OSHKOSH on November 14, 2015
