Surg Endosc DOI 10.1007/s00464-014-3884-0
and Other Interventional Techniques
Current status of endoscopic simulation in gastroenterology fellowship training programs Pichamol Jirapinyo • Christopher C. Thompson
Received: 30 June 2014 / Accepted: 4 September 2014 Ó Springer Science+Business Media New York 2014
Abstract Background and aims Recent guidelines have encouraged gastroenterology and surgical training programs to integrate simulation into their core endoscopic curricula. However, the role that simulation currently has within training programs is unknown. This study aims to assess the current status of simulation among gastroenterology fellowship programs. Methods Design: This questionnaire study consisted of 38 fields divided into two sections. The first section queried program directors’ experience on simulation and assessed the current status of simulation at their institution. The second portion surveyed their opinion on the potential role of simulation on the training curriculum. Setting: The study was conducted at the 2013 American Gastroenterological Association Training Directors’ Workshop in Phoenix, Arizona. Participants: The participants were program directors from Accreditation Council for Graduate Medical Education accredited gastroenterology training programs, who attended the workshop. Results The questionnaire was returned by 69 of 97 program directors (response rate of 71 %). 42 % of programs had an endoscopic simulator. Computerized simulators (61.5 %) were the most common, followed by mechanical (30.8 %) and animal tissue (7.7 %) simulators, respectively. Eleven
P. Jirapinyo Department of Internal Medicine, Yale University School of Medicine, New Haven, USA C. C. Thompson (&) Division of Gastroenterology, Hepatology and Endoscopy, Brigham and Women’s Hospital, Boston, USA e-mail: [email protected] C. C. Thompson Harvard Medical School, Boston, USA
programs (15 %) required fellows to use simulation prior to clinical cases. Only one program has a minimum number of hours fellows have to participate in simulation training. Current simulators are deemed as easy to use (76 %) and good educational tools (65 %). Problems are cost (72 %) and accessibility (69 %). The majority of program directors believe that there is a need for endoscopic simulator training, with only 8 % disagreeing. Additionally, a majority believe there is a role for simulation prior to initiation of clinical cases with 15 % disagreeing. Conclusion Gastroenterology fellowship program directors widely recognize the importance of simulation. Nevertheless, simulation is used by only 42 % of programs and only 15 % of programs require that trainees use simulation prior to clinical cases. No programs currently use simulation as part of the evaluation process. Keywords Education Training Simulation Endoscopy Traditionally, training in endoscopy has been based upon an apprenticeship model, with trainees learning endoscopy under the supervision of experienced endoscopists in the clinical setting. The process involves a progression with trainees initially observing procedures, then attempting less technically demanding aspects of procedures, then performing entire procedures. In most programs, trainees learn basic scope manipulation and practice fundamental techniques during initial human cases [1]. This process prolongs procedure time and may lead to increased patient risk and discomfort. Additionally, mental workload has been shown to increase substantially when a trainee is required to learn a new tool at the same time as they are learning a new procedure that would be performed with that tool [2].
123
Surg Endosc
In an attempt to shorten the learning curve, a wide variety of endoscopic simulators has been developed over the past decade [3–16]. These include mechanical, explanted animal tissue, and computerized models. With simulation, trainees would be able to acquire basic skills through sustained deliberate practice without jeopardizing patient care. To date, a few preliminary studies have confirmed the benefits of simulation on improving endoscopic skills especially in the early stages of training [17–22]. Recently, the American Board of Surgery (ABS) has adopted and mandated all surgical residency programs to use the Flexible Endoscopic Curriculum (FEC). The curriculum is a stepwise instructional program that provides cognitive and technical milestones for each of the five surgical postgraduate years. In addition, all graduating surgical residents are required to pass the Fundamentals of Endoscopic Surgery (FES), which is the final milestone in the FEC, prior to applying for ABS certification. The FES program includes a web-based didactic lecture, a written multiple choice test on cognitive aspects of endoscopy, and also a virtual reality skills exam to assess hands-on technical skills [23–25]. While surgical residency programs are moving toward requiring trainees to use simulation as a part of longitudinal endoscopic education and have mandated it for certification testing, the role that simulation plays in endoscopy education in gastroenterology fellowship training programs remains unknown. This study aims to evaluate the current status of endoscopic simulation among gastroenterology fellowship programs in the United States. Additionally, program directors’ attitudes and opinions on simulation are assessed.
Table 1 Training program demographics
N (%) Geographic location Northeast
27 (39)
Midwest
15 (22)
South
15 (22)
West
11 (16)
Type of training program Academic
57 (83)
Community
9 (13)
Table 2 Simulation experience N Yes (% Yes) Did you (program director) Use simulators for preclinical instruction in medical school?
5 (7)
Use simulators to learn clinical skills during training?
22 (32)
Get tested on a simulator during training?
15 (22)
Use or try the current endoscopic simulator at your institution?a
27 (93)
Does your program have an endoscopic simulator? Computerized simulator
b
7
AccuTouch
2
Colonoscope training model
123
7 8 (30.8) 3
Erlanger plastic mannequin
2
Upper GI trainer
1
Others
2
Explanted animal tissue simulatorb
The study was conducted at the Annual American Gastroenterological Association Training Directors’ Workshop in Phoenix, Arizona, in February 2013. Program directors and associate program directors from 97 training programs, who attended the workshop, were asked to participate in the questionnaire study. The questionnaire consisted of two parts—the first focusing on program demographics, structure, and direct experience with current simulators, and the second focusing on training curriculum and opinions regarding simulation. Specifically, the first part assessed how many training programs use simulation and how it is employed. Additionally, perceived benefits and limitations of the current simulators were assessed. For programs that do not have a simulator, limiting factors were queried. The second part of the questionnaire surveyed program directors on the potential role that simulation may play in the training curriculum. They were asked to grade their agreement to statements regarding the impact of simulation on a 5-point Likert scale (1-strongly disagree, 3-neutral, 5-strongly agree).
16 (61.5)
GI mentor Others Mechanical simulatorb
Materials and methods
29 (42)
2 (7.7)
EASIE
1
Others
1
No response to what kind of endoscopic simulator
3 (–)
a
27 out of 29 program directors from the programs that have a simulator reported having used the current simulator at their institution
b
Percentage of each simulator model compared to a total of 26 programs reporting simulator model
Data were analyzed using Microsoft Excel 2011. The terms strongly agree and agree were combined into a percentage of agreement to each statement. Results Part 1: Program characteristics and simulation experience A total of 69 questionnaires were returned for a response rate of 71 % (69 of the 97 program directors attending the
Surg Endosc Fig. 1 Types of endoscopic simulators a GI mentor, computerized virtual reality simulator (Simbionix, Cleveland, OH). Taken from http://simbionix.com/simulators/ gi-bronch-gi-mentor/. b AccuTouch, computerized virtual reality simulator (Immersion, now CAE Healthcare, Sarasota, FL). Taken from https://depts. washington.edu/biointel/immer sion.ppt c and d Colonoscope Training Model, mechanical simulator (Kyoto Kagaku, Kyoto, Japan). Taken from https://www.3-dmed.com/sites/ default/files/product-additional/ productspec/Colonoscopy% 20Trainer%20Product%20Cata log.pdf e and f EASIE, animal tissue simulator (Endosim, Hudson, MA). Taken from http://www.endosim.com/his tory.html
meeting). Demographics of the training programs are demonstrated in Table 1. Each program has a median of 3 fellows [range 1–12] per academic year. On average, fellows in the clinical track have 1, 4, and 3 months of protected research time during their first, second, and third year, respectively. In comparison, fellows in the research track have an average of 2, 9, and 7 months of protected research time during their first, second, and third year, respectively. Most program directors reported that they did not have prior exposure to simulation during their own clinical training (Table 2). However, almost all of them have used the endoscopic simulator at their current institution.
Out of the 69 training programs that responded, 42 % (29 of 69) have an endoscopic simulator (Table 2). Of these, computerized simulators (61.5 %) are the most common, followed by mechanical (30.8 %) and animal tissue (7.7 %) simulators, respectively (Fig. 1). Eleven programs (15.9 %) require fellows to use simulation prior to their initial clinical case. However, only one program requires a minimum number of hours, or threshold of competence, be reached on a simulator prior to initiation of clinical cases. Seven of the 29 programs with simulators (24 %) keep track of the number of hours fellows spend on simulation. On average, fellows spend about 0–5 h per month on the simulator. No programs have a minimum number of hours fellows need to spend on simulation
123
Surg Endosc Table 3 Fellow endoscopic competency evaluation % Yes Direct Observation of Procedural Skills Survey
69
Clinical parameters
24
Mayo Colonoscopy Skills Assessment Tool
7
Simulation based evaluation
0
Table 4 Advantages and disadvantages of the current endoscopic simulators % Yes % Program directors Believes simulator is easy to use
76
Believes simulator is a good educational tool
65
Believes simulator is realistic
45
Believes there is a perceived correlation between hours spent on simulator and endoscopic performance
10
Problems with existing simulator(s) High cost
72
Not easily accessible
69
Not validated Not useful
14 14
Complicated to set up or perform
10
Others
28
before graduating each fellowship year, nor is competence assessed via simulation. Instead, evaluation of their endoscopic skill and competency is based solely on observed clinical performance (Table 3). This is achieved using Direct Observation of Procedural Skills Survey, Mayo Colonoscopy Skills Assessment Tool, or other clinical parameters, such as cecal intubation rate and complication rates. The majority of program directors reported that their current simulator is easy to use and is a good educational tool (Table 4). The two most common problems identified by program directors are high cost and difficult access due to inconvenient simulator location. In a number of programs, the simulator is located in a simulation center, which limits fellow accessibility for a variety of reasons. Additionally, only 10 % of program directors believe that hours spent on current simulators correlate with endoscopic performance. Among the 40 training programs that do not have a simulator, the main limiting factor that prevents them from obtaining one is high cost (85 %), followed by perceived ineffectiveness (33 %), inaccessibility to fellows (5 %), unnecessary due to large program with high case load (3 %), and program too small to justify purchase (3 %).
123
Part 2: Opinions on simulation and its potential role on the training curriculum The majority of program directors believe that there is a need for endoscopic simulator training in addition to the clinical endoscopic experience, with only 8 % disagreeing (Table 5). Additionally, a majority believes that simulators allow the trainee to practice in a controlled environment and that there is a role for simulators prior to initiation of clinical cases. Most program directors agree that simulators cannot replace hands-on clinical experience, with only 11 % disagreeing. There was no census regarding simulator validity, cost-effectiveness, ability of simulators to assess technical skill, improvement in patient outcomes, and use in credentialing (Table 5). Overall, program directors from the institutions that have and do not have simulators have similar opinions regarding the role of endoscopic simulation (Table 5). One exception is their opinion regarding faculty support of simulator training. More program directors from institutions that do not have simulators believe that their faculty would support this type of training (66 % versus 41 %; p = 0.038).
Discussion Gastrointestinal endoscopy represents a common group of procedures with over 20 million procedures being performed annually in the United States [26]. A recent prospective study of outpatient endoscopies revealed that 1.07 % of upper endoscopies and 0.95 % of screening colonoscopies result in complications [27]. Adverse events are more common in therapeutic endoscopy with rates as high as 15 % for certain procedures, such as therapeutic endoscopic retrograde cholangiopancreatography (ERCP) [28, 29]. These findings highlight the need to improve patient safety during these common procedures. Endoscopy is a multifaceted procedure that requires a considerable amount of training to attain competence. Traditionally, endoscopic training has relied upon clinical experience with trainees learning the basic skills under the supervision of experienced preceptors. However, this type of training is challenging due to several reasons. There is an ‘‘all of none’’ phenomenon, which forces an instructor to give up complete control of the endoscope in order to allow the trainee to learn the technique [30]. In addition, appropriate concern for patient comfort and safety often impacts the learning experience. Furthermore, training opportunities are largely random and variable. Similar to endoscopic training, assessment of endoscopic ability remains challenging and is often subjective. To date, there is still no standardized methodology to
Surg Endosc Table 5 Opinions of program directors regarding simulation and its potential roles on the training curriculum
Bold numbers represent the most common response (agree versus neutral versus disagree) among (1) all programs (2) programs with simulators and (3) programs without simulators All Opinions of all program directors, W sim Opinions of the program directors from the programs that have a simulator, W/o sim Opinions of the program directors from the programs that do not have a simulation
% Agree
% Neutral
% Disagree
All
W sim
W/o sim
All
W sim
W/o sim
All
W sim
W/o sim
There is a need for technical skills training outside the endoscopy suite
66
63
68
26
30
24
8
7
8
Simulators have a role in preclinical endoscopic training
56
59
54
29
24
33
15
18
13
Simulator education can reduce patient risk and complications
33
30
36
41
37
44
26
33
21
Simulators can provide objective assessment ? measurement of technical skills
35
30
40
42
44
40
23
26
21
There is a role for endoscopic simulators in maintenance of skills
23
22
24
35
33
37
42
44
40
There is a role for endoscopic simulators in the credentialing process
18
19
18
33
30
36
49
52
46
Simulators will allow the learner to practice in a controlled environment
82
77
85
9
12
8
9
12
8
Simulators are NOT an adequate substitute for hands-on instruction
74
74
74
15
11
18
11
15
8
Simulators canNOT be easily incorporated into the current training curriculum
26
19
32
22
19
24
52
63
45
Simulators have NOT been adequately validated as an educational tool
39
41
38
44
33
51
17
26
11
Current simulators are NOT a cost-effective method of learning
34
26
40
48
52
45
19
22
16
Gastroenterology fellows would willingly participate in this type of training
75
70
79
17
15
18
8
15
3
Gastroenterology faculty would support this type of training
55
41
66
37
48
29
8
11
5
determine competence. In the past, the number of procedures performed was used as a surrogate for technical competence. However, the requisite number of procedures remains unknown and does not correlate with high-quality performance [31, 32]. Focus then shifted to general competence assessment, including tools such as Direct Observation of Procedural Skills Survey and the Mayo Colonoscopy Skills Assessment Tool. However, these tools remain subjective and often cumbersome, although they have proven value [33, 34]. In order to address the limitations associated with traditional endoscopic training and assessment, many forms of simulators have been developed over the past decade [35, 36]. As more surgical training programs are implementing simulation as part of the endoscopy curriculum and simulation testing has been mandated as part of the certification process [23–25], it is important to understand the role that simulation currently plays in gastroenterology fellowship training. In this study, we demonstrate the prevalence of endoscopic simulators among gastroenterology fellowship programs in the United States. The study also reveals the current status of simulator use and opinions
of program directors on potential roles of simulators in the fellowship training curriculum. The majority of program directors believe that there is a need for endoscopic simulator training with only 8 % disagreeing. However, only 42 % of the fellowship programs currently have a simulator. Among those, computerized simulation is the most commonly used type of simulation (61.5 % computerized versus 30.8 % mechanical versus 7.7 % animal explanted). Its main advantages include its capability to provide real-time feedback and a library of clinical cases with a variety of difficulty levels [30]. However, each simulator costs approximately $110,000 to $140,000 [37]. Additionally, they are bulky and usually located in a simulation center, which makes it difficult for gastroenterology fellows to access. Evidence to date only shows its benefit in the early phase of colonoscopy and hemostasis training without shortening the learning curve or improving patient outcomes [18, 21]. As demonstrated in our study, these factors are commonly cited limitations that prevent most programs from obtaining a simulator. The other two less commonly used simulator models include mechanical and explanted animal simulation.
123
Surg Endosc
While these types of simulation tend to be more affordable, to date, there is no evidence proving the benefit of mechanical simulators on endoscopic training. Additionally, a mechanical model provides no interaction between the simulator and the trainee. A mentor therefore must be present for initial use to guide the training. Animal models are difficult to prepare and use. They usually require special animal-use endoscopes, the procurement of animal organs, and extensive preparation and disposal processes. Due to these limitations, most fellowship programs do not favor these types of simulation. Newer mechanical simulators are now being developed to address these shortcomings [38]. The study also demonstrates the heterogeneity of how simulators are currently being employed. Specifically, while about two-thirds of program directors believe that simulation should be used in the preclinical stage, only 15 % of training programs require fellows to use simulation prior to initiation of clinical cases. Among these, only one program has a minimum number of hours fellows have to spend on a simulator. Similarly, program directors widely recognize the role that simulation may play on skill assessment. However, no training programs currently use simulation to evaluate fellows’ endoscopic performance. Instead, the method of evaluation remains diverse and based on subjective data. These include the use of Direct Observation of Procedural Skills Surveys, which is a validated tool for assessing procedural skills and used in approximately 67 % of training programs [32]. However, this survey has never been specifically validated for evaluation of endoscopy skills. While Mayo Colonoscopy Skills Assessment Tool is a more targeted tool for endoscopic skill assessment and has proven validity, it remains cumbersome and somewhat subjective. Overall, opinions on the role of simulators do not vary substantially between programs. An exception includes the fact that more programs that do not have a simulator believe that their faculty would support using a simulator to train endoscopy. This confirms that the main limiting factor that prevents the programs from obtaining a simulator is not their perception of the usefulness of simulation. Instead, cost and access are the major limitations as demonstrated in the study. There is no census regarding simulator validity, costeffectiveness, ability of simulators to assess technical skill, improvement in patient outcomes, and use in credentialing. Further study is warranted to settle these controversial issues. Specifically, an adequately powered, controlled trial is needed to demonstrate the benefit of incorporating simulation into a training curriculum. Development of a more affordable yet effective simulator and a study to prove its validation is also needed. Lastly, for a simulator to be a good skill assessment and/or credentialing tool, a study to
123
correlate simulation performance to clinical performance, such as cecal intubation rate or adenoma detection rates, is warranted. Limitations of this study are its response rate and the program directors’ relatively limited experience with simulation. While the response rate is less than 100 %, programs from all regions of the United States are adequately represented. The majority of program directors reported having limited exposure to simulation during their medical training. However, almost all of them have used the endoscopic simulator at their current institution. As simulation is becoming more integral to endoscopy education, different types of simulators will likely be needed. Simulators may be particularly useful in teaching less common procedures, such as variceal banding or stricture dilatation. They also may be useful in maintaining skills for practicing physicians who do less endoscopy. It is also important that different simulators are used to teach and to formally assess endoscopic skills. If a subject is trained and formally assessed on the same simulator, the result will be uninterpretable. In summary, gastroenterology fellowship program directors widely recognize that simulation is important and should be integrated into the training curriculum. However, less than half of the fellowship programs currently use simulation to train fellows, with very few requiring simulation work prior to clinical cases, and none utilizing simulation as part of the assessment process. Major limitations to implementation of endoscopic simulation are cost and accessibility. For broader adoption within programs, an affordable, validated, and easily accessible endoscopic simulator is needed. Acknowledgments Materials and financial support were provided by Harvard Digestive Diseases Center at Harvard Medical School (DK034854). Disclosures Dr. Jirapinyo has no conflicts of interest or financial ties to disclose. Dr. Thompson is a Consultant and has received Research Support from Olympus (not related to this project).
References 1. ASGE Training Committee (2012) Principles of training in GI endoscopy. Gastrointest Endosc 75(2):231–235 2. Carswell CM, Clarke D, Seales WB (2005) Assessing mental demand during laparoscopic surgery. Surg Innov 12(1):80–90 3. Greenwald D, Cohen J (2006) Evolution of endoscopy simulators and their application. Gastrointest Endosc Clin N Am 16:389–406 4. Classen M, Ruppin H (1974) Practical training using a new gastrointestinal phantom. Endoscopy 6:127–131 5. Williams CB, Baillie J, Gillies DF, Borislow D, Cotton PB (1990) Teaching gastrointestinal endoscopy by computer simulation: a prototype for colonoscopy and ERCP. Gastrointest Endosc 36:49–54
Surg Endosc 6. Noar MD (1992) Robotics interactive endoscopy simulation of ERCP/sphincterotomy and EGD. Endoscopy 24(Suppl 2): 539–541 7. Moorthy K, Munz Y, Orchard TR, Gould S, Rockall T, Darzi A (2004) An innovative method for the assessment of skills in lower gastrointestinal endoscopy. Surg Endosc 18:1613–1619 8. Mahmood T, Darzi A (2003) A study to validate the colonoscopy simulator. Surg Endosc 17:1583–1589 9. Datta Vk, Mandalia M, Mackay SD, Darzi AW (2001) Evaluation and validation of a virtual reality based flexible sigmoidoscopy trainer. Gut 48:A97–A98 10. Grantcharov TP, Carstensen L, Schulze S (2005) Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS 9:130–133 11. Ferlitsch A, Glauninger P, Gupper A et al (2002) Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy 3:698–702 12. Ritter EM, McClusky DA 3rd, Lederman AB, Gallagher AG, Smith CD (2003) Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg 7:871–877; discussion 877–878 13. Felsher JJ, Olesevich M, Farres H et al (2005) Validation of a flexible endoscopy simulator. Am J Surg 189:497–500 14. Moorthy K, Munz Y, Jiwanji M. Bann S, Chang A, Darzi A (2004) Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surg Endosc 18:328–333 15. Leung J et al (2012) Head-to-head comparison of practice with endoscopic retrograde cholangiopancreatography computer and mechanical simulators by experienced endoscopists and trainees. Dig Endosc 24(3):175–181 16. Cohen J, Thompson CC (2013) The next generation of endoscopic simulation. Am J Gastroenterol 108(7):1036–1039 17. Sedlack RE, Kolars JC (2004) Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 99:33–37 18. Di Guiulio E, Fregonese D, Casetti T et al (2004) Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: a randomized controlled trial. Gastrointest Endosc 60:196–200 19. Cohen J, Cohen SA, Vora KC et al (2006) Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 64:361–368 20. Gerson LB (2006) Evidence-based assessment of endoscopic simulators for training. Gastrointest Endosc Clin N Am 16:489–509, vii–viii 21. Shirai Y, Yoshida T, Shiraishi R et al (2008) Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol 23:1046–1050
22. Haycock AV, Youd P, Bassett P et al (2009) Simulator training improves practical skills in therapeutic GI endoscopy: results from a randomized, blinded, controlled study. Gastrointest Endosc 70:835–845 23. Hazey JW, Marks JM, Mellinger JD et al (2014) Why fundamentals of endoscopic surgery (FES)? Surg Endosc 28(3):701–703 24. Poulose BK, Vassilliou MC, Dunkin BJ et al (2014) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc 28(2):631–638 25. Vassilliou MC, Dunkin BJ, Fried GM et al (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28(3):704–711 26. Cherry DK, Hing E, Woodwell DA et al (2008) National Ambulatory Medical Survey: 2006 summary. Natl Health Stat Rep 6(3):1–39 27. Leffler DA, Kheraj R, Garud S et al (2010) The incidence and cost of unexpected hospital use after scheduled outpatient endoscopy. Arch Intern Med 170(19):1752–1757 28. Mallery JS, Baron TH, Dominitz JA et al (2003) Standards of Practice Committee. American Society for Gastrointestinal Endoscopy. Complications of ERCP. Gastrointest Endosc 57(6):633–638 29. Freeman ML, Nelson DB, Sherman S, Haber GB, Herman ME, Dorsher PJ, Moore JP, Fennerty MB, Ryan ME, Shaw MJ, Lande JD, Pheley AM (1996) Complications of endoscopic biliary sphincterotomy. N Engl J Med 335(13):909–918 30. Dunkin BJ (2003) Flexible endoscopy simulators. Semin Laparosc Surg 10(1):29–35 31. Spier BJ, Benson M, Pfau PR et al (2010) Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 71:319–324 32. Sedlack RE (2011) Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 74:355–366 33. Naeem N (2013) Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). J Coll Physicians Surg Pak. 23(1):77–82 34. Sedlack PE (2010) The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 72:1125–1133 35. Issenberg SB et al (1999) Simulation technology for health care professional skills training and assessment. JAMA 282(9):861–866 36. Issenberg SB et al (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28 37. ASGE Training Committee (2011) Endoscopic simulators. Gastrointest Endosc 73(5):861–867 38. Thompson CC, Jirapinyo P, Kumar N et al (2014) Development and initial validation of endoscopic part-task training box. Endoscopy 46(9):735–744
123
and Other Interventional Techniques
Current status of endoscopic simulation in gastroenterology fellowship training programs Pichamol Jirapinyo • Christopher C. Thompson
Received: 30 June 2014 / Accepted: 4 September 2014 Ó Springer Science+Business Media New York 2014
Abstract Background and aims Recent guidelines have encouraged gastroenterology and surgical training programs to integrate simulation into their core endoscopic curricula. However, the role that simulation currently has within training programs is unknown. This study aims to assess the current status of simulation among gastroenterology fellowship programs. Methods Design: This questionnaire study consisted of 38 fields divided into two sections. The first section queried program directors’ experience on simulation and assessed the current status of simulation at their institution. The second portion surveyed their opinion on the potential role of simulation on the training curriculum. Setting: The study was conducted at the 2013 American Gastroenterological Association Training Directors’ Workshop in Phoenix, Arizona. Participants: The participants were program directors from Accreditation Council for Graduate Medical Education accredited gastroenterology training programs, who attended the workshop. Results The questionnaire was returned by 69 of 97 program directors (response rate of 71 %). 42 % of programs had an endoscopic simulator. Computerized simulators (61.5 %) were the most common, followed by mechanical (30.8 %) and animal tissue (7.7 %) simulators, respectively. Eleven
P. Jirapinyo Department of Internal Medicine, Yale University School of Medicine, New Haven, USA C. C. Thompson (&) Division of Gastroenterology, Hepatology and Endoscopy, Brigham and Women’s Hospital, Boston, USA e-mail: [email protected] C. C. Thompson Harvard Medical School, Boston, USA
programs (15 %) required fellows to use simulation prior to clinical cases. Only one program has a minimum number of hours fellows have to participate in simulation training. Current simulators are deemed as easy to use (76 %) and good educational tools (65 %). Problems are cost (72 %) and accessibility (69 %). The majority of program directors believe that there is a need for endoscopic simulator training, with only 8 % disagreeing. Additionally, a majority believe there is a role for simulation prior to initiation of clinical cases with 15 % disagreeing. Conclusion Gastroenterology fellowship program directors widely recognize the importance of simulation. Nevertheless, simulation is used by only 42 % of programs and only 15 % of programs require that trainees use simulation prior to clinical cases. No programs currently use simulation as part of the evaluation process. Keywords Education Training Simulation Endoscopy Traditionally, training in endoscopy has been based upon an apprenticeship model, with trainees learning endoscopy under the supervision of experienced endoscopists in the clinical setting. The process involves a progression with trainees initially observing procedures, then attempting less technically demanding aspects of procedures, then performing entire procedures. In most programs, trainees learn basic scope manipulation and practice fundamental techniques during initial human cases [1]. This process prolongs procedure time and may lead to increased patient risk and discomfort. Additionally, mental workload has been shown to increase substantially when a trainee is required to learn a new tool at the same time as they are learning a new procedure that would be performed with that tool [2].
123
Surg Endosc
In an attempt to shorten the learning curve, a wide variety of endoscopic simulators has been developed over the past decade [3–16]. These include mechanical, explanted animal tissue, and computerized models. With simulation, trainees would be able to acquire basic skills through sustained deliberate practice without jeopardizing patient care. To date, a few preliminary studies have confirmed the benefits of simulation on improving endoscopic skills especially in the early stages of training [17–22]. Recently, the American Board of Surgery (ABS) has adopted and mandated all surgical residency programs to use the Flexible Endoscopic Curriculum (FEC). The curriculum is a stepwise instructional program that provides cognitive and technical milestones for each of the five surgical postgraduate years. In addition, all graduating surgical residents are required to pass the Fundamentals of Endoscopic Surgery (FES), which is the final milestone in the FEC, prior to applying for ABS certification. The FES program includes a web-based didactic lecture, a written multiple choice test on cognitive aspects of endoscopy, and also a virtual reality skills exam to assess hands-on technical skills [23–25]. While surgical residency programs are moving toward requiring trainees to use simulation as a part of longitudinal endoscopic education and have mandated it for certification testing, the role that simulation plays in endoscopy education in gastroenterology fellowship training programs remains unknown. This study aims to evaluate the current status of endoscopic simulation among gastroenterology fellowship programs in the United States. Additionally, program directors’ attitudes and opinions on simulation are assessed.
Table 1 Training program demographics
N (%) Geographic location Northeast
27 (39)
Midwest
15 (22)
South
15 (22)
West
11 (16)
Type of training program Academic
57 (83)
Community
9 (13)
Table 2 Simulation experience N Yes (% Yes) Did you (program director) Use simulators for preclinical instruction in medical school?
5 (7)
Use simulators to learn clinical skills during training?
22 (32)
Get tested on a simulator during training?
15 (22)
Use or try the current endoscopic simulator at your institution?a
27 (93)
Does your program have an endoscopic simulator? Computerized simulator
b
7
AccuTouch
2
Colonoscope training model
123
7 8 (30.8) 3
Erlanger plastic mannequin
2
Upper GI trainer
1
Others
2
Explanted animal tissue simulatorb
The study was conducted at the Annual American Gastroenterological Association Training Directors’ Workshop in Phoenix, Arizona, in February 2013. Program directors and associate program directors from 97 training programs, who attended the workshop, were asked to participate in the questionnaire study. The questionnaire consisted of two parts—the first focusing on program demographics, structure, and direct experience with current simulators, and the second focusing on training curriculum and opinions regarding simulation. Specifically, the first part assessed how many training programs use simulation and how it is employed. Additionally, perceived benefits and limitations of the current simulators were assessed. For programs that do not have a simulator, limiting factors were queried. The second part of the questionnaire surveyed program directors on the potential role that simulation may play in the training curriculum. They were asked to grade their agreement to statements regarding the impact of simulation on a 5-point Likert scale (1-strongly disagree, 3-neutral, 5-strongly agree).
16 (61.5)
GI mentor Others Mechanical simulatorb
Materials and methods
29 (42)
2 (7.7)
EASIE
1
Others
1
No response to what kind of endoscopic simulator
3 (–)
a
27 out of 29 program directors from the programs that have a simulator reported having used the current simulator at their institution
b
Percentage of each simulator model compared to a total of 26 programs reporting simulator model
Data were analyzed using Microsoft Excel 2011. The terms strongly agree and agree were combined into a percentage of agreement to each statement. Results Part 1: Program characteristics and simulation experience A total of 69 questionnaires were returned for a response rate of 71 % (69 of the 97 program directors attending the
Surg Endosc Fig. 1 Types of endoscopic simulators a GI mentor, computerized virtual reality simulator (Simbionix, Cleveland, OH). Taken from http://simbionix.com/simulators/ gi-bronch-gi-mentor/. b AccuTouch, computerized virtual reality simulator (Immersion, now CAE Healthcare, Sarasota, FL). Taken from https://depts. washington.edu/biointel/immer sion.ppt c and d Colonoscope Training Model, mechanical simulator (Kyoto Kagaku, Kyoto, Japan). Taken from https://www.3-dmed.com/sites/ default/files/product-additional/ productspec/Colonoscopy% 20Trainer%20Product%20Cata log.pdf e and f EASIE, animal tissue simulator (Endosim, Hudson, MA). Taken from http://www.endosim.com/his tory.html
meeting). Demographics of the training programs are demonstrated in Table 1. Each program has a median of 3 fellows [range 1–12] per academic year. On average, fellows in the clinical track have 1, 4, and 3 months of protected research time during their first, second, and third year, respectively. In comparison, fellows in the research track have an average of 2, 9, and 7 months of protected research time during their first, second, and third year, respectively. Most program directors reported that they did not have prior exposure to simulation during their own clinical training (Table 2). However, almost all of them have used the endoscopic simulator at their current institution.
Out of the 69 training programs that responded, 42 % (29 of 69) have an endoscopic simulator (Table 2). Of these, computerized simulators (61.5 %) are the most common, followed by mechanical (30.8 %) and animal tissue (7.7 %) simulators, respectively (Fig. 1). Eleven programs (15.9 %) require fellows to use simulation prior to their initial clinical case. However, only one program requires a minimum number of hours, or threshold of competence, be reached on a simulator prior to initiation of clinical cases. Seven of the 29 programs with simulators (24 %) keep track of the number of hours fellows spend on simulation. On average, fellows spend about 0–5 h per month on the simulator. No programs have a minimum number of hours fellows need to spend on simulation
123
Surg Endosc Table 3 Fellow endoscopic competency evaluation % Yes Direct Observation of Procedural Skills Survey
69
Clinical parameters
24
Mayo Colonoscopy Skills Assessment Tool
7
Simulation based evaluation
0
Table 4 Advantages and disadvantages of the current endoscopic simulators % Yes % Program directors Believes simulator is easy to use
76
Believes simulator is a good educational tool
65
Believes simulator is realistic
45
Believes there is a perceived correlation between hours spent on simulator and endoscopic performance
10
Problems with existing simulator(s) High cost
72
Not easily accessible
69
Not validated Not useful
14 14
Complicated to set up or perform
10
Others
28
before graduating each fellowship year, nor is competence assessed via simulation. Instead, evaluation of their endoscopic skill and competency is based solely on observed clinical performance (Table 3). This is achieved using Direct Observation of Procedural Skills Survey, Mayo Colonoscopy Skills Assessment Tool, or other clinical parameters, such as cecal intubation rate and complication rates. The majority of program directors reported that their current simulator is easy to use and is a good educational tool (Table 4). The two most common problems identified by program directors are high cost and difficult access due to inconvenient simulator location. In a number of programs, the simulator is located in a simulation center, which limits fellow accessibility for a variety of reasons. Additionally, only 10 % of program directors believe that hours spent on current simulators correlate with endoscopic performance. Among the 40 training programs that do not have a simulator, the main limiting factor that prevents them from obtaining one is high cost (85 %), followed by perceived ineffectiveness (33 %), inaccessibility to fellows (5 %), unnecessary due to large program with high case load (3 %), and program too small to justify purchase (3 %).
123
Part 2: Opinions on simulation and its potential role on the training curriculum The majority of program directors believe that there is a need for endoscopic simulator training in addition to the clinical endoscopic experience, with only 8 % disagreeing (Table 5). Additionally, a majority believes that simulators allow the trainee to practice in a controlled environment and that there is a role for simulators prior to initiation of clinical cases. Most program directors agree that simulators cannot replace hands-on clinical experience, with only 11 % disagreeing. There was no census regarding simulator validity, cost-effectiveness, ability of simulators to assess technical skill, improvement in patient outcomes, and use in credentialing (Table 5). Overall, program directors from the institutions that have and do not have simulators have similar opinions regarding the role of endoscopic simulation (Table 5). One exception is their opinion regarding faculty support of simulator training. More program directors from institutions that do not have simulators believe that their faculty would support this type of training (66 % versus 41 %; p = 0.038).
Discussion Gastrointestinal endoscopy represents a common group of procedures with over 20 million procedures being performed annually in the United States [26]. A recent prospective study of outpatient endoscopies revealed that 1.07 % of upper endoscopies and 0.95 % of screening colonoscopies result in complications [27]. Adverse events are more common in therapeutic endoscopy with rates as high as 15 % for certain procedures, such as therapeutic endoscopic retrograde cholangiopancreatography (ERCP) [28, 29]. These findings highlight the need to improve patient safety during these common procedures. Endoscopy is a multifaceted procedure that requires a considerable amount of training to attain competence. Traditionally, endoscopic training has relied upon clinical experience with trainees learning the basic skills under the supervision of experienced preceptors. However, this type of training is challenging due to several reasons. There is an ‘‘all of none’’ phenomenon, which forces an instructor to give up complete control of the endoscope in order to allow the trainee to learn the technique [30]. In addition, appropriate concern for patient comfort and safety often impacts the learning experience. Furthermore, training opportunities are largely random and variable. Similar to endoscopic training, assessment of endoscopic ability remains challenging and is often subjective. To date, there is still no standardized methodology to
Surg Endosc Table 5 Opinions of program directors regarding simulation and its potential roles on the training curriculum
Bold numbers represent the most common response (agree versus neutral versus disagree) among (1) all programs (2) programs with simulators and (3) programs without simulators All Opinions of all program directors, W sim Opinions of the program directors from the programs that have a simulator, W/o sim Opinions of the program directors from the programs that do not have a simulation
% Agree
% Neutral
% Disagree
All
W sim
W/o sim
All
W sim
W/o sim
All
W sim
W/o sim
There is a need for technical skills training outside the endoscopy suite
66
63
68
26
30
24
8
7
8
Simulators have a role in preclinical endoscopic training
56
59
54
29
24
33
15
18
13
Simulator education can reduce patient risk and complications
33
30
36
41
37
44
26
33
21
Simulators can provide objective assessment ? measurement of technical skills
35
30
40
42
44
40
23
26
21
There is a role for endoscopic simulators in maintenance of skills
23
22
24
35
33
37
42
44
40
There is a role for endoscopic simulators in the credentialing process
18
19
18
33
30
36
49
52
46
Simulators will allow the learner to practice in a controlled environment
82
77
85
9
12
8
9
12
8
Simulators are NOT an adequate substitute for hands-on instruction
74
74
74
15
11
18
11
15
8
Simulators canNOT be easily incorporated into the current training curriculum
26
19
32
22
19
24
52
63
45
Simulators have NOT been adequately validated as an educational tool
39
41
38
44
33
51
17
26
11
Current simulators are NOT a cost-effective method of learning
34
26
40
48
52
45
19
22
16
Gastroenterology fellows would willingly participate in this type of training
75
70
79
17
15
18
8
15
3
Gastroenterology faculty would support this type of training
55
41
66
37
48
29
8
11
5
determine competence. In the past, the number of procedures performed was used as a surrogate for technical competence. However, the requisite number of procedures remains unknown and does not correlate with high-quality performance [31, 32]. Focus then shifted to general competence assessment, including tools such as Direct Observation of Procedural Skills Survey and the Mayo Colonoscopy Skills Assessment Tool. However, these tools remain subjective and often cumbersome, although they have proven value [33, 34]. In order to address the limitations associated with traditional endoscopic training and assessment, many forms of simulators have been developed over the past decade [35, 36]. As more surgical training programs are implementing simulation as part of the endoscopy curriculum and simulation testing has been mandated as part of the certification process [23–25], it is important to understand the role that simulation currently plays in gastroenterology fellowship training. In this study, we demonstrate the prevalence of endoscopic simulators among gastroenterology fellowship programs in the United States. The study also reveals the current status of simulator use and opinions
of program directors on potential roles of simulators in the fellowship training curriculum. The majority of program directors believe that there is a need for endoscopic simulator training with only 8 % disagreeing. However, only 42 % of the fellowship programs currently have a simulator. Among those, computerized simulation is the most commonly used type of simulation (61.5 % computerized versus 30.8 % mechanical versus 7.7 % animal explanted). Its main advantages include its capability to provide real-time feedback and a library of clinical cases with a variety of difficulty levels [30]. However, each simulator costs approximately $110,000 to $140,000 [37]. Additionally, they are bulky and usually located in a simulation center, which makes it difficult for gastroenterology fellows to access. Evidence to date only shows its benefit in the early phase of colonoscopy and hemostasis training without shortening the learning curve or improving patient outcomes [18, 21]. As demonstrated in our study, these factors are commonly cited limitations that prevent most programs from obtaining a simulator. The other two less commonly used simulator models include mechanical and explanted animal simulation.
123
Surg Endosc
While these types of simulation tend to be more affordable, to date, there is no evidence proving the benefit of mechanical simulators on endoscopic training. Additionally, a mechanical model provides no interaction between the simulator and the trainee. A mentor therefore must be present for initial use to guide the training. Animal models are difficult to prepare and use. They usually require special animal-use endoscopes, the procurement of animal organs, and extensive preparation and disposal processes. Due to these limitations, most fellowship programs do not favor these types of simulation. Newer mechanical simulators are now being developed to address these shortcomings [38]. The study also demonstrates the heterogeneity of how simulators are currently being employed. Specifically, while about two-thirds of program directors believe that simulation should be used in the preclinical stage, only 15 % of training programs require fellows to use simulation prior to initiation of clinical cases. Among these, only one program has a minimum number of hours fellows have to spend on a simulator. Similarly, program directors widely recognize the role that simulation may play on skill assessment. However, no training programs currently use simulation to evaluate fellows’ endoscopic performance. Instead, the method of evaluation remains diverse and based on subjective data. These include the use of Direct Observation of Procedural Skills Surveys, which is a validated tool for assessing procedural skills and used in approximately 67 % of training programs [32]. However, this survey has never been specifically validated for evaluation of endoscopy skills. While Mayo Colonoscopy Skills Assessment Tool is a more targeted tool for endoscopic skill assessment and has proven validity, it remains cumbersome and somewhat subjective. Overall, opinions on the role of simulators do not vary substantially between programs. An exception includes the fact that more programs that do not have a simulator believe that their faculty would support using a simulator to train endoscopy. This confirms that the main limiting factor that prevents the programs from obtaining a simulator is not their perception of the usefulness of simulation. Instead, cost and access are the major limitations as demonstrated in the study. There is no census regarding simulator validity, costeffectiveness, ability of simulators to assess technical skill, improvement in patient outcomes, and use in credentialing. Further study is warranted to settle these controversial issues. Specifically, an adequately powered, controlled trial is needed to demonstrate the benefit of incorporating simulation into a training curriculum. Development of a more affordable yet effective simulator and a study to prove its validation is also needed. Lastly, for a simulator to be a good skill assessment and/or credentialing tool, a study to
123
correlate simulation performance to clinical performance, such as cecal intubation rate or adenoma detection rates, is warranted. Limitations of this study are its response rate and the program directors’ relatively limited experience with simulation. While the response rate is less than 100 %, programs from all regions of the United States are adequately represented. The majority of program directors reported having limited exposure to simulation during their medical training. However, almost all of them have used the endoscopic simulator at their current institution. As simulation is becoming more integral to endoscopy education, different types of simulators will likely be needed. Simulators may be particularly useful in teaching less common procedures, such as variceal banding or stricture dilatation. They also may be useful in maintaining skills for practicing physicians who do less endoscopy. It is also important that different simulators are used to teach and to formally assess endoscopic skills. If a subject is trained and formally assessed on the same simulator, the result will be uninterpretable. In summary, gastroenterology fellowship program directors widely recognize that simulation is important and should be integrated into the training curriculum. However, less than half of the fellowship programs currently use simulation to train fellows, with very few requiring simulation work prior to clinical cases, and none utilizing simulation as part of the assessment process. Major limitations to implementation of endoscopic simulation are cost and accessibility. For broader adoption within programs, an affordable, validated, and easily accessible endoscopic simulator is needed. Acknowledgments Materials and financial support were provided by Harvard Digestive Diseases Center at Harvard Medical School (DK034854). Disclosures Dr. Jirapinyo has no conflicts of interest or financial ties to disclose. Dr. Thompson is a Consultant and has received Research Support from Olympus (not related to this project).
References 1. ASGE Training Committee (2012) Principles of training in GI endoscopy. Gastrointest Endosc 75(2):231–235 2. Carswell CM, Clarke D, Seales WB (2005) Assessing mental demand during laparoscopic surgery. Surg Innov 12(1):80–90 3. Greenwald D, Cohen J (2006) Evolution of endoscopy simulators and their application. Gastrointest Endosc Clin N Am 16:389–406 4. Classen M, Ruppin H (1974) Practical training using a new gastrointestinal phantom. Endoscopy 6:127–131 5. Williams CB, Baillie J, Gillies DF, Borislow D, Cotton PB (1990) Teaching gastrointestinal endoscopy by computer simulation: a prototype for colonoscopy and ERCP. Gastrointest Endosc 36:49–54
Surg Endosc 6. Noar MD (1992) Robotics interactive endoscopy simulation of ERCP/sphincterotomy and EGD. Endoscopy 24(Suppl 2): 539–541 7. Moorthy K, Munz Y, Orchard TR, Gould S, Rockall T, Darzi A (2004) An innovative method for the assessment of skills in lower gastrointestinal endoscopy. Surg Endosc 18:1613–1619 8. Mahmood T, Darzi A (2003) A study to validate the colonoscopy simulator. Surg Endosc 17:1583–1589 9. Datta Vk, Mandalia M, Mackay SD, Darzi AW (2001) Evaluation and validation of a virtual reality based flexible sigmoidoscopy trainer. Gut 48:A97–A98 10. Grantcharov TP, Carstensen L, Schulze S (2005) Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS 9:130–133 11. Ferlitsch A, Glauninger P, Gupper A et al (2002) Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy 3:698–702 12. Ritter EM, McClusky DA 3rd, Lederman AB, Gallagher AG, Smith CD (2003) Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg 7:871–877; discussion 877–878 13. Felsher JJ, Olesevich M, Farres H et al (2005) Validation of a flexible endoscopy simulator. Am J Surg 189:497–500 14. Moorthy K, Munz Y, Jiwanji M. Bann S, Chang A, Darzi A (2004) Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback. Surg Endosc 18:328–333 15. Leung J et al (2012) Head-to-head comparison of practice with endoscopic retrograde cholangiopancreatography computer and mechanical simulators by experienced endoscopists and trainees. Dig Endosc 24(3):175–181 16. Cohen J, Thompson CC (2013) The next generation of endoscopic simulation. Am J Gastroenterol 108(7):1036–1039 17. Sedlack RE, Kolars JC (2004) Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 99:33–37 18. Di Guiulio E, Fregonese D, Casetti T et al (2004) Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: a randomized controlled trial. Gastrointest Endosc 60:196–200 19. Cohen J, Cohen SA, Vora KC et al (2006) Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 64:361–368 20. Gerson LB (2006) Evidence-based assessment of endoscopic simulators for training. Gastrointest Endosc Clin N Am 16:489–509, vii–viii 21. Shirai Y, Yoshida T, Shiraishi R et al (2008) Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol 23:1046–1050
22. Haycock AV, Youd P, Bassett P et al (2009) Simulator training improves practical skills in therapeutic GI endoscopy: results from a randomized, blinded, controlled study. Gastrointest Endosc 70:835–845 23. Hazey JW, Marks JM, Mellinger JD et al (2014) Why fundamentals of endoscopic surgery (FES)? Surg Endosc 28(3):701–703 24. Poulose BK, Vassilliou MC, Dunkin BJ et al (2014) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc 28(2):631–638 25. Vassilliou MC, Dunkin BJ, Fried GM et al (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28(3):704–711 26. Cherry DK, Hing E, Woodwell DA et al (2008) National Ambulatory Medical Survey: 2006 summary. Natl Health Stat Rep 6(3):1–39 27. Leffler DA, Kheraj R, Garud S et al (2010) The incidence and cost of unexpected hospital use after scheduled outpatient endoscopy. Arch Intern Med 170(19):1752–1757 28. Mallery JS, Baron TH, Dominitz JA et al (2003) Standards of Practice Committee. American Society for Gastrointestinal Endoscopy. Complications of ERCP. Gastrointest Endosc 57(6):633–638 29. Freeman ML, Nelson DB, Sherman S, Haber GB, Herman ME, Dorsher PJ, Moore JP, Fennerty MB, Ryan ME, Shaw MJ, Lande JD, Pheley AM (1996) Complications of endoscopic biliary sphincterotomy. N Engl J Med 335(13):909–918 30. Dunkin BJ (2003) Flexible endoscopy simulators. Semin Laparosc Surg 10(1):29–35 31. Spier BJ, Benson M, Pfau PR et al (2010) Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 71:319–324 32. Sedlack RE (2011) Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 74:355–366 33. Naeem N (2013) Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). J Coll Physicians Surg Pak. 23(1):77–82 34. Sedlack PE (2010) The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 72:1125–1133 35. Issenberg SB et al (1999) Simulation technology for health care professional skills training and assessment. JAMA 282(9):861–866 36. Issenberg SB et al (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28 37. ASGE Training Committee (2011) Endoscopic simulators. Gastrointest Endosc 73(5):861–867 38. Thompson CC, Jirapinyo P, Kumar N et al (2014) Development and initial validation of endoscopic part-task training box. Endoscopy 46(9):735–744
123
