Original Article
67
Arthroscopic Training Resources in Orthopedic Resident Education Jeffrey Lawler, MD3
1 School of Medicine and Dentistry, University of Rochester, Rochester,
New York 2 Department of Orthopaedic Surgery, Yale University, New Haven, Connecticut 3 Department of Orthopaedics, Clinton M. Ray M.D. Orthopaedics and Sports Medicine, Oxford, Alabama 4 Department of Orthopaedic Surgery, Duke University, Durham, North Carolina 5 Department of Orthopaedic Surgery and Rehabilitation, University of Rochester, Rochester, New York
Claude Moorman III, MD4
Address for correspondence Ryan Koehler, MSEd, School of Medicine and Dentistry, University of Rochester, 601 Elmwood Avenue, Box 200, Rochester, NY 14642 (e-mail: [email protected]).
J Knee Surg 2015;28:67–74.
Abstract
Keywords
► arthroscopy ► education ► training resources
The purpose of this study was to determine the frequency of use, perceived effectiveness, and preference for arthroscopic surgical skill training resources. An electronic survey was sent to orthopedics residents, residency program directors, and orthopedic sports medicine attending physicians in the United States. The frequency and perceived effectiveness of 10 types of adjunctive arthroscopic skills training was assessed. Residents and faculty members were asked to rate their confidence in resident ability to perform common arthroscopic procedures. Surveys were completed by 40 of 152 (26.3%) orthopedic residency program directors, 70 of 426 (16.4%) sports medicine faculty, and 235 of 3,170 (7.4%) orthopedic residents. The use of adjunctive methods of training varied from only 9.8% of programs with virtual reality training to 80.5% of programs that used reading of published materials to develop arthroscopic skill. Practice on cadaveric specimens was viewed as the most effective and preferred adjunctive method of training. Residents trained on cadaveric specimens reported increased confidence in their ability to perform arthroscopic procedures. The resources for developing arthroscopic surgical skill vary considerably across orthopedic residency programs in the United States. Adjunctive training methods were perceived to be effective at supplementing traditional training in the operating room.
Concern for patient safety and increased emphasis on competency-based graduate medical education has led to an increased interest in developing portions of the surgeon’s learning curve outside the operating room through surgical skills simulation.1 In addition to enhanced patient safety, surgical skills simulation has many advantages, such as the ability to develop a standardized national curriculum that can be objectively assessed and tailored to fit within the constraints of current work-hour restrictions. The Accreditation
Council for Graduate Medical Education now requires that all general surgery residency programs “include simulation and skills laboratories” and demonstrate the “acquisition and maintenance of skills with a competency-based method of evaluation.”2 In addition, successful completion of the Fundamentals of Laparoscopic Surgery (FLS) program is required to sit for the American Board of Surgery Qualifying Exam.3 Orthopedic surgery has been slower to embrace simulation as part of resident education. There is currently no
received August 18, 2013 accepted January 1, 2014 published online February 7, 2014
Copyright © 2015 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.
DOI http://dx.doi.org/ 10.1055/s-0034-1368142. ISSN 1538-8506.
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Ryan Koehler, MSEd1 Tamara John, MD2 Gregg Nicandri, MD5
Arthroscopic Training Resources in Orthopedic Resident Education requirement for arthroscopic surgical skills simulation during residency training. Karam et al recently found that while 76% of responding orthopedic residency programs had a dedicated surgical skills laboratory, less than half of these programs had a structured surgical skills curriculum.4 Of the programs with a dedicated curriculum, a majority devoted less than 1 hour/week of protected time to structured surgical skill simulation.4 The development of arthroscopic surgical skill is one area within orthopedics that is well suited for simulation because it requires skills that differ considerably from those used in open cases. In comparison to open procedures, arthroscopic cases require the surgeon to overcome the loss of tactile sensation, lack of depth perception, limited degrees of freedom, increased length of arthroscopic instruments, and varying levels of eye–hand coordination. Many of the skills specific to arthroscopy can be simulated outside of the operating room using a variety of resources including lowfidelity dry models, cadaveric tissues, and virtual reality trainers.5 Assessment of arthroscopic technical skill may also prove to be more practical in the simulation laboratory as there is currently no consensus on the number of procedures required to establish proficiency in the operating room.6 However, the frequency of use and effectiveness of the variety of resources currently being used to develop arthroscopic surgical skill is relatively unknown. Arthroscopy towers and instruments were reported to be the second most common form of equipment available for surgical skills laboratories behind surrogate bone models.4 Vitale et al found that hands-on instructional courses and practice on cadaveric specimens were rated as the second and third most important resources, respectively, behind completing a sports medicine fellowship for training to perform all-arthroscopic rotator cuff repair.7 The purpose of this study was to assess the frequency of use and perceived effectiveness of the most commonly used resources in knee and shoulder arthroscopic skills training. Our hypothesis was that residents and attending surgeons would differ in their views of the most effective modality of arthroscopic training. Additionally, we sought to determine if residents with greater exposure to adjunctive arthroscopic training resources would have increased confidence in performing common arthroscopic procedures.
Methods
Koehler et al.
not complete the survey. Two months after the original email, data collection was considered complete and any subsequent responses were no longer collected. A similar method has been used successfully in the past to assess training resources used in arthroscopic rotator cuff repair.7
Survey Items Both versions of the survey were divided into three sections: program demographics and available resources, perceived effectiveness of available arthroscopic training resources, and level of confidence performing common orthopedic procedures. Demographic information about each training program included the number of residents, time spent in dedicated sports or arthroscopy rotations, and what arthroscopic training resources were either provided (faculty) or used (residents). Subjects ranked the perceived effectiveness of the 10 arthroscopic skill development resources using a 5point Likert scale ranging from “least effective” to “most effective.” Subjects were also asked to identify which one method of training they prefer in addition to observing and assisting in the operating room. Confidence performing 15 common arthroscopic and nonarthroscopic procedures was assessed with a 10-point Likert scale, ranging from “not confident” to “extremely confident.” Residents were asked to self-assess their own confidence performing the procedures while program directors and faculty were asked to assess their confidence in graduating residents from their program performing the same procedures. Applying a short leg splint was used as a positive control because it is a basic skill that all residents should be confident about performing. In contrast, performing an open reduction with internal fixation (ORIF) of a both-column acetabular fracture was used as a negative control, as few residents should be confident about performing this procedure.
Data Analysis All item responses provided were included in the statistical analysis, but the response rate was conservatively calculated based on the number of surveys completed in their entirety. Descriptive statistics including frequency of responses and mean with 95% confidence interval (CI) were used to present the responses to survey items. Comparison of ordinal responses to perceived effectiveness of training resources and the confidence performing procedures was compared between residents and program directors using a single-factor analysis of variance with any differences at p < 0.05 considered to be significant.
Survey Design and Distribution This study was approved by the institutional review board of our institution. Two versions of an online survey (SurveyMonkey.com, Palo Alto, CA) were created with one worded for program directors and orthopedic faculty (30 items) and the other version worded for residents (30 items). An e-mail was sent to 3,170 orthopedic residents, 152 residency program directors, and 426 orthopedic sports medicine attending physicians practicing at institutions affiliated with residency programs in January 2009. The initial e-mail described the study and asked for anonymous participation. A reminder email was sent 1 week after the initial e-mail to those who did The Journal of Knee Surgery
Vol. 28
No. 1/2015
Results Surveys were completed by 40 of 152 (26.3%) orthopedic residency program directors, 70 of 426 (16.4%) sports medicine faculty, and 235 of 3,170 (7.4%) orthopedic residents. The availability of different arthroscopic training resources ranged from very limited (virtual reality) to universal (dedicated sports rotation) (►Table 1). The average time spent by chief resident respondents on a dedicated sports or arthroscopy rotation was 5.48 months (95% CI, 4.95–6.00) with a range from 2 to 12 months. There was considerable variability
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
68
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
69
Arthroscopic training resources
Percent of programs reporting use
Dedicated sports or arthroscopy rotation
100.0
Lecture or didactic instruction (no laboratory)
56.1
Hands-on instructional courses (with laboratory)
68.3
Practice on artificial models
61.0
Practice on cadaveric specimens
61.0
Practice on virtual reality simulators
9.8
Observing arthroscopic procedures in the operating room
97.6
Assisting/performing arthroscopic procedures in the operating room
100.0
Reading published materials
97.6
Reading industry-sponsored text or viewing instructional presentations
80.5
Using web-based instruction
61.0
in the percentage of residents who had used the various training resources at the different stages of training with a general trend of increased exposure with increased time in training (►Table 2). When asked about the perceived effectiveness of available arthroscopic training modalities, assisting or performing surgery along with practice on cadaveric specimens was perceived to be the most effective resources. Lectures and didactic instruction were perceived as the least effective (►Table 3). Sports medicine faculty assigned a significantly higher mean perceived effectiveness than residents for observation of arthroscopic procedures in the operating room (p < 0.05). No other significant differences in mean perceived effectiveness were observed between groups (►Table 3). When asked which one adjunctive training method was preferred in addition to observation and active participation in the operating room, practice with cadaveric specimens and
a hands-on instructional course were the most frequently selected by both residency program directors and residents (►Table 4). The level of confidence for residents varied from procedure to procedure with the positive control (short leg splint application) attaining the highest mean score and the negative control (ORIF of both-column acetabular fracture) getting the lowest mean score. The level of confidence was higher for basic arthroscopic procedures than for more advanced procedures (►Fig. 1). The level of confidence of chief residents to perform advanced arthroscopic procedures was significantly lower than the level of confidence assigned by program directors (p < 0.01) (►Fig. 2). The level of confidence performing arthroscopic procedures was compared between residents without exposure to cadaver training and those who had prior cadaver training. There was a significant difference in the level of confidence between the two groups
Table 2 Resident participation in different arthroscopic training methods by year in training PGY-1 (%)
PGY-2 (%)
PGY-3 (%)
PGY-4 (%)
PGY-5/6 (%)
Completed sports/arthroscopy rotation
38.2
53.8
79.4
87.3
95.6
Attended lecture or didactic instructional course (no laboratory)
22.6
33.3
55.2
45.5
60.3
Attended hands-on instructional course
12.5
11.1
28.4
36.4
46.0
Practiced on artificial models
15.6
30.6
41.8
27.3
46.8
Practiced on cadaveric specimens
18.8
38.9
53.7
64.8
77.8
Practiced on virtual reality simulators
3.1
8.3
10.6
5.7
11.3
Observed in surgery
93.8
94.4
100.0
98.1
98.4
Assisted or performed surgery
65.6
94.4
100.0
96.4
100.0
Read published material
84.4
97.2
94.0
96.4
98.4
Read industry text or watched instructional presentations
38.7
47.2
55.2
63.6
77.4
Used web-based instruction
12.5
22.9
38.8
38.2
45.9
Abbreviation: PGY, postgraduate year. The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Table 1 Availability of arthroscopic training resources
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
Table 3 Perceived effectiveness of arthroscopic training resources Arthroscopic training resources
Program directors
Sports attendings
Residents
Mean
95% CI
Mean
95% CI
Mean
95% CI
Lecture or didactic instruction (no laboratory)
2.39
2.12–2.66
2.50
2.26–2.74
2.48
2.37–2.60
Hands-on instructional courses (with laboratory)
3.95
3.68–4.22
4.03
3.82–4.23
4.14
4.02–4.26
Practice on artificial models
3.32
3.06–3.58
3.29
3.09–3.49
3.46
3.33–3.58
Practice on cadaveric specimens
4.15
3.83–4.47
4.37
4.19–4.55
4.31
4.20–4.41
Practice on virtual reality simulators
3.33
3.05–3.60
3.13
2.89–3.36
3.20
Observing arthroscopic procedures in the operating room
3.42
3.10–3.73
3.47
Assisting/performing arthroscopic procedures in the operating room
4.88
4.77–4.98
4.72
a
3.27–3.68
3.14
4.61–4.84
4.78
3.06–3.34 a
3.03–3.26 4.71–4.85
Reading published materials
2.83
2.53–3.13
3.05
2.81–3.29
2.84
2.73–2.95
Reading industry-sponsored text or viewing instructional presentations
2.56
2.25–2.87
2.75
2.55–2.95
2.79
2.68–2.90
Using web-based instruction
2.80
2.53–3.07
2.88
2.66–3.09
2.84
2.73–2.95
Abbreviation: CI, confidence interval. a Mean difference was significant at p < 0.05 level.
for all arthroscopic procedures (p < 0.01) (►Fig. 3). No difference was observed between groups for level of confidence performing short leg splint or ORIF of both-column acetabular fracture (p > 0.05).
Discussion As orthopedic surgery training programs transition to the Next Accreditation System and Milestone-based assessment, surgical skill simulation and assessment will play an expanding role in residency training.8 In this study we identified the frequency of use for various methods of arthroscopic skills training. Traditional means of training such as a dedicated sports or arthroscopy rotation, assisting or performing arthroscopic surgery, observing in the operating room, and reading published materials were the most common arthroscopic training resources available. The availability of adjunctive resources was more variable from program to program. One of the concerns with this variability is the possibility that residents may not have access to sufficient resources to
ensure the development of competency performing basic arthroscopic procedures. Hall et al reported that graduating orthopedic residents felt less prepared in arthroscopic techniques as compared with open techniques.9 In our study, we found that chief residents were significantly less confident in their ability to perform advanced arthroscopic procedures when compared with the level of confidence of program directors for the ability of graduating residents. It is important to determine if this difference is due to uncertainty of residents in their own abilities or reflects a true need for more structure training in arthroscopy. However, this was not the purpose of our study and we are unable to make any assumptions based on our data. In contrast to our hypothesis we found that both residents and faculty members had similar perceptions of the effectiveness of different training resources. Both groups identified practice with cadaveric specimens as the most effective adjunctive method of training. Practice on cadaveric tissue is considered to be the gold standard for arthroscopic surgical skills simulation because it closely mimics the experience of
Table 4 Preferred adjunctive method of arthroscopic training
Lecture or didactic instruction (no laboratory)
Program directors (%)
Residents (%)
0
0.40
Hands-on instructional courses (with laboratory)
34.10
38.90
Practice on artificial models
4.90
4.60
Practice on cadaveric specimens
46.30
49.40
Practice on virtual reality simulators
9.80
5.80
Reading published materials
2.40
0.40
Reading industry-sponsored text or viewing instructional presentations
0
0
Using web-based instruction
2.40
0.40
The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
70
Koehler et al.
Fig. 1 Mean level of confidence of all residents for performing selected procedures (1 ¼ not confident to 10 ¼ extremely confident). Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; both column, ORIF of both-column acetabular fracture (negative control); DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair; SLS, short leg splint (positive control).
the actual operating room. We found that residents with prior practice using cadaveric specimens were more confident in their ability to perform common arthroscopic procedures (►Fig. 3). The availability of lower fidelity artificial bench-top models was similar to that of cadaveric training. However, this
resource was perceived to be less effective than training with cadaveric specimens by all subjects. Interestingly, hands-on training with low-fidelity models was perceived to be more effective than observation of arthroscopic procedures in the operating room. Howells et al was able to show that training with a bench-top knee model led to improved technical
Fig. 2 Comparison of the reported level of confidence for the ability of chief residents to perform common arthroscopic procedures reported by chief residents and program directors. Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair. The difference was significant at the p < 0.01 level. The difference was significant at the p < 0.001 level. The Journal of Knee Surgery
Vol. 28
No. 1/2015
71
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Arthroscopic Training Resources in Orthopedic Resident Education
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
Fig. 3 Comparison of the level of confidence performing common arthroscopic procedures in residents with and without prior practice on cadaveric specimens. Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair. The difference was significant at the p < 0.01 level.
performance in the operating room.10 Therefore, there appears to be utility in providing residents with both low- and high-fidelity surgical skill simulation opportunities as they advance through training. Identifying the most desired and effective methods of teaching arthroscopic skill is an important step in developing a national curriculum. Both residents and program directors have expressed a strong desire for the development of a standardized orthopedic surgical skills curriculum.4 The Fundamental of Arthroscopic Surgery Training (FAST) program is an ongoing initiative of the Arthroscopy Association of North America, American Academy of Orthopaedic Surgeons, and American Board of Orthopaedic Surgery to develop a structured curriculum for teaching and assessing arthroscopy in a step-wise manner.11 Similar to the FLS for general surgery, this program will consist of a formal curriculum and proficiency assessment.11 It is important to ensure that the most effective adjunctive methods of training be included in this program to maximize the benefit of all contact hours.
Limitations There were several limitations to this survey study. Our response rate was low but similar to previous survey studies of orthopedic residents and faculty members.4,9 While this response rate is sufficient to make general observations about the availability of arthroscopic training modalities, it does not allow for in-depth subgroup analysis. This may be the reason for the significant difference observed in level of confidence performing common arthroscopic procedures in residents The Journal of Knee Surgery
Vol. 28
No. 1/2015
with and without prior practice on cadaveric specimens. Our number of subjects was not sufficient to determine if this effect was confounded by year in training because senior residents would likely have had more opportunities to be exposed to cadaveric training. Another limitation of our study was the limited exposure of residents and faculty members to all modalities of training available making the perceived level of effectiveness uncertain. Virtual reality training was perceived to be as effective as observing in the operating room and practice on artificial models. However, only 9.8% of responding programs reported having access to virtual reality training available, making it difficult to draw any generalizations about its true effectiveness. Additionally, the effectiveness of a particular training modality is likely to change as a resident advances through a training program. For example, low-fidelity bench-top training is appropriately challenging for a junior resident with little experience performing arthroscopic surgery but provides little benefit to a chief resident. We did not attempt to differentiate the appropriateness of each training modality for different levels of experience with this study, but identifying the appropriate modality for level of experience will be an important aspect of the development of a standardized arthroscopic skills curriculum.
Conclusion The resources for developing arthroscopic surgical skill vary considerably across orthopedic residency programs in the United States. Residents and faculty have similar perceptions on the preference and effectiveness of available training
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
72
Arthroscopic Training Resources in Orthopedic Resident Education
5 Michelson JD. Simulation in orthopaedic education: an overview
6
7
References 1 Reznick RK, MacRae H. Teaching surgical skills—changes in the
wind. N Engl J Med 2006;355(25):2664–2669
8
2 Accreditation Council for Graduate Medical Education. General
surgery program requirements. Available at: http://www.acgme. org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/ 440_general_surgery_01012008_07012012.pdf. Accessed April 14, 2013 3 American Board of Surgery. Training requirements for general surgery certification. Available at: http://www.absurgery.org/ default.jsp?certgsqe_training. Accessed April 14, 2013 4 Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am 2013;95(1):e4
73
9
10
11
of theory and practice. J Bone Joint Surg Am 2006;88(6): 1405–1411 O’Neill PJ, Cosgarea AJ, Freedman JA, Queale WS, McFarland EG. Arthroscopic proficiency: a survey of orthopaedic sports medicine fellowship directors and orthopaedic surgery department chairs. Arthroscopy 2002;18(7):795–800 Vitale MA, Kleweno CP, Jacir AM, Levine WN, Bigliani LU, Ahmad CS. Training resources in arthroscopic rotator cuff repair. J Bone Joint Surg Am 2007;89(6):1393–1398 Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med 2012;366(11): 1051–1056 Hall MP, Kaplan KM, Gorczynski CT, Zuckerman JD, Rosen JE. Assessment of arthroscopic training in U.S. orthopedic surgery residency programs—a resident self-assessment. Bull NYU Hosp Jt Dis 2010;68(1):5–10 Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 2008;90(4):494–499 Leahy M. Using simulation, metrics to improve orthopaedic surgical skills. AAOS Now. Available at: http://www.aaos.org/news/ aaosnow/feb12/clinical9.asp. Accessed April 14, 2013
The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
resources. Adjunctive training methods such as cadaveric training and hands-on instructional courses are perceived to be effective and should be considered as a supplement to traditional training in the operating room.
Koehler et al.
67
Arthroscopic Training Resources in Orthopedic Resident Education Jeffrey Lawler, MD3
1 School of Medicine and Dentistry, University of Rochester, Rochester,
New York 2 Department of Orthopaedic Surgery, Yale University, New Haven, Connecticut 3 Department of Orthopaedics, Clinton M. Ray M.D. Orthopaedics and Sports Medicine, Oxford, Alabama 4 Department of Orthopaedic Surgery, Duke University, Durham, North Carolina 5 Department of Orthopaedic Surgery and Rehabilitation, University of Rochester, Rochester, New York
Claude Moorman III, MD4
Address for correspondence Ryan Koehler, MSEd, School of Medicine and Dentistry, University of Rochester, 601 Elmwood Avenue, Box 200, Rochester, NY 14642 (e-mail: [email protected]).
J Knee Surg 2015;28:67–74.
Abstract
Keywords
► arthroscopy ► education ► training resources
The purpose of this study was to determine the frequency of use, perceived effectiveness, and preference for arthroscopic surgical skill training resources. An electronic survey was sent to orthopedics residents, residency program directors, and orthopedic sports medicine attending physicians in the United States. The frequency and perceived effectiveness of 10 types of adjunctive arthroscopic skills training was assessed. Residents and faculty members were asked to rate their confidence in resident ability to perform common arthroscopic procedures. Surveys were completed by 40 of 152 (26.3%) orthopedic residency program directors, 70 of 426 (16.4%) sports medicine faculty, and 235 of 3,170 (7.4%) orthopedic residents. The use of adjunctive methods of training varied from only 9.8% of programs with virtual reality training to 80.5% of programs that used reading of published materials to develop arthroscopic skill. Practice on cadaveric specimens was viewed as the most effective and preferred adjunctive method of training. Residents trained on cadaveric specimens reported increased confidence in their ability to perform arthroscopic procedures. The resources for developing arthroscopic surgical skill vary considerably across orthopedic residency programs in the United States. Adjunctive training methods were perceived to be effective at supplementing traditional training in the operating room.
Concern for patient safety and increased emphasis on competency-based graduate medical education has led to an increased interest in developing portions of the surgeon’s learning curve outside the operating room through surgical skills simulation.1 In addition to enhanced patient safety, surgical skills simulation has many advantages, such as the ability to develop a standardized national curriculum that can be objectively assessed and tailored to fit within the constraints of current work-hour restrictions. The Accreditation
Council for Graduate Medical Education now requires that all general surgery residency programs “include simulation and skills laboratories” and demonstrate the “acquisition and maintenance of skills with a competency-based method of evaluation.”2 In addition, successful completion of the Fundamentals of Laparoscopic Surgery (FLS) program is required to sit for the American Board of Surgery Qualifying Exam.3 Orthopedic surgery has been slower to embrace simulation as part of resident education. There is currently no
received August 18, 2013 accepted January 1, 2014 published online February 7, 2014
Copyright © 2015 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.
DOI http://dx.doi.org/ 10.1055/s-0034-1368142. ISSN 1538-8506.
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Ryan Koehler, MSEd1 Tamara John, MD2 Gregg Nicandri, MD5
Arthroscopic Training Resources in Orthopedic Resident Education requirement for arthroscopic surgical skills simulation during residency training. Karam et al recently found that while 76% of responding orthopedic residency programs had a dedicated surgical skills laboratory, less than half of these programs had a structured surgical skills curriculum.4 Of the programs with a dedicated curriculum, a majority devoted less than 1 hour/week of protected time to structured surgical skill simulation.4 The development of arthroscopic surgical skill is one area within orthopedics that is well suited for simulation because it requires skills that differ considerably from those used in open cases. In comparison to open procedures, arthroscopic cases require the surgeon to overcome the loss of tactile sensation, lack of depth perception, limited degrees of freedom, increased length of arthroscopic instruments, and varying levels of eye–hand coordination. Many of the skills specific to arthroscopy can be simulated outside of the operating room using a variety of resources including lowfidelity dry models, cadaveric tissues, and virtual reality trainers.5 Assessment of arthroscopic technical skill may also prove to be more practical in the simulation laboratory as there is currently no consensus on the number of procedures required to establish proficiency in the operating room.6 However, the frequency of use and effectiveness of the variety of resources currently being used to develop arthroscopic surgical skill is relatively unknown. Arthroscopy towers and instruments were reported to be the second most common form of equipment available for surgical skills laboratories behind surrogate bone models.4 Vitale et al found that hands-on instructional courses and practice on cadaveric specimens were rated as the second and third most important resources, respectively, behind completing a sports medicine fellowship for training to perform all-arthroscopic rotator cuff repair.7 The purpose of this study was to assess the frequency of use and perceived effectiveness of the most commonly used resources in knee and shoulder arthroscopic skills training. Our hypothesis was that residents and attending surgeons would differ in their views of the most effective modality of arthroscopic training. Additionally, we sought to determine if residents with greater exposure to adjunctive arthroscopic training resources would have increased confidence in performing common arthroscopic procedures.
Methods
Koehler et al.
not complete the survey. Two months after the original email, data collection was considered complete and any subsequent responses were no longer collected. A similar method has been used successfully in the past to assess training resources used in arthroscopic rotator cuff repair.7
Survey Items Both versions of the survey were divided into three sections: program demographics and available resources, perceived effectiveness of available arthroscopic training resources, and level of confidence performing common orthopedic procedures. Demographic information about each training program included the number of residents, time spent in dedicated sports or arthroscopy rotations, and what arthroscopic training resources were either provided (faculty) or used (residents). Subjects ranked the perceived effectiveness of the 10 arthroscopic skill development resources using a 5point Likert scale ranging from “least effective” to “most effective.” Subjects were also asked to identify which one method of training they prefer in addition to observing and assisting in the operating room. Confidence performing 15 common arthroscopic and nonarthroscopic procedures was assessed with a 10-point Likert scale, ranging from “not confident” to “extremely confident.” Residents were asked to self-assess their own confidence performing the procedures while program directors and faculty were asked to assess their confidence in graduating residents from their program performing the same procedures. Applying a short leg splint was used as a positive control because it is a basic skill that all residents should be confident about performing. In contrast, performing an open reduction with internal fixation (ORIF) of a both-column acetabular fracture was used as a negative control, as few residents should be confident about performing this procedure.
Data Analysis All item responses provided were included in the statistical analysis, but the response rate was conservatively calculated based on the number of surveys completed in their entirety. Descriptive statistics including frequency of responses and mean with 95% confidence interval (CI) were used to present the responses to survey items. Comparison of ordinal responses to perceived effectiveness of training resources and the confidence performing procedures was compared between residents and program directors using a single-factor analysis of variance with any differences at p < 0.05 considered to be significant.
Survey Design and Distribution This study was approved by the institutional review board of our institution. Two versions of an online survey (SurveyMonkey.com, Palo Alto, CA) were created with one worded for program directors and orthopedic faculty (30 items) and the other version worded for residents (30 items). An e-mail was sent to 3,170 orthopedic residents, 152 residency program directors, and 426 orthopedic sports medicine attending physicians practicing at institutions affiliated with residency programs in January 2009. The initial e-mail described the study and asked for anonymous participation. A reminder email was sent 1 week after the initial e-mail to those who did The Journal of Knee Surgery
Vol. 28
No. 1/2015
Results Surveys were completed by 40 of 152 (26.3%) orthopedic residency program directors, 70 of 426 (16.4%) sports medicine faculty, and 235 of 3,170 (7.4%) orthopedic residents. The availability of different arthroscopic training resources ranged from very limited (virtual reality) to universal (dedicated sports rotation) (►Table 1). The average time spent by chief resident respondents on a dedicated sports or arthroscopy rotation was 5.48 months (95% CI, 4.95–6.00) with a range from 2 to 12 months. There was considerable variability
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
68
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
69
Arthroscopic training resources
Percent of programs reporting use
Dedicated sports or arthroscopy rotation
100.0
Lecture or didactic instruction (no laboratory)
56.1
Hands-on instructional courses (with laboratory)
68.3
Practice on artificial models
61.0
Practice on cadaveric specimens
61.0
Practice on virtual reality simulators
9.8
Observing arthroscopic procedures in the operating room
97.6
Assisting/performing arthroscopic procedures in the operating room
100.0
Reading published materials
97.6
Reading industry-sponsored text or viewing instructional presentations
80.5
Using web-based instruction
61.0
in the percentage of residents who had used the various training resources at the different stages of training with a general trend of increased exposure with increased time in training (►Table 2). When asked about the perceived effectiveness of available arthroscopic training modalities, assisting or performing surgery along with practice on cadaveric specimens was perceived to be the most effective resources. Lectures and didactic instruction were perceived as the least effective (►Table 3). Sports medicine faculty assigned a significantly higher mean perceived effectiveness than residents for observation of arthroscopic procedures in the operating room (p < 0.05). No other significant differences in mean perceived effectiveness were observed between groups (►Table 3). When asked which one adjunctive training method was preferred in addition to observation and active participation in the operating room, practice with cadaveric specimens and
a hands-on instructional course were the most frequently selected by both residency program directors and residents (►Table 4). The level of confidence for residents varied from procedure to procedure with the positive control (short leg splint application) attaining the highest mean score and the negative control (ORIF of both-column acetabular fracture) getting the lowest mean score. The level of confidence was higher for basic arthroscopic procedures than for more advanced procedures (►Fig. 1). The level of confidence of chief residents to perform advanced arthroscopic procedures was significantly lower than the level of confidence assigned by program directors (p < 0.01) (►Fig. 2). The level of confidence performing arthroscopic procedures was compared between residents without exposure to cadaver training and those who had prior cadaver training. There was a significant difference in the level of confidence between the two groups
Table 2 Resident participation in different arthroscopic training methods by year in training PGY-1 (%)
PGY-2 (%)
PGY-3 (%)
PGY-4 (%)
PGY-5/6 (%)
Completed sports/arthroscopy rotation
38.2
53.8
79.4
87.3
95.6
Attended lecture or didactic instructional course (no laboratory)
22.6
33.3
55.2
45.5
60.3
Attended hands-on instructional course
12.5
11.1
28.4
36.4
46.0
Practiced on artificial models
15.6
30.6
41.8
27.3
46.8
Practiced on cadaveric specimens
18.8
38.9
53.7
64.8
77.8
Practiced on virtual reality simulators
3.1
8.3
10.6
5.7
11.3
Observed in surgery
93.8
94.4
100.0
98.1
98.4
Assisted or performed surgery
65.6
94.4
100.0
96.4
100.0
Read published material
84.4
97.2
94.0
96.4
98.4
Read industry text or watched instructional presentations
38.7
47.2
55.2
63.6
77.4
Used web-based instruction
12.5
22.9
38.8
38.2
45.9
Abbreviation: PGY, postgraduate year. The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Table 1 Availability of arthroscopic training resources
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
Table 3 Perceived effectiveness of arthroscopic training resources Arthroscopic training resources
Program directors
Sports attendings
Residents
Mean
95% CI
Mean
95% CI
Mean
95% CI
Lecture or didactic instruction (no laboratory)
2.39
2.12–2.66
2.50
2.26–2.74
2.48
2.37–2.60
Hands-on instructional courses (with laboratory)
3.95
3.68–4.22
4.03
3.82–4.23
4.14
4.02–4.26
Practice on artificial models
3.32
3.06–3.58
3.29
3.09–3.49
3.46
3.33–3.58
Practice on cadaveric specimens
4.15
3.83–4.47
4.37
4.19–4.55
4.31
4.20–4.41
Practice on virtual reality simulators
3.33
3.05–3.60
3.13
2.89–3.36
3.20
Observing arthroscopic procedures in the operating room
3.42
3.10–3.73
3.47
Assisting/performing arthroscopic procedures in the operating room
4.88
4.77–4.98
4.72
a
3.27–3.68
3.14
4.61–4.84
4.78
3.06–3.34 a
3.03–3.26 4.71–4.85
Reading published materials
2.83
2.53–3.13
3.05
2.81–3.29
2.84
2.73–2.95
Reading industry-sponsored text or viewing instructional presentations
2.56
2.25–2.87
2.75
2.55–2.95
2.79
2.68–2.90
Using web-based instruction
2.80
2.53–3.07
2.88
2.66–3.09
2.84
2.73–2.95
Abbreviation: CI, confidence interval. a Mean difference was significant at p < 0.05 level.
for all arthroscopic procedures (p < 0.01) (►Fig. 3). No difference was observed between groups for level of confidence performing short leg splint or ORIF of both-column acetabular fracture (p > 0.05).
Discussion As orthopedic surgery training programs transition to the Next Accreditation System and Milestone-based assessment, surgical skill simulation and assessment will play an expanding role in residency training.8 In this study we identified the frequency of use for various methods of arthroscopic skills training. Traditional means of training such as a dedicated sports or arthroscopy rotation, assisting or performing arthroscopic surgery, observing in the operating room, and reading published materials were the most common arthroscopic training resources available. The availability of adjunctive resources was more variable from program to program. One of the concerns with this variability is the possibility that residents may not have access to sufficient resources to
ensure the development of competency performing basic arthroscopic procedures. Hall et al reported that graduating orthopedic residents felt less prepared in arthroscopic techniques as compared with open techniques.9 In our study, we found that chief residents were significantly less confident in their ability to perform advanced arthroscopic procedures when compared with the level of confidence of program directors for the ability of graduating residents. It is important to determine if this difference is due to uncertainty of residents in their own abilities or reflects a true need for more structure training in arthroscopy. However, this was not the purpose of our study and we are unable to make any assumptions based on our data. In contrast to our hypothesis we found that both residents and faculty members had similar perceptions of the effectiveness of different training resources. Both groups identified practice with cadaveric specimens as the most effective adjunctive method of training. Practice on cadaveric tissue is considered to be the gold standard for arthroscopic surgical skills simulation because it closely mimics the experience of
Table 4 Preferred adjunctive method of arthroscopic training
Lecture or didactic instruction (no laboratory)
Program directors (%)
Residents (%)
0
0.40
Hands-on instructional courses (with laboratory)
34.10
38.90
Practice on artificial models
4.90
4.60
Practice on cadaveric specimens
46.30
49.40
Practice on virtual reality simulators
9.80
5.80
Reading published materials
2.40
0.40
Reading industry-sponsored text or viewing instructional presentations
0
0
Using web-based instruction
2.40
0.40
The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
70
Koehler et al.
Fig. 1 Mean level of confidence of all residents for performing selected procedures (1 ¼ not confident to 10 ¼ extremely confident). Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; both column, ORIF of both-column acetabular fracture (negative control); DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair; SLS, short leg splint (positive control).
the actual operating room. We found that residents with prior practice using cadaveric specimens were more confident in their ability to perform common arthroscopic procedures (►Fig. 3). The availability of lower fidelity artificial bench-top models was similar to that of cadaveric training. However, this
resource was perceived to be less effective than training with cadaveric specimens by all subjects. Interestingly, hands-on training with low-fidelity models was perceived to be more effective than observation of arthroscopic procedures in the operating room. Howells et al was able to show that training with a bench-top knee model led to improved technical
Fig. 2 Comparison of the reported level of confidence for the ability of chief residents to perform common arthroscopic procedures reported by chief residents and program directors. Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair. The difference was significant at the p < 0.01 level. The difference was significant at the p < 0.001 level. The Journal of Knee Surgery
Vol. 28
No. 1/2015
71
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
Arthroscopic Training Resources in Orthopedic Resident Education
Arthroscopic Training Resources in Orthopedic Resident Education
Koehler et al.
Fig. 3 Comparison of the level of confidence performing common arthroscopic procedures in residents with and without prior practice on cadaveric specimens. Error bars represent the 95% confidence interval of the mean. ACL, anterior cruciate ligament reconstruction (any type); Bankart, arthroscopic Bankart repair; DKS, diagnostic knee scope; DSS, diagnostic shoulder scope; PMM, partial medial meniscectomy; RCR, rotator cuff repair; SAD, subacromial decompression; SLAP, superior labrum anterior to posterior repair. The difference was significant at the p < 0.01 level.
performance in the operating room.10 Therefore, there appears to be utility in providing residents with both low- and high-fidelity surgical skill simulation opportunities as they advance through training. Identifying the most desired and effective methods of teaching arthroscopic skill is an important step in developing a national curriculum. Both residents and program directors have expressed a strong desire for the development of a standardized orthopedic surgical skills curriculum.4 The Fundamental of Arthroscopic Surgery Training (FAST) program is an ongoing initiative of the Arthroscopy Association of North America, American Academy of Orthopaedic Surgeons, and American Board of Orthopaedic Surgery to develop a structured curriculum for teaching and assessing arthroscopy in a step-wise manner.11 Similar to the FLS for general surgery, this program will consist of a formal curriculum and proficiency assessment.11 It is important to ensure that the most effective adjunctive methods of training be included in this program to maximize the benefit of all contact hours.
Limitations There were several limitations to this survey study. Our response rate was low but similar to previous survey studies of orthopedic residents and faculty members.4,9 While this response rate is sufficient to make general observations about the availability of arthroscopic training modalities, it does not allow for in-depth subgroup analysis. This may be the reason for the significant difference observed in level of confidence performing common arthroscopic procedures in residents The Journal of Knee Surgery
Vol. 28
No. 1/2015
with and without prior practice on cadaveric specimens. Our number of subjects was not sufficient to determine if this effect was confounded by year in training because senior residents would likely have had more opportunities to be exposed to cadaveric training. Another limitation of our study was the limited exposure of residents and faculty members to all modalities of training available making the perceived level of effectiveness uncertain. Virtual reality training was perceived to be as effective as observing in the operating room and practice on artificial models. However, only 9.8% of responding programs reported having access to virtual reality training available, making it difficult to draw any generalizations about its true effectiveness. Additionally, the effectiveness of a particular training modality is likely to change as a resident advances through a training program. For example, low-fidelity bench-top training is appropriately challenging for a junior resident with little experience performing arthroscopic surgery but provides little benefit to a chief resident. We did not attempt to differentiate the appropriateness of each training modality for different levels of experience with this study, but identifying the appropriate modality for level of experience will be an important aspect of the development of a standardized arthroscopic skills curriculum.
Conclusion The resources for developing arthroscopic surgical skill vary considerably across orthopedic residency programs in the United States. Residents and faculty have similar perceptions on the preference and effectiveness of available training
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
72
Arthroscopic Training Resources in Orthopedic Resident Education
5 Michelson JD. Simulation in orthopaedic education: an overview
6
7
References 1 Reznick RK, MacRae H. Teaching surgical skills—changes in the
wind. N Engl J Med 2006;355(25):2664–2669
8
2 Accreditation Council for Graduate Medical Education. General
surgery program requirements. Available at: http://www.acgme. org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/ 440_general_surgery_01012008_07012012.pdf. Accessed April 14, 2013 3 American Board of Surgery. Training requirements for general surgery certification. Available at: http://www.absurgery.org/ default.jsp?certgsqe_training. Accessed April 14, 2013 4 Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am 2013;95(1):e4
73
9
10
11
of theory and practice. J Bone Joint Surg Am 2006;88(6): 1405–1411 O’Neill PJ, Cosgarea AJ, Freedman JA, Queale WS, McFarland EG. Arthroscopic proficiency: a survey of orthopaedic sports medicine fellowship directors and orthopaedic surgery department chairs. Arthroscopy 2002;18(7):795–800 Vitale MA, Kleweno CP, Jacir AM, Levine WN, Bigliani LU, Ahmad CS. Training resources in arthroscopic rotator cuff repair. J Bone Joint Surg Am 2007;89(6):1393–1398 Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med 2012;366(11): 1051–1056 Hall MP, Kaplan KM, Gorczynski CT, Zuckerman JD, Rosen JE. Assessment of arthroscopic training in U.S. orthopedic surgery residency programs—a resident self-assessment. Bull NYU Hosp Jt Dis 2010;68(1):5–10 Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 2008;90(4):494–499 Leahy M. Using simulation, metrics to improve orthopaedic surgical skills. AAOS Now. Available at: http://www.aaos.org/news/ aaosnow/feb12/clinical9.asp. Accessed April 14, 2013
The Journal of Knee Surgery
Vol. 28
No. 1/2015
Downloaded by: Nanyang Technological University NTU. Copyrighted material.
resources. Adjunctive training methods such as cadaveric training and hands-on instructional courses are perceived to be effective and should be considered as a supplement to traditional training in the operating room.
Koehler et al.
