EDUCATIONAL INNOVATION
Using a Team-Based Learning Approach at National Meetings to Teach Residents Genomic Pathology Richard L. Haspel, MD, PhD Asma M. Ali, MS Grace C. Huang, MD ABSTRACT Background Accumulating data suggest that team-based learning (TBL) is more effective than lecture-based teaching strategies. Educational sessions at national meetings, however, tend to be lecture-based, and unlike most examples of TBL, involve participants who do not know each other or the instructor. Objective We evaluated a 1-day TBL genomic pathology workshop for residents held at 3 national meetings. Methods A committee of experts developed the workshop. Prior to attending, participants were provided access to readings and asked to answer preparation questions. Each of the 4 modules within the workshop consisted of a 60-minute TBL activity flanked by 15- to 30-minute preactivity and postactivity lectures. We used surveys to acquire participant evaluation of the workshop. Results From 2013–2014, 86 pathology residents from 61 programs participated in 3 workshops at national meetings. All workshops were well received, with over 90% of attendees indicating that they would recommend them to other residents and that the material would help them as practicing pathologists. An incremental approach facilitated decreasing faculty presence at the workshops: the first 2 workshops had 7 faculty each (1 facilitator for each team and 1 circulating faculty member), while the final workshop involved only 2 faculty for 6 teams. For this final session, participants agreed that circulating faculty provided adequate support. Participant ‘‘buy-in’’ (requiring completion of a preworkshop survey) was critical in enabling a TBL approach. Conclusions These results demonstrate that TBL is a feasible and effective strategy for teaching genomic medicine that is acceptable to pathology residents at national meetings.
Introduction National conferences are important opportunities for continuing education and dissemination of new findings. Sessions at these meetings are often lecture based. Many trainees also attend these meetings, and interactive instructional strategies, including teambased learning (TBL), may be more effective with this generation of learners.1 As an important innovation in medical education, TBL emphasizes learner preparation outside of class and knowledge application inside the classroom.2 Medical schools and some residency programs have already incorporated TBL into their curricula.3 In these settings, TBL is often implemented during multiple longitudinal sessions, and teams involve learners who know each other and the instructor. Learners also are motivated to review DOI: http://dx.doi.org/10.4300/JGME-D-15-00221.1 Editor’s Note: The online version of this article contains a workshop preparation checklist, preworkshop and postworkshop survey results, and a figure of individual module perceptions.
80
Journal of Graduate Medical Education, February 1, 2016
preclass materials because they may be formally evaluated on their learning, and performance in front of classmates may be a source of positive social pressure.4 In contrast, at national meetings, attendees often do not know each other or the instructors, attend only a single session, and are not graded on performance. These differences may influence interest, participation, and instructional effectiveness. We describe the development and associated outcomes of a 1-day TBL genomic pathology workshop for residents held at 3 national conferences. The importance of genomic medicine training of health care professionals has been widely recognized.5 Furthermore, the need for hands-on instruction using online tools and consensus discussions on complicated topics (eg, reporting of variants of uncertain significance and ethical issues) make this area an especially ideal subject for TBL. To our knowledge, TBL has not been previously implemented and evaluated at a national meeting, and our results demonstrate the utility of this innovative approach.
EDUCATIONAL INNOVATION
Methods Workshop Development In 2010, the Training Residents in Genomics Working Group (TRIG WG) was formed through the Program Directors Section of the Association of Pathology Chairs to develop teaching tools for pathology residents.6 This group includes experts in genomic pathology, medical education, genetic counseling, and medical genetics, with representation from major pathology organizations, the National Society of Genetic Counselors, and the American College of Medical Genetics and Genomics. The TRIG WG began by revising a pathology resident curriculum originally developed at Beth Israel Deaconess Medical Center.7 Members of the TRIG WG refined the curriculum in an iterative, collaborative fashion into a 1-day TBL workshop for residents attending national pathology meetings.8 The workshop consisted of 4 modules following the clinical narrative of a single patient with breast cancer through the course of her disease and the different testing modalities utilized: single gene testing (module 1), prognostic gene panel testing (module 2), cancer gene panels (module 3), and whole exome sequencing (module 4). Next-generation sequencing is best understood with some molecular pathology background. As such, the curriculum was developed for residents who had completed a molecular pathology rotation. Approximately 2 to 3 weeks prior to the workshop, residents were e-mailed prereading and multiplechoice preparation questions. For the in-person session, each module consisted of 3 components: 1. An instructor-delivered 15- to 30-minute PowerPoint lecture reviewing the answers to the preparation questions and relaying other content needed for the TBL activity. 2. A 60-minute activity consisting of teams of 3 to 6 residents answering a series of questions, including ‘‘reveal’’ type and other questions necessitating the use of online genomic tools (participants were asked to bring a laptop computer or tablet to the workshop).9 3. An instructor-delivered 15- to 30-minute PowerPoint lecture presenting answers to the questions and incorporating a discussion of team responses. The workshop concluded with a 30-minute summary lecture that included major teaching points. Participants were given access to the activities at the workshop electronically (initially Google Docs and
What was known and gap While evidence supports team-based learning (TBL) as more effective, sessions at national meetings generally are lecture based. What is new An objective evaluation of a 1-day TBL workshop in genomic pathology for residents held at 3 national meetings. Limitations Self-selection of participants limits generalizability; the study did not assess learning outcomes. Bottom line A TBL approach at national meetings is feasible from a workforce and logistics perspective, and was well received by participants.
later Google Forms).10 Eight hours (with lunch and breaks) were allotted for the overall workshop, 90 minutes for each module. The workshop was implemented at the 2013 and 2014 American Society for Clinical Pathology (ASCP) Annual Meeting, and the 2014 United States and Canadian Academy of Pathology (USCAP) Annual Meeting. We solicited information about residents via preworkshop surveys to ensure appropriate team composition (eg, balance across postgraduate years, access to a computer or tablet, and background in molecular pathology). While participation for both ASCP 2013 and 2014 required completion of this survey, it was not required for USCAP 2014. Participating residents paid registration fees for the annual meetings, and no additional fees were required to attend the workshops. The targeted enrollment for each workshop was 20 to 40 residents. TBL typically involves 1 instructor managing multiple teams of known learners; however, given the novel application of TBL to a national meeting, we used an incremental approach to decrease the number of required faculty over time. At the first 2 workshops, there were 7 faculty at each session. One or 2 faculty members provided the preactivity and postactivity lectures. One facilitator was also placed at each of 6 team tables (4 to 6 residents), and the remaining faculty member circulated to monitor progress. The facilitators were TRIG WG members with knowledge of the curriculum and some background, but not necessarily expertise, in molecular pathology or genomics. Facilitators participated in a preworkshop, 60-minute conference call to review formats and were instructed to primarily observe and to only assist if the team was having trouble moving forward. For the third workshop, to implement true TBL, 2 workshop faculty served as Journal of Graduate Medical Education, February 1, 2016
81
EDUCATIONAL INNOVATION
instructors, and there were no facilitators for the 6 teams (consisting of 3 to 5 residents in each team). Both faculty had participated in 1 of the prior workshops; 1 had content expertise while the other had expertise in TBL. This second faculty member circulated among teams during the activity component, primarily observing and intervening only if a team had significant difficulty moving forward with the module activity. Workshop development as well as other genomics education initiatives were funded by a R25 grant from the National Cancer Institute awarded to the chair of the TRIG WG in 2012. This provided support for the chair (30%), a TRIG WG faculty member (~5%), an evaluation expert (~5%), and an educational design subcontract with ASCP (~$95,000 per year). Other faculty were not remunerated for participation in the workshops or the monthly 60-minute TRIG WG conference calls. The pathology organizations and the grant provided funding for workshop costs, which varied by venue and included room setup, lunch for attendees, and faculty travel. Estimated workshop preparation time was 4 to 6 hours for lecturers and 2 hours for facilitators. This project was approved for exempt status by the Institutional Review Board at Beth Israel Deaconess Medical Center.
Methods of Evaluation The resident postworkshop survey was developed by 2 authors with expertise in psychometrics and evaluation (A.M.A. and G.C.H.) and was piloted (including cognitive interviews) with Beth Israel Deaconess Medical Center pathology residents who participated in a local version of the workshop. The survey solicited the following information: demographics, global and module-specific perceptions, and free-text responses for workshop strengths and areas for improvement. Responses were anonymous and tabulated by workshop. Results were presented at TRIG WG meetings and used to refine the workshop.
Results The 3 workshops were attended by 86 residents, and the response rate of the postworkshop surveys was 72% (62 of 86). Sixty-one residency programs were represented, and the majority of participants (78%, 67 of 86) were from US programs. The composition by postgraduate year was similar across workshops. Data from the preworkshop and postworkshop surveys are provided as online supplemental material.
82
Journal of Graduate Medical Education, February 1, 2016
The first workshop had 30 participants, with almost all attendees (. 90%) having positive perceptions (TABLE and SUPPLEMENTAL FIGURE). Areas for improvement suggested by participants included website technical issues and more next-generation sequencing–related instruction. To address these concerns, we provided instructions to the participants in the event of a slow website, and modules 3 and 4 were reworked to expand the introduction to nextgeneration sequencing and include an activity involving genomic alignment data analysis. The second workshop had 30 participants. In contrast to the first workshop, a smaller number of residents rated their preparation as ‘‘just right’’ (36% [8 of 22] versus 75% [18 of 24]) and agreed that they ‘‘had put effort into preparing for the workshop’’ (50% [11 of 22] versus 88% [21 of 24]). Potentially due to the lack of preparation, a greater percentage of participants ‘‘agreed or strongly agreed’’ that the workshop material was difficult. Overall perceptions were still positive, with 100% (22 of 22) indicating that they would recommend the workshop to others and would use what they learned as a practicing pathologist (TABLE and SUPPLEMENTAL FIGURE). The major areas for improvement involved technical issues and confusion regarding website use. To address these concerns, for the third workshop, we provided links to the websites in preworkshop materials and asked participants to ‘‘take a few minutes’’ to familiarize themselves in advance. We also switched from Google Docs to Google Forms, which are viewable on laptops and tablets, require no login or application download, allow easy linking to online tools, and provide participants with a record of their responses.10 In discussion, the facilitators also decided that preworkshop survey completion by participants should be required because this demonstrated commitment to full workshop participation. The third workshop had 26 participants. Two faculty instructors participated, and there were no table facilitators. The workshop was very well received. Addressing the use of fewer faculty, 75% of participants from the third workshop felt the number of faculty was ‘‘just right’’ with 25% (n ¼ 4) selecting ‘‘too little.’’ Comments from 2 of these residents also suggested increasing faculty support. Two attendees, however, stated that the best aspects of the workshop were ‘‘practical cases to work through with great faculty support’’ and that ‘‘we worked through the problems ourselves.’’ All respondents also agreed or strongly agreed with the statement: ‘‘Circulating faculty, as opposed to a faculty member sitting at each table throughout the session, provided adequate
EDUCATIONAL INNOVATION TABLE
Overall Workshop Perceptions Responding ‘‘Agree’’ or ‘‘Strongly Agree,’’ No. (%) ASCP 2013 (n ¼ 24)
USCAP 2014 (n ¼ 22)
ASCP 2014 (n ¼ 16)
Totals, No. (%)
The workshop met the stated learning objectives.
23 (96)
21 (95)
16 (100)
60 (97)
The workshop helped me to understand the clinical relevance of genomic pathology.
23 (96)
21 (95)
16 (100)
60 (97)
This workshop was appropriate for my current PGY training level.
21 (88)
19 (86)
15 (94)
55 (89)
The workshop enhanced my critical thinking skills.
22 (92)
21 (95)
16 (100)
59 (95)
The workshop discussions were engaging.
23 (96)
22 (100)
15 (100)
60 (97)
I felt comfortable using the presented online tools for genomics.
23 (96)
19 (86)
15 (94)
57 (92)
Our workshop discussions were productive.
23 (96)
22 (100)
15 (94)
60 (97)
I was an active participant in the small group discussions.
23 (96)
19 (86)
16 (100)
58 (94)
The workshop was a valuable use of my time.
22 (92)
21 (95)
16 (94)
59 (95)
I would recommend this workshop to other residents.
22 (92)
22 (100)
16 (100)
60 (97)
This workshop will help me as a practicing pathologist.
21 (88)
21 (95)
16 (100)
58 (94)
Abbreviations: ASCP, American Society for Clinical Pathology; USCAP, United States and Canadian Academy of Pathology; PGY, postgraduate year.
support.’’ Aside from the above, there were no other integrated into the TBL process. In addition, our consistent themes for improvement. evaluation approach does not yet measure knowledge acquisition. To further evolve our innovation, we are developDiscussion ing a knowledge test and implementing 2- and 3-hour Using an incremental approach, we implemented a 1- workshops to assess the utility of this model in day TBL genomic pathology workshop for residents settings with less available session time. We have also made all teaching materials and an instructor that garnered high ratings at 3 national meetings. There have been several reports of TBL in graduate handbook available online at no cost and are medical education consisting of multiple sessions planning train-the-trainer sessions to facilitate broad8 taking place at single programs.11,12 Similar to our er dissemination for local implementation. work, the results of those studies demonstrated the feasibility and acceptability of this approach. Our effort has shown that TBL can succeed with a single session and a diverse group of residents not familiar with each other or the instructor. Given our positive outcome and literature supporting the effectiveness of TBL, our innovative approach could be applied to other specialties, topics, and learners at regional or national meetings. Our study has some limitations. The residents attending the workshops were self-selected (ie, they chose to attend the workshop), limiting generalizability. While the 2014 ASCP workshop was well received, there were requests for more faculty involvement by a minority of residents. In the future, we plan to have both faculty instructors more actively
Conclusion Our work is the first, to our knowledge, to describe the implementation and evaluation of a TBL workshop at a national meeting. The approach allowed a small number of faculty to instruct a relatively large number of learners, and was found to be highly acceptable by resident participants.
References 1. Twenge JM. Generational changes and their impact in the classroom: teaching Generation Me. Med Educ. 2009;43(5):398–405.
Journal of Graduate Medical Education, February 1, 2016
83
EDUCATIONAL INNOVATION
2. Parmelee D, Michaelsen LK, Cook S, Hudes PD. Teambased learning: a practical guide: AMEE guide no. 65. Med Teach. 2012;34(5):e275–e287. 3. Sisk RJ. Team-based learning: systematic research review. J Nurs Educ. 2011;50(12):665–669. 4. Haidet P, Levine RE, Parmelee DX, Crow S, Kennedy F, Kelly PA, et al. Perspective: guidelines for reporting team-based learning activities in the medical and health sciences education literature. Acad Med. 2012;87(3):292–299. 5. Feero WG, Green ED. Genomics education for health care professionals in the 21st century. JAMA. 2011;306(9):989–990. 6. Haspel RL, Saffitz JE. Genomic oncology education: an urgent need, a new approach. Cancer J. 2014;20(1):91–95. 7. Haspel RL, Arnaout R, Briere L, Kantarci S, Marchand K, Tonellato P, et al. A call to action: training pathology residents in genomics and personalized medicine. Am J Clin Pathol. 2010;133(6):832–834. 8. Training Residents in Genomics. Resources. http:// www.pathologylearning.org/trig/resources. Accessed August 24, 2015. 9. Michaelsen LK, Sweet M. The essential elements of team-based learning. New Dir Teach Learn. 2008;2008(116):7–27. 10. Google. About Google docs and forms. http://www. google.com/forms/about. Accessed August 24, 2015.
84
Journal of Graduate Medical Education, February 1, 2016
11. McMullen I, Cartledge J, Levine R, Iversen A. Teambased learning for psychiatry residents: a mixed methods study. BMC Med Educ. 2013;13:124. 12. Brandler TC, Laser J, Williamson AK, Louie J, Esposito MJ. Team-based learning in a pathology residency training program. Am J Clin Pathol. 2014;142(1):23–28.
Richard L. Haspel, MD, PhD, is Chair, Training Residents in Genomics Working Group, and Director, Pathology Residency Training Program, Beth Israel Deaconess Medical Center, and Assistant Professor of Pathology, Harvard Medical School; Asma M. Ali, MS, is Director of Evaluation, Measurement, and Assessment, American Society for Clinical Pathology; and Grace C. Huang, MD, is Associate Program Director, Internal Medicine Residency Program, and Director of Assessment, Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, and Associate Professor of Medicine, Harvard Medical School. Funding: This work was supported by the National Institutes of Health (1R25CA168544-01). Conflict of interest: The authors declare they have no competing interests. The authors would like to thank Matthew H. Smith for his review of the manuscript. Corresponding author: Richard L. Haspel, MD, PhD, Beth Israel Deaconess Medical Center, Yamins 309, 330 Brookline Avenue, Boston, MA 02215, 617.667.3648, [email protected] Received May 25, 2015; revision received July 30, 2015; accepted August 12, 2015.
Using a Team-Based Learning Approach at National Meetings to Teach Residents Genomic Pathology Richard L. Haspel, MD, PhD Asma M. Ali, MS Grace C. Huang, MD ABSTRACT Background Accumulating data suggest that team-based learning (TBL) is more effective than lecture-based teaching strategies. Educational sessions at national meetings, however, tend to be lecture-based, and unlike most examples of TBL, involve participants who do not know each other or the instructor. Objective We evaluated a 1-day TBL genomic pathology workshop for residents held at 3 national meetings. Methods A committee of experts developed the workshop. Prior to attending, participants were provided access to readings and asked to answer preparation questions. Each of the 4 modules within the workshop consisted of a 60-minute TBL activity flanked by 15- to 30-minute preactivity and postactivity lectures. We used surveys to acquire participant evaluation of the workshop. Results From 2013–2014, 86 pathology residents from 61 programs participated in 3 workshops at national meetings. All workshops were well received, with over 90% of attendees indicating that they would recommend them to other residents and that the material would help them as practicing pathologists. An incremental approach facilitated decreasing faculty presence at the workshops: the first 2 workshops had 7 faculty each (1 facilitator for each team and 1 circulating faculty member), while the final workshop involved only 2 faculty for 6 teams. For this final session, participants agreed that circulating faculty provided adequate support. Participant ‘‘buy-in’’ (requiring completion of a preworkshop survey) was critical in enabling a TBL approach. Conclusions These results demonstrate that TBL is a feasible and effective strategy for teaching genomic medicine that is acceptable to pathology residents at national meetings.
Introduction National conferences are important opportunities for continuing education and dissemination of new findings. Sessions at these meetings are often lecture based. Many trainees also attend these meetings, and interactive instructional strategies, including teambased learning (TBL), may be more effective with this generation of learners.1 As an important innovation in medical education, TBL emphasizes learner preparation outside of class and knowledge application inside the classroom.2 Medical schools and some residency programs have already incorporated TBL into their curricula.3 In these settings, TBL is often implemented during multiple longitudinal sessions, and teams involve learners who know each other and the instructor. Learners also are motivated to review DOI: http://dx.doi.org/10.4300/JGME-D-15-00221.1 Editor’s Note: The online version of this article contains a workshop preparation checklist, preworkshop and postworkshop survey results, and a figure of individual module perceptions.
80
Journal of Graduate Medical Education, February 1, 2016
preclass materials because they may be formally evaluated on their learning, and performance in front of classmates may be a source of positive social pressure.4 In contrast, at national meetings, attendees often do not know each other or the instructors, attend only a single session, and are not graded on performance. These differences may influence interest, participation, and instructional effectiveness. We describe the development and associated outcomes of a 1-day TBL genomic pathology workshop for residents held at 3 national conferences. The importance of genomic medicine training of health care professionals has been widely recognized.5 Furthermore, the need for hands-on instruction using online tools and consensus discussions on complicated topics (eg, reporting of variants of uncertain significance and ethical issues) make this area an especially ideal subject for TBL. To our knowledge, TBL has not been previously implemented and evaluated at a national meeting, and our results demonstrate the utility of this innovative approach.
EDUCATIONAL INNOVATION
Methods Workshop Development In 2010, the Training Residents in Genomics Working Group (TRIG WG) was formed through the Program Directors Section of the Association of Pathology Chairs to develop teaching tools for pathology residents.6 This group includes experts in genomic pathology, medical education, genetic counseling, and medical genetics, with representation from major pathology organizations, the National Society of Genetic Counselors, and the American College of Medical Genetics and Genomics. The TRIG WG began by revising a pathology resident curriculum originally developed at Beth Israel Deaconess Medical Center.7 Members of the TRIG WG refined the curriculum in an iterative, collaborative fashion into a 1-day TBL workshop for residents attending national pathology meetings.8 The workshop consisted of 4 modules following the clinical narrative of a single patient with breast cancer through the course of her disease and the different testing modalities utilized: single gene testing (module 1), prognostic gene panel testing (module 2), cancer gene panels (module 3), and whole exome sequencing (module 4). Next-generation sequencing is best understood with some molecular pathology background. As such, the curriculum was developed for residents who had completed a molecular pathology rotation. Approximately 2 to 3 weeks prior to the workshop, residents were e-mailed prereading and multiplechoice preparation questions. For the in-person session, each module consisted of 3 components: 1. An instructor-delivered 15- to 30-minute PowerPoint lecture reviewing the answers to the preparation questions and relaying other content needed for the TBL activity. 2. A 60-minute activity consisting of teams of 3 to 6 residents answering a series of questions, including ‘‘reveal’’ type and other questions necessitating the use of online genomic tools (participants were asked to bring a laptop computer or tablet to the workshop).9 3. An instructor-delivered 15- to 30-minute PowerPoint lecture presenting answers to the questions and incorporating a discussion of team responses. The workshop concluded with a 30-minute summary lecture that included major teaching points. Participants were given access to the activities at the workshop electronically (initially Google Docs and
What was known and gap While evidence supports team-based learning (TBL) as more effective, sessions at national meetings generally are lecture based. What is new An objective evaluation of a 1-day TBL workshop in genomic pathology for residents held at 3 national meetings. Limitations Self-selection of participants limits generalizability; the study did not assess learning outcomes. Bottom line A TBL approach at national meetings is feasible from a workforce and logistics perspective, and was well received by participants.
later Google Forms).10 Eight hours (with lunch and breaks) were allotted for the overall workshop, 90 minutes for each module. The workshop was implemented at the 2013 and 2014 American Society for Clinical Pathology (ASCP) Annual Meeting, and the 2014 United States and Canadian Academy of Pathology (USCAP) Annual Meeting. We solicited information about residents via preworkshop surveys to ensure appropriate team composition (eg, balance across postgraduate years, access to a computer or tablet, and background in molecular pathology). While participation for both ASCP 2013 and 2014 required completion of this survey, it was not required for USCAP 2014. Participating residents paid registration fees for the annual meetings, and no additional fees were required to attend the workshops. The targeted enrollment for each workshop was 20 to 40 residents. TBL typically involves 1 instructor managing multiple teams of known learners; however, given the novel application of TBL to a national meeting, we used an incremental approach to decrease the number of required faculty over time. At the first 2 workshops, there were 7 faculty at each session. One or 2 faculty members provided the preactivity and postactivity lectures. One facilitator was also placed at each of 6 team tables (4 to 6 residents), and the remaining faculty member circulated to monitor progress. The facilitators were TRIG WG members with knowledge of the curriculum and some background, but not necessarily expertise, in molecular pathology or genomics. Facilitators participated in a preworkshop, 60-minute conference call to review formats and were instructed to primarily observe and to only assist if the team was having trouble moving forward. For the third workshop, to implement true TBL, 2 workshop faculty served as Journal of Graduate Medical Education, February 1, 2016
81
EDUCATIONAL INNOVATION
instructors, and there were no facilitators for the 6 teams (consisting of 3 to 5 residents in each team). Both faculty had participated in 1 of the prior workshops; 1 had content expertise while the other had expertise in TBL. This second faculty member circulated among teams during the activity component, primarily observing and intervening only if a team had significant difficulty moving forward with the module activity. Workshop development as well as other genomics education initiatives were funded by a R25 grant from the National Cancer Institute awarded to the chair of the TRIG WG in 2012. This provided support for the chair (30%), a TRIG WG faculty member (~5%), an evaluation expert (~5%), and an educational design subcontract with ASCP (~$95,000 per year). Other faculty were not remunerated for participation in the workshops or the monthly 60-minute TRIG WG conference calls. The pathology organizations and the grant provided funding for workshop costs, which varied by venue and included room setup, lunch for attendees, and faculty travel. Estimated workshop preparation time was 4 to 6 hours for lecturers and 2 hours for facilitators. This project was approved for exempt status by the Institutional Review Board at Beth Israel Deaconess Medical Center.
Methods of Evaluation The resident postworkshop survey was developed by 2 authors with expertise in psychometrics and evaluation (A.M.A. and G.C.H.) and was piloted (including cognitive interviews) with Beth Israel Deaconess Medical Center pathology residents who participated in a local version of the workshop. The survey solicited the following information: demographics, global and module-specific perceptions, and free-text responses for workshop strengths and areas for improvement. Responses were anonymous and tabulated by workshop. Results were presented at TRIG WG meetings and used to refine the workshop.
Results The 3 workshops were attended by 86 residents, and the response rate of the postworkshop surveys was 72% (62 of 86). Sixty-one residency programs were represented, and the majority of participants (78%, 67 of 86) were from US programs. The composition by postgraduate year was similar across workshops. Data from the preworkshop and postworkshop surveys are provided as online supplemental material.
82
Journal of Graduate Medical Education, February 1, 2016
The first workshop had 30 participants, with almost all attendees (. 90%) having positive perceptions (TABLE and SUPPLEMENTAL FIGURE). Areas for improvement suggested by participants included website technical issues and more next-generation sequencing–related instruction. To address these concerns, we provided instructions to the participants in the event of a slow website, and modules 3 and 4 were reworked to expand the introduction to nextgeneration sequencing and include an activity involving genomic alignment data analysis. The second workshop had 30 participants. In contrast to the first workshop, a smaller number of residents rated their preparation as ‘‘just right’’ (36% [8 of 22] versus 75% [18 of 24]) and agreed that they ‘‘had put effort into preparing for the workshop’’ (50% [11 of 22] versus 88% [21 of 24]). Potentially due to the lack of preparation, a greater percentage of participants ‘‘agreed or strongly agreed’’ that the workshop material was difficult. Overall perceptions were still positive, with 100% (22 of 22) indicating that they would recommend the workshop to others and would use what they learned as a practicing pathologist (TABLE and SUPPLEMENTAL FIGURE). The major areas for improvement involved technical issues and confusion regarding website use. To address these concerns, for the third workshop, we provided links to the websites in preworkshop materials and asked participants to ‘‘take a few minutes’’ to familiarize themselves in advance. We also switched from Google Docs to Google Forms, which are viewable on laptops and tablets, require no login or application download, allow easy linking to online tools, and provide participants with a record of their responses.10 In discussion, the facilitators also decided that preworkshop survey completion by participants should be required because this demonstrated commitment to full workshop participation. The third workshop had 26 participants. Two faculty instructors participated, and there were no table facilitators. The workshop was very well received. Addressing the use of fewer faculty, 75% of participants from the third workshop felt the number of faculty was ‘‘just right’’ with 25% (n ¼ 4) selecting ‘‘too little.’’ Comments from 2 of these residents also suggested increasing faculty support. Two attendees, however, stated that the best aspects of the workshop were ‘‘practical cases to work through with great faculty support’’ and that ‘‘we worked through the problems ourselves.’’ All respondents also agreed or strongly agreed with the statement: ‘‘Circulating faculty, as opposed to a faculty member sitting at each table throughout the session, provided adequate
EDUCATIONAL INNOVATION TABLE
Overall Workshop Perceptions Responding ‘‘Agree’’ or ‘‘Strongly Agree,’’ No. (%) ASCP 2013 (n ¼ 24)
USCAP 2014 (n ¼ 22)
ASCP 2014 (n ¼ 16)
Totals, No. (%)
The workshop met the stated learning objectives.
23 (96)
21 (95)
16 (100)
60 (97)
The workshop helped me to understand the clinical relevance of genomic pathology.
23 (96)
21 (95)
16 (100)
60 (97)
This workshop was appropriate for my current PGY training level.
21 (88)
19 (86)
15 (94)
55 (89)
The workshop enhanced my critical thinking skills.
22 (92)
21 (95)
16 (100)
59 (95)
The workshop discussions were engaging.
23 (96)
22 (100)
15 (100)
60 (97)
I felt comfortable using the presented online tools for genomics.
23 (96)
19 (86)
15 (94)
57 (92)
Our workshop discussions were productive.
23 (96)
22 (100)
15 (94)
60 (97)
I was an active participant in the small group discussions.
23 (96)
19 (86)
16 (100)
58 (94)
The workshop was a valuable use of my time.
22 (92)
21 (95)
16 (94)
59 (95)
I would recommend this workshop to other residents.
22 (92)
22 (100)
16 (100)
60 (97)
This workshop will help me as a practicing pathologist.
21 (88)
21 (95)
16 (100)
58 (94)
Abbreviations: ASCP, American Society for Clinical Pathology; USCAP, United States and Canadian Academy of Pathology; PGY, postgraduate year.
support.’’ Aside from the above, there were no other integrated into the TBL process. In addition, our consistent themes for improvement. evaluation approach does not yet measure knowledge acquisition. To further evolve our innovation, we are developDiscussion ing a knowledge test and implementing 2- and 3-hour Using an incremental approach, we implemented a 1- workshops to assess the utility of this model in day TBL genomic pathology workshop for residents settings with less available session time. We have also made all teaching materials and an instructor that garnered high ratings at 3 national meetings. There have been several reports of TBL in graduate handbook available online at no cost and are medical education consisting of multiple sessions planning train-the-trainer sessions to facilitate broad8 taking place at single programs.11,12 Similar to our er dissemination for local implementation. work, the results of those studies demonstrated the feasibility and acceptability of this approach. Our effort has shown that TBL can succeed with a single session and a diverse group of residents not familiar with each other or the instructor. Given our positive outcome and literature supporting the effectiveness of TBL, our innovative approach could be applied to other specialties, topics, and learners at regional or national meetings. Our study has some limitations. The residents attending the workshops were self-selected (ie, they chose to attend the workshop), limiting generalizability. While the 2014 ASCP workshop was well received, there were requests for more faculty involvement by a minority of residents. In the future, we plan to have both faculty instructors more actively
Conclusion Our work is the first, to our knowledge, to describe the implementation and evaluation of a TBL workshop at a national meeting. The approach allowed a small number of faculty to instruct a relatively large number of learners, and was found to be highly acceptable by resident participants.
References 1. Twenge JM. Generational changes and their impact in the classroom: teaching Generation Me. Med Educ. 2009;43(5):398–405.
Journal of Graduate Medical Education, February 1, 2016
83
EDUCATIONAL INNOVATION
2. Parmelee D, Michaelsen LK, Cook S, Hudes PD. Teambased learning: a practical guide: AMEE guide no. 65. Med Teach. 2012;34(5):e275–e287. 3. Sisk RJ. Team-based learning: systematic research review. J Nurs Educ. 2011;50(12):665–669. 4. Haidet P, Levine RE, Parmelee DX, Crow S, Kennedy F, Kelly PA, et al. Perspective: guidelines for reporting team-based learning activities in the medical and health sciences education literature. Acad Med. 2012;87(3):292–299. 5. Feero WG, Green ED. Genomics education for health care professionals in the 21st century. JAMA. 2011;306(9):989–990. 6. Haspel RL, Saffitz JE. Genomic oncology education: an urgent need, a new approach. Cancer J. 2014;20(1):91–95. 7. Haspel RL, Arnaout R, Briere L, Kantarci S, Marchand K, Tonellato P, et al. A call to action: training pathology residents in genomics and personalized medicine. Am J Clin Pathol. 2010;133(6):832–834. 8. Training Residents in Genomics. Resources. http:// www.pathologylearning.org/trig/resources. Accessed August 24, 2015. 9. Michaelsen LK, Sweet M. The essential elements of team-based learning. New Dir Teach Learn. 2008;2008(116):7–27. 10. Google. About Google docs and forms. http://www. google.com/forms/about. Accessed August 24, 2015.
84
Journal of Graduate Medical Education, February 1, 2016
11. McMullen I, Cartledge J, Levine R, Iversen A. Teambased learning for psychiatry residents: a mixed methods study. BMC Med Educ. 2013;13:124. 12. Brandler TC, Laser J, Williamson AK, Louie J, Esposito MJ. Team-based learning in a pathology residency training program. Am J Clin Pathol. 2014;142(1):23–28.
Richard L. Haspel, MD, PhD, is Chair, Training Residents in Genomics Working Group, and Director, Pathology Residency Training Program, Beth Israel Deaconess Medical Center, and Assistant Professor of Pathology, Harvard Medical School; Asma M. Ali, MS, is Director of Evaluation, Measurement, and Assessment, American Society for Clinical Pathology; and Grace C. Huang, MD, is Associate Program Director, Internal Medicine Residency Program, and Director of Assessment, Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, and Associate Professor of Medicine, Harvard Medical School. Funding: This work was supported by the National Institutes of Health (1R25CA168544-01). Conflict of interest: The authors declare they have no competing interests. The authors would like to thank Matthew H. Smith for his review of the manuscript. Corresponding author: Richard L. Haspel, MD, PhD, Beth Israel Deaconess Medical Center, Yamins 309, 330 Brookline Avenue, Boston, MA 02215, 617.667.3648, [email protected] Received May 25, 2015; revision received July 30, 2015; accepted August 12, 2015.
