208
Original article
Modular continuing professional development for emergency physicians – the MNSHA masterclass programme
Eefje E.P. van den Berga, Amanda Braceyc, A. Pieter G. van Drielb, Femke E.C. Geijsel and Sander Mandersd,e Introduction Emergency physician (EP) training in the Netherlands is currently a 3-year competency-based programme. Upon its completion, many Dutch EPs feel that they lack some of the skills and the knowledge necessary to be fully prepared for the challenges of emergency medicine. We perceived a need for new methods to deliver continuing professional development (CPD). Methods A needs analysis survey was conducted to ascertain whether our perceived need for a new CPD programme was genuine. A new course was developed, incorporating innovative learning methods. The results of the needs analysis were incorporated into the programme. Another survey was held among participants of the first two editions of the programme. Results, discussion and conclusion Modular CPD for Emergency Physicians (Dutch: MNSHA) is a modular programme that aims to deliver CPD to Dutch EPs. It combines innovative educational methods, such as asynchronous learning in a flipped classroom, with webbased mentoring. The aim is for participants to develop
Introduction Emergency physician (EP) training in the Netherlands is currently a 3-year competency-based training programme. It is not recognized as a specialist training programme within the European Union, as the European Union Doctor’s Directive mandates that a training scheme for EPs must be a minimum of 5 years [1]. Upon completion of the programme, many EPs in the Netherlands feel that they still lack some of the skills and the knowledge necessary to be fully prepared for the challenges of emergency medicine (EM). This is due to time constraints within the 3-year training programme [2], a paucity of role models with specific training and experience in EM, and the often limited independence EPs have in managing Emergency Department patients within Dutch hospitals. We wanted to create a modular continuing professional development (CPD) programme that specifically focuses on the needs of Dutch EPs, combining several educational methods.
Methods First, we conducted a brief survey among all EPs registered in the Netherlands. This needs analysis intended All supplementary digital content is available directly from the corresponding author. 0969-9546 Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
effective, individualized and sustainable methods to gain and maintain knowledge and skills as a part of their ongoing professional education. The participant survey showed encouraging results, strongly suggesting an improvement in confidence. A more robust study would be required to better assess the outcomes of our programme. European Journal of Emergency Medicine 23:208–213 Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved. European Journal of Emergency Medicine 2016, 23:208–213 Keywords: asynchronous learning, continuing professional development, emergency medicine, emergency physician, flipped classroom, MNSHA, web-based learning a
Emergency Department, TweeSteden Ziekenhuis, bEmergency Department, St. Elisabeth Ziekenhuis, Tilburg, The Netherlands, cEmergency Department, Gosford Hospital, Gosford, New South Wales, dEmergency Department, Royal Darwin Hospital and eCareFlight Top End Medical Retrieval Service Correspondence to Eefje E.P. van den Berg, Saliehof 28, 5044 AR Tilburg, the Netherlands E-mail: [email protected] Received 2 April 2014 Accepted 11 November 2014
to explore whether we were correct in our perception that there was a need for a new CPD programme, as well as to get an idea of which topics Dutch EPs most wanted to be included in this programme. Subsequently, we identified several innovative learning methods that have been described recently in the literature [3–13] and that seemed suitable for use in our CPD programme. We will further address these below. We then proceeded to develop the first module of our programme, which was named ‘Modular CPD for Emergency Physicians’ (Dutch: MNSHA). Finally, during and after the first two times the MNSHA course was conducted, we surveyed participants to assess whether they felt that the programme was helpful. Participants were followed prospectively and were asked to fill in three surveys: one premasterclass survey and two postmasterclass surveys [at the end of the last day of the masterclass and 10 months (for the first group) or 6 months (for the second group) after finishing the masterclass, respectively]. All surveys contained questions about the setup of the masterclass, as well as questions intended to stimulate reflection on the development of participants’ abilities and confidence. DOI: 10.1097/MEJ.0000000000000235
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 209
SPSS (SPSS Inc., Chicago, Illinois, USA) and Excel 2010 were used for statistical analysis. T-test with paired samples was used, and results were considered to be significant at a level of 0.05.
Results Needs analysis
In September 2012, we conducted a survey among all 231 EPs registered in the Netherlands to explore any perceived limitations in their knowledge and skills. A total of 85 EPs (37%) completed the survey. The characteristics of this group are listed in the Appendix, Supplementary Table 1.1. Of the responding EPs, 80% stated that they felt that they did not sufficiently keep their knowledge and skills up-to-date. As many as 40% did not set aside as much time as they felt necessary to remain current with their CPD (Appendix, Supplementary Table 1.2 a–d). The most important reasons given were the lack of time and the inability to efficiently find the right material out of all the information available (Appendix, Supplementary Table 1.2 e). Further questions focussed specifically on cardiovascular and respiratory topics, which were the themes of the first MNSHA module, as we will discuss below. Participants were asked which topics, knowledge, or skills they felt they needed to gain more confidence in their daily practice. The five most commonly given answers are listed in Table 1 (see also Appendix 1, Supplementary Table 1.3). Learning methods
Several innovative learning methods have been described in the literature recently [3–13]. We selected learning and educational methods that we felt could work effectively in a CPD programme for EPs within the Dutch context. An overview of these methods is given in Table 2. The combined use of these educational methods, together with a strong mentoring programme, has not been previously described for CPD of EPs. Course description Course development
MNSHA has been developed by a small group of EPs from the Netherlands and Australia to aid in the Table 1 Topics, knowledge, or skills participants think they need to gain more confidence in their daily practice Cardiovascular EM Arrhythmias ECG interpretation Medication use Heart failure Acute coronary syndromes
Respiratory EM 44% 22% 18% 14% 14%
Antibiotics and other medication Chest drain insertion Asthma and COPD Pulmonary function tests Chest radiograph interpretation
22% 15% 12% 12% 6%
COPD, chronic obstructive pulmonary disease; ECG, electrocardiogram; EM, emergency medicine.
professional development of Dutch EPs. It is targeted at EPs who have completed the 3-year Dutch EM training programme and are currently practicing in Emergency Departments throughout the Netherlands. The MNSHA programme is primarily aimed at providing their knowledge and skills necessary for improved, consultant-level decision-making. EPs should develop an effective and sustainable way to gain, maintain and develop knowledge and skills as part of their ongoing professional education. MNSHA has been designed as a modular programme, with the first two modules developed to date. Cardiovascular and respiratory emergencies were selected as the themes for the first module, as these are core areas to which all Dutch EPs are regularly exposed. A second module, which covers a variety of acute medical emergencies, was launched in January 2014. It is envisaged that several more modules will be developed, with topics covering the depth and breadth of EM. Within the time constraints of a relatively short programme, it is impossible to cover all topics in a given area of EM. Therefore, a conscious decision was made to address a limited number of crucial topics in depth, so that the participants can master these at a consultant level. For this reason, we describe our course as a ‘masterclass’. Topics have been carefully selected for their richness of content, layers of evidence, and areas of controversy. Approximately a quarter of all cases are paediatric, and the development of a specific paediatric module is being considered. Upon completion of the masterclass, participants should be able to utilize the skills that they have practiced, to continue their professional development by addressing topics that have not been encountered during the masterclass. Each MNSHA masterclass spans a total of 15 weeks, of which most time is spent in a self-directed study with regular mentor contact. There are two face-to-face meetings with all participants, and the masterclass culminates in three more face-to-face days, described below. A study guide was written, in which clinical cases are used to pose relevant questions to the participants. A limited number of learning goals are provided, along with a rough timeline to facilitate participants’ progress through the material and to ensure that all participants have covered the same topics at the face-to-face days. A list of suggested resources is also provided. Through selfdirected study with guidance from mentors, participants must develop techniques that allow them to access and use the wealth of information available in a timeappropriate manner. Participants should define gaps in their knowledge, fill these gaps with up-to-date information, strengthen their ability to critically appraise evidence and guidelines, and establish reasoned opinions on current controversies in EM. This acquisition of content is learner driven, with no prescribed target level. The further refinement and integration of knowledge and the
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
210 European Journal of Emergency Medicine 2016, Vol 23 No 3
Table 2
Learning and educational methods
Learning method
Explanation
Asynchronous learning
Distance learning, Internet learning, FOAM
Self-directed learning
Flipped classroom
Mentoring
Problem-based learning
An educational method that utilizes individualized learning resources, to enable student-centred teaching to take place outside the limitations of time and place. Asynchronous learning gives learners the opportunity to use their preferred learning style, and targets an individual’s knowledge gaps [3]. It is also referred to as individual interactive instruction. This technique is being increasingly incorporated into education schemes of medical students and junior doctors, alongside the more traditional synchronous didactic methods [4,5] Distance learning provides access to learning when the source of information and the learners are separated by time, distance, or both. Internet learning is an example of modern distance learning [6]. One of the newer developments in Internet medical education is Free Open-Access Medical Education (FOAM). It comprises an exploding collection of constantly evolving, collaborative, and interactive open-access medical education resources that are being freely distributed over the Internet. FOAM includes blogs, podcasts, online videos, text documents, and photographs [7] In self-directed learning, learners are responsible for their own learning process. They are self-managing (context, social setting, resources, and actions) and self-monitoring (evaluating and regulating their cognitive learning strategies). Teachers model learning strategies and work with students so that they develop the ability to use them on their own. Selfdirected learning is asynchronous [8] The flipped classroom is an educational model in which traditional lecture material is studied before attending a teaching session. Teaching time is then free to be used for active learning and the application of what has been learned through thought-provoking exercises [4,9] and the development of meta-cognition (e.g. critical thinking, problem-solving [10]) A mentor is a trusted and experienced person who has a direct interest in the professional development and education of another person, and provides guidance and support [11,12]. The mentor may have a number of roles including advisor, supporter, tutor, and role model. Individual mentoring, group mentoring, and telementoring (mentoring through the Internet) are different forms of mentoring [11] A learning method that aims at efficient acquisition and structuring of knowledge, arising out of working in active, iterative, and self-directed ways. It comprises a progressive framework of problems providing context, relevance, and motivation (problem-first learning), and builds on prior knowledge integration, critical thinking, reflection on learning, and enjoyment. It achieves its goals through facilitated small-group work and independent study [13]
development of metacognition are aided by the mentoring process. The use of an online forum, open to participants and mentors, is encouraged for the sharing of resources, ideas and questions. It is hoped that this will ultimately translate to more sound, evidence-based clinical reasoning and improved patient care in daily practice. An overview of the MNSHA programme is accessible through the MNSHA website (http://www.mnsha.nl/en). Mentoring
Each participant is assigned a mentor with whom they have regular contact throughout the module. All mentors are EM specialists with experience in education. They are mostly based overseas (predominantly in Australia and the UK), as the required level of expertise is not currently widely available in the Netherlands. Mentor sessions are primarily web-based to allow live conversation. The two major aims for mentors are to function as a role model (i.e. an EP functioning at a level participants can aspire to achieve) and to help participants integrate information to enhance insight into the subject matter, enabling them to achieve the desired learning outcomes. Mentor sessions are intended to be learner driven, with the mentor providing personalized guidance, stimulating critical thinking and problem-solving, and tailoring discussions to the participants’ individual needs. Face-to-face meetings
At the start of the masterclass, an initial meeting allows participants to meet each other and some of the mentors. It introduces the format and learning objectives of the course, and gives suggestions on how to approach the vast
array of medical resources available. Aims and expectations are discussed, and the desired level of knowledge and skills is clarified. A second face-to-face meeting with all the participants is held approximately a third of the way through the masterclass. Participants are asked to complete formative multiple-choice questions before this session, which are related to the topics studied to that date. The multiplechoice questions, several cases and visual material are utilized to stimulate discussion and challenge participants in a small group setting. Participants are able to reflect upon their learning to that date and their knowledge is challenged. The masterclass is concluded with three consecutive face-to-face days, which allow participants to discuss the topics studied and develop a meta-understanding of the subject matter [14]. Case-based, clinically relevant discussions are facilitated by faculty, in small and larger groups. Other educational methods used include a journal club, pro–con debate and workshops. Participants are encouraged to develop strategies for dealing with controversies in complex clinical situations, as well as a personal strategy for their CPD. Although there is no defined target level of knowledge, mentors and facilitators demonstrate by example what level the participants should be aspiring to achieve. The emphasis is on thinking processes, the general approach to a problem, and ‘consultant-level’ decision-making. However, a certain level of content acquisition is needed as well. To assist with their understanding of this level, participants complete a formative assessment at the end
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 211
of the masterclass days. This assessment consists of objective structured clinical examination questions in a formal examination setting.
Fig. 2
9 8 7
Figures 1–3 show the improvements in self-ranking on knowledge, the ability to treat patients and the ability to discuss assessment and management controversies and cases with other specialists directly after the masterclass compared with before the masterclass (P-value ranges from < 0.001 to 0.59) and 6 or 10 months after the masterclass compared with before the masterclass (P-value ranges from < 0.001 to 0.44).
F O PD
C
C
C
th m a Sy nc pi op ra Ta tor e y ch ya arre st Re rrh yt sp h m ira ia to s ry Pn fa eu ilur Ao mo e th rti or c ax di ss ec tio n
PE
As
ar di or es C
Overall, participants gave the first MNSHA masterclass an average score of 8.9 on a 10-point scale. Before the masterclass (Figs 1–3, ‘pre’), none of the participants felt that they were sufficiently up-to-date, whereas immediately after the masterclass (‘post 1’) and after 10/6 months (‘post 2’), 100 and 82%, respectively, felt that they were up-todate. A significant improvement is also seen in the selfranking of their ability to find and efficiently use resources (from 5.1 to 8.1, and 7.5, respectively, out of 10, P < 0.001).
S
From September to December 2012, 21 participants took part in the first edition, and from March to June 2013 another 16 participants took part in the second edition. All 37 participants who started the masterclass also finished the masterclass. Twenty-two participants completed the premasterclass survey (59%), 33 responded to the first postmasterclass survey (89%) and 18 responded to the second postmasterclass survey (48%).
6 5 4 3 2 1 0 AC
Participant survey
Pre
Post 1
Post 2
How comfortable do you feel treating a patient with... Significant: P-value < 0.05. ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Fig. 3
9 8 7 6 5 4 3 2
As
8 7
th m a es Sy nc pi ra o Ta tory pe ch ar ya re st Re rrh yt sp h m ira ia to s ry Pn fa eu ilur Ao mo e th rti or c ax di ss ec tio n C C F C O PD
AC
S
9
PE
1 0
Fig. 1
C
ar
di
or
6 5 4 3 2 1
Pre
PD
C F C
O C
P As E th di m or es Syn a pi co ra Ta tor pe y ch ar y r Re arrh est yt sp ira hm ia to s ry fa Pn eu ilur Ao mo e th rti or c ax di ss ec tio n C ar
AC
S
0
Pre
Post 1
Post 2
Please rate yourself as to what you feel your current knowledge is of... Significant: P-value < 0.05. ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Post 1
Post 2
How comfortable do you feel in a discussion with a Respiratory Physician or Cardiologist on a patient with a working diagnosis of... Significant: P-value < 0.05; ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Discussion and conclusion Several innovative educational methods were embraced by the MNSHA programme to deliver the course. An asynchronous approach to learning in a flipped classroom was utilized. The more traditional concept of mentoring was combined with these newer techniques in a modern, web-based environment.
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
212 European Journal of Emergency Medicine 2016, Vol 23 No 3
Needs analysis
From our brief survey, we concluded that a sizeable proportion of Dutch EPs do indeed perceive deficiencies in knowledge and skills in some of the most important topics within the specialty. We found that neither are the needs of EPs met fully during the 3-year training programme, nor are EPs currently able to effectively address their needs as part of their CPD. A discussion on the length of the Dutch EM training programme and perhaps improving the way it is delivered is outside the scope of this article. Our survey supported the prior perception that there is a need for better methods of delivering CPD to Dutch EPs. The majority of the topics that were most frequently requested were incorporated into the first module of the MNSHA programme. Asynchronous learning
Asynchronous learning is student-centred teaching, which targets the needs of the learner more specifically than traditional (synchronous) educational methods [3]. The elements of asynchronous learning incorporated into the MNSHA programme have intuitive advantages. EPs usually work shifts and are frequently unable to attend scheduled educational sessions. MNSHA participants can choose when they will study, within a rough guideline, so that over a period of time all participants arrive at the same learning outcomes. The participants’ diverse educational backgrounds are accommodated by asynchronous education, as they are free to access a vast array of educational material and can select what is appropriate to build on their current knowledge of a particular topic. To date, the majority of the literature on the use of asynchronous techniques pertains to education of medical students and junior medical doctors, not medical specialists. The literature on the use of asynchronous learning in EM is scarce [3,15]. A major disadvantage of asynchronous learning is that it relies on the learners’ abilities to identify their own knowledge gaps. When they are unable to do this effectively, the learners do not realize where their deficiencies lie and may therefore not study those areas (unconscious incompetence). Another disadvantage is that there is no immediate opportunity to discuss a topic with a group or ask questions, and there is no immediate feedback. These disadvantages are counterbalanced in our programme by the use of mentors, study groups and an online forum. Flipped classroom
Although there is little peer-reviewed literature supporting the benefits of the flipped classroom approach [4,9], this new approach to education offers several intuitive advantages. Participants have time to reflect on their study material and have the opportunity to rewind and repeat online lectures. Participants are able to control
when and where they access the educational material, and the education is learner driven as the responsibility for preclass preparation lies primarily with the participant [8]. Mentoring
Mentoring has been shown to be crucial to career development and success [1,12,16,17]. Effective mentoring in medicine leads to greater career satisfaction [17], and physicians with mentors have more confidence in their abilities [18]. Mentoring in EM is a poorly represented topic in the literature [11], especially with regard to mentoring of EPs themselves, despite the fact that mentoring is seen to play a critical role in all career stages [12]. There is as yet no published research assessing whether mentoring of physicians is correlated with improved patient outcomes. Participants survey
Our survey intended to evaluate whether the masterclass improved the knowledge and skills of the participants. It showed an improvement directly after the masterclass in the subjective belief of being up-to-date, in self-ranking of the ability to find and efficiently use resources, and in the confidence with every subject that was discussed in the masterclass. After 6 or 10 months, some of these selfrankings decreased, although they remained above premasterclass levels. These results are encouraging, but subject to several limitations. The main limitation of this survey is that it only looked at subjective improvements as rated by the participants. It was not practically feasible to perform a more objective testing of knowledge before and after the completion of the programme. Designing a study to assess for improvements in patient safety or clinical outcomes was also not feasible; hence, we instead chose to investigate subjective improvements in knowledge and confidence. A further limitation of this survey was the small number of participants, not all of whom returned all three surveys. Some bias may also have been caused by the fact that two groups where surveyed with different follow-up times (6 vs. 10 months after the masterclass). Conclusion
MNSHA combines innovative educational methods to deliver CPD to Dutch EPs. This combination of methods has not been previously described for a medical specialist CPD programme. The educational methods used by MNSHA seem intuitively advantageous to physicians, especially those in EM. They take previous knowledge and experience into consideration, and also allow physicians to access the educational material where and when they like. This is probably more suitable for EPs than traditional, synchronous educational strategies. Appropriate role models with specific knowledge and experience in EM are not yet widely available within the
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 213
Netherlands. A major component to the success of this course is the involvement of EPs from other, more mature EM systems as mentors to guide Dutch EPs through their learning process. Mentoring enables better application and retention of knowledge, as well as the development of consultant-level thinking. Our survey of participants showed some encouraging results, strongly suggesting an improvement in confidence. A more robust study would be required to better assess the outcomes of our programme. The MNSHA programme was developed specifically for EPs in the Netherlands. However, the authors believe that the combination of methods used could find wider application in medical education in other countries.
3
4 5
6
7 8 9
10
Acknowledgements A.P.vD, F.E.C.G. and S.M. have developed and are currently running the MNSHA programme, which this article describes. Conflicts of interest
11 12 13
14
There are no conflicts of interest.
15
References 1 EuSEM Taskforce on Curriculum. European curriculum for emergency medicine. 2009:25–4. Available at: http://www.eusem.org/cms/assets/1/ pdf/european_curriculum_for_em-aug09-djw.pdf. [Accessed 20 October 2013]. 2 Koning SW, Gaakeer MI, Veugelers R. Three-year emergency medicine training program in The Netherlands: first evaluation from the residents’ perspective. Int J Emerg Med 2013; 6:30.
16
17 18
Reiter DA, Lakoff DJ, Trueger NS, Shah KH. Individual interactive instruction: an innovative enhancement to resident education. Ann Emerg Med 2013; 61:110–113. Prober CG, Heath C. Lecture halls without lectures – a proposal for medical education. N Engl J Med 2012; 366:1657–1659. Sadosty AT, Goyal DG, Gene Hern H Jr, et al. Alternatives to the conference status quo: summary recommendations from the 2008 CORD Academic Assembly Conference Alternatives Workgroup. Acad Emerg Med 2009; 16 (Suppl 2):S25–S31. US Department of Education. Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. 2010. Available at: http://www2.ed.gov/rschstat/eval/tech/evidence-basedpractices/finalreport.pdf. [Accessed 15 October 2013]. Nickson CP, Cadogan MD. Free Open Access Medical education (FOAM) for the emergency physician. Emerg Med Australas 2014; 26:76–83. Abdullah MH. Self-directed learning. Available at: http://www.education.com/ reference/article/Ref_Self_Directed/mp. [Accessed 15 October 2013]. Educause Learning Initiative. 7 things you should know about flipped classrooms. 2012. Available at: http://www.educause.edu/library/resources. [Accessed 09 October 2013]. Martinez ME. What is metacognition? Phi Delta Kappan 2006; 87.9:696–699. Yeung M, Nuth J, Stiell IG. Mentoring in emergency medicine: the art and the evidence. CJEM 2010; 12:143–149. Garmel GM. Mentoring medical students in academic emergency medicine. Acad Emerg Med 2004; 11:1351–1357. Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009; 18:1–9. Metcalfe J, Shimamura AP. Metacognition: knowing about knowing. Cambridge, MA: MIT Press; 1994. Burnette K, Ramundo M, Stevenson M, Beeson MS. Evaluation of a webbased asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad Emerg Med 2009; 16 Suppl 2: S46–S50. Stamm M, Buddeberg-Fischer B. The impact of mentoring during postgraduate training on doctors’ career success. Med Educ 2011; 45:488–496. Farrell SE, Digioia NM, Broderick KB, Coates WC. Mentoring for clinicianeducators. Acad Emerg Med 2004; 11:1346–1350. Ramanan RA, Phillips RS, Davis RB, Silen W, Reede JY. Mentoring in medicine: keys to satisfaction. Am J Med 2002; 112:336–341.
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Original article
Modular continuing professional development for emergency physicians – the MNSHA masterclass programme
Eefje E.P. van den Berga, Amanda Braceyc, A. Pieter G. van Drielb, Femke E.C. Geijsel and Sander Mandersd,e Introduction Emergency physician (EP) training in the Netherlands is currently a 3-year competency-based programme. Upon its completion, many Dutch EPs feel that they lack some of the skills and the knowledge necessary to be fully prepared for the challenges of emergency medicine. We perceived a need for new methods to deliver continuing professional development (CPD). Methods A needs analysis survey was conducted to ascertain whether our perceived need for a new CPD programme was genuine. A new course was developed, incorporating innovative learning methods. The results of the needs analysis were incorporated into the programme. Another survey was held among participants of the first two editions of the programme. Results, discussion and conclusion Modular CPD for Emergency Physicians (Dutch: MNSHA) is a modular programme that aims to deliver CPD to Dutch EPs. It combines innovative educational methods, such as asynchronous learning in a flipped classroom, with webbased mentoring. The aim is for participants to develop
Introduction Emergency physician (EP) training in the Netherlands is currently a 3-year competency-based training programme. It is not recognized as a specialist training programme within the European Union, as the European Union Doctor’s Directive mandates that a training scheme for EPs must be a minimum of 5 years [1]. Upon completion of the programme, many EPs in the Netherlands feel that they still lack some of the skills and the knowledge necessary to be fully prepared for the challenges of emergency medicine (EM). This is due to time constraints within the 3-year training programme [2], a paucity of role models with specific training and experience in EM, and the often limited independence EPs have in managing Emergency Department patients within Dutch hospitals. We wanted to create a modular continuing professional development (CPD) programme that specifically focuses on the needs of Dutch EPs, combining several educational methods.
Methods First, we conducted a brief survey among all EPs registered in the Netherlands. This needs analysis intended All supplementary digital content is available directly from the corresponding author. 0969-9546 Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
effective, individualized and sustainable methods to gain and maintain knowledge and skills as a part of their ongoing professional education. The participant survey showed encouraging results, strongly suggesting an improvement in confidence. A more robust study would be required to better assess the outcomes of our programme. European Journal of Emergency Medicine 23:208–213 Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved. European Journal of Emergency Medicine 2016, 23:208–213 Keywords: asynchronous learning, continuing professional development, emergency medicine, emergency physician, flipped classroom, MNSHA, web-based learning a
Emergency Department, TweeSteden Ziekenhuis, bEmergency Department, St. Elisabeth Ziekenhuis, Tilburg, The Netherlands, cEmergency Department, Gosford Hospital, Gosford, New South Wales, dEmergency Department, Royal Darwin Hospital and eCareFlight Top End Medical Retrieval Service Correspondence to Eefje E.P. van den Berg, Saliehof 28, 5044 AR Tilburg, the Netherlands E-mail: [email protected] Received 2 April 2014 Accepted 11 November 2014
to explore whether we were correct in our perception that there was a need for a new CPD programme, as well as to get an idea of which topics Dutch EPs most wanted to be included in this programme. Subsequently, we identified several innovative learning methods that have been described recently in the literature [3–13] and that seemed suitable for use in our CPD programme. We will further address these below. We then proceeded to develop the first module of our programme, which was named ‘Modular CPD for Emergency Physicians’ (Dutch: MNSHA). Finally, during and after the first two times the MNSHA course was conducted, we surveyed participants to assess whether they felt that the programme was helpful. Participants were followed prospectively and were asked to fill in three surveys: one premasterclass survey and two postmasterclass surveys [at the end of the last day of the masterclass and 10 months (for the first group) or 6 months (for the second group) after finishing the masterclass, respectively]. All surveys contained questions about the setup of the masterclass, as well as questions intended to stimulate reflection on the development of participants’ abilities and confidence. DOI: 10.1097/MEJ.0000000000000235
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 209
SPSS (SPSS Inc., Chicago, Illinois, USA) and Excel 2010 were used for statistical analysis. T-test with paired samples was used, and results were considered to be significant at a level of 0.05.
Results Needs analysis
In September 2012, we conducted a survey among all 231 EPs registered in the Netherlands to explore any perceived limitations in their knowledge and skills. A total of 85 EPs (37%) completed the survey. The characteristics of this group are listed in the Appendix, Supplementary Table 1.1. Of the responding EPs, 80% stated that they felt that they did not sufficiently keep their knowledge and skills up-to-date. As many as 40% did not set aside as much time as they felt necessary to remain current with their CPD (Appendix, Supplementary Table 1.2 a–d). The most important reasons given were the lack of time and the inability to efficiently find the right material out of all the information available (Appendix, Supplementary Table 1.2 e). Further questions focussed specifically on cardiovascular and respiratory topics, which were the themes of the first MNSHA module, as we will discuss below. Participants were asked which topics, knowledge, or skills they felt they needed to gain more confidence in their daily practice. The five most commonly given answers are listed in Table 1 (see also Appendix 1, Supplementary Table 1.3). Learning methods
Several innovative learning methods have been described in the literature recently [3–13]. We selected learning and educational methods that we felt could work effectively in a CPD programme for EPs within the Dutch context. An overview of these methods is given in Table 2. The combined use of these educational methods, together with a strong mentoring programme, has not been previously described for CPD of EPs. Course description Course development
MNSHA has been developed by a small group of EPs from the Netherlands and Australia to aid in the Table 1 Topics, knowledge, or skills participants think they need to gain more confidence in their daily practice Cardiovascular EM Arrhythmias ECG interpretation Medication use Heart failure Acute coronary syndromes
Respiratory EM 44% 22% 18% 14% 14%
Antibiotics and other medication Chest drain insertion Asthma and COPD Pulmonary function tests Chest radiograph interpretation
22% 15% 12% 12% 6%
COPD, chronic obstructive pulmonary disease; ECG, electrocardiogram; EM, emergency medicine.
professional development of Dutch EPs. It is targeted at EPs who have completed the 3-year Dutch EM training programme and are currently practicing in Emergency Departments throughout the Netherlands. The MNSHA programme is primarily aimed at providing their knowledge and skills necessary for improved, consultant-level decision-making. EPs should develop an effective and sustainable way to gain, maintain and develop knowledge and skills as part of their ongoing professional education. MNSHA has been designed as a modular programme, with the first two modules developed to date. Cardiovascular and respiratory emergencies were selected as the themes for the first module, as these are core areas to which all Dutch EPs are regularly exposed. A second module, which covers a variety of acute medical emergencies, was launched in January 2014. It is envisaged that several more modules will be developed, with topics covering the depth and breadth of EM. Within the time constraints of a relatively short programme, it is impossible to cover all topics in a given area of EM. Therefore, a conscious decision was made to address a limited number of crucial topics in depth, so that the participants can master these at a consultant level. For this reason, we describe our course as a ‘masterclass’. Topics have been carefully selected for their richness of content, layers of evidence, and areas of controversy. Approximately a quarter of all cases are paediatric, and the development of a specific paediatric module is being considered. Upon completion of the masterclass, participants should be able to utilize the skills that they have practiced, to continue their professional development by addressing topics that have not been encountered during the masterclass. Each MNSHA masterclass spans a total of 15 weeks, of which most time is spent in a self-directed study with regular mentor contact. There are two face-to-face meetings with all participants, and the masterclass culminates in three more face-to-face days, described below. A study guide was written, in which clinical cases are used to pose relevant questions to the participants. A limited number of learning goals are provided, along with a rough timeline to facilitate participants’ progress through the material and to ensure that all participants have covered the same topics at the face-to-face days. A list of suggested resources is also provided. Through selfdirected study with guidance from mentors, participants must develop techniques that allow them to access and use the wealth of information available in a timeappropriate manner. Participants should define gaps in their knowledge, fill these gaps with up-to-date information, strengthen their ability to critically appraise evidence and guidelines, and establish reasoned opinions on current controversies in EM. This acquisition of content is learner driven, with no prescribed target level. The further refinement and integration of knowledge and the
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
210 European Journal of Emergency Medicine 2016, Vol 23 No 3
Table 2
Learning and educational methods
Learning method
Explanation
Asynchronous learning
Distance learning, Internet learning, FOAM
Self-directed learning
Flipped classroom
Mentoring
Problem-based learning
An educational method that utilizes individualized learning resources, to enable student-centred teaching to take place outside the limitations of time and place. Asynchronous learning gives learners the opportunity to use their preferred learning style, and targets an individual’s knowledge gaps [3]. It is also referred to as individual interactive instruction. This technique is being increasingly incorporated into education schemes of medical students and junior doctors, alongside the more traditional synchronous didactic methods [4,5] Distance learning provides access to learning when the source of information and the learners are separated by time, distance, or both. Internet learning is an example of modern distance learning [6]. One of the newer developments in Internet medical education is Free Open-Access Medical Education (FOAM). It comprises an exploding collection of constantly evolving, collaborative, and interactive open-access medical education resources that are being freely distributed over the Internet. FOAM includes blogs, podcasts, online videos, text documents, and photographs [7] In self-directed learning, learners are responsible for their own learning process. They are self-managing (context, social setting, resources, and actions) and self-monitoring (evaluating and regulating their cognitive learning strategies). Teachers model learning strategies and work with students so that they develop the ability to use them on their own. Selfdirected learning is asynchronous [8] The flipped classroom is an educational model in which traditional lecture material is studied before attending a teaching session. Teaching time is then free to be used for active learning and the application of what has been learned through thought-provoking exercises [4,9] and the development of meta-cognition (e.g. critical thinking, problem-solving [10]) A mentor is a trusted and experienced person who has a direct interest in the professional development and education of another person, and provides guidance and support [11,12]. The mentor may have a number of roles including advisor, supporter, tutor, and role model. Individual mentoring, group mentoring, and telementoring (mentoring through the Internet) are different forms of mentoring [11] A learning method that aims at efficient acquisition and structuring of knowledge, arising out of working in active, iterative, and self-directed ways. It comprises a progressive framework of problems providing context, relevance, and motivation (problem-first learning), and builds on prior knowledge integration, critical thinking, reflection on learning, and enjoyment. It achieves its goals through facilitated small-group work and independent study [13]
development of metacognition are aided by the mentoring process. The use of an online forum, open to participants and mentors, is encouraged for the sharing of resources, ideas and questions. It is hoped that this will ultimately translate to more sound, evidence-based clinical reasoning and improved patient care in daily practice. An overview of the MNSHA programme is accessible through the MNSHA website (http://www.mnsha.nl/en). Mentoring
Each participant is assigned a mentor with whom they have regular contact throughout the module. All mentors are EM specialists with experience in education. They are mostly based overseas (predominantly in Australia and the UK), as the required level of expertise is not currently widely available in the Netherlands. Mentor sessions are primarily web-based to allow live conversation. The two major aims for mentors are to function as a role model (i.e. an EP functioning at a level participants can aspire to achieve) and to help participants integrate information to enhance insight into the subject matter, enabling them to achieve the desired learning outcomes. Mentor sessions are intended to be learner driven, with the mentor providing personalized guidance, stimulating critical thinking and problem-solving, and tailoring discussions to the participants’ individual needs. Face-to-face meetings
At the start of the masterclass, an initial meeting allows participants to meet each other and some of the mentors. It introduces the format and learning objectives of the course, and gives suggestions on how to approach the vast
array of medical resources available. Aims and expectations are discussed, and the desired level of knowledge and skills is clarified. A second face-to-face meeting with all the participants is held approximately a third of the way through the masterclass. Participants are asked to complete formative multiple-choice questions before this session, which are related to the topics studied to that date. The multiplechoice questions, several cases and visual material are utilized to stimulate discussion and challenge participants in a small group setting. Participants are able to reflect upon their learning to that date and their knowledge is challenged. The masterclass is concluded with three consecutive face-to-face days, which allow participants to discuss the topics studied and develop a meta-understanding of the subject matter [14]. Case-based, clinically relevant discussions are facilitated by faculty, in small and larger groups. Other educational methods used include a journal club, pro–con debate and workshops. Participants are encouraged to develop strategies for dealing with controversies in complex clinical situations, as well as a personal strategy for their CPD. Although there is no defined target level of knowledge, mentors and facilitators demonstrate by example what level the participants should be aspiring to achieve. The emphasis is on thinking processes, the general approach to a problem, and ‘consultant-level’ decision-making. However, a certain level of content acquisition is needed as well. To assist with their understanding of this level, participants complete a formative assessment at the end
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 211
of the masterclass days. This assessment consists of objective structured clinical examination questions in a formal examination setting.
Fig. 2
9 8 7
Figures 1–3 show the improvements in self-ranking on knowledge, the ability to treat patients and the ability to discuss assessment and management controversies and cases with other specialists directly after the masterclass compared with before the masterclass (P-value ranges from < 0.001 to 0.59) and 6 or 10 months after the masterclass compared with before the masterclass (P-value ranges from < 0.001 to 0.44).
F O PD
C
C
C
th m a Sy nc pi op ra Ta tor e y ch ya arre st Re rrh yt sp h m ira ia to s ry Pn fa eu ilur Ao mo e th rti or c ax di ss ec tio n
PE
As
ar di or es C
Overall, participants gave the first MNSHA masterclass an average score of 8.9 on a 10-point scale. Before the masterclass (Figs 1–3, ‘pre’), none of the participants felt that they were sufficiently up-to-date, whereas immediately after the masterclass (‘post 1’) and after 10/6 months (‘post 2’), 100 and 82%, respectively, felt that they were up-todate. A significant improvement is also seen in the selfranking of their ability to find and efficiently use resources (from 5.1 to 8.1, and 7.5, respectively, out of 10, P < 0.001).
S
From September to December 2012, 21 participants took part in the first edition, and from March to June 2013 another 16 participants took part in the second edition. All 37 participants who started the masterclass also finished the masterclass. Twenty-two participants completed the premasterclass survey (59%), 33 responded to the first postmasterclass survey (89%) and 18 responded to the second postmasterclass survey (48%).
6 5 4 3 2 1 0 AC
Participant survey
Pre
Post 1
Post 2
How comfortable do you feel treating a patient with... Significant: P-value < 0.05. ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Fig. 3
9 8 7 6 5 4 3 2
As
8 7
th m a es Sy nc pi ra o Ta tory pe ch ar ya re st Re rrh yt sp h m ira ia to s ry Pn fa eu ilur Ao mo e th rti or c ax di ss ec tio n C C F C O PD
AC
S
9
PE
1 0
Fig. 1
C
ar
di
or
6 5 4 3 2 1
Pre
PD
C F C
O C
P As E th di m or es Syn a pi co ra Ta tor pe y ch ar y r Re arrh est yt sp ira hm ia to s ry fa Pn eu ilur Ao mo e th rti or c ax di ss ec tio n C ar
AC
S
0
Pre
Post 1
Post 2
Please rate yourself as to what you feel your current knowledge is of... Significant: P-value < 0.05. ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Post 1
Post 2
How comfortable do you feel in a discussion with a Respiratory Physician or Cardiologist on a patient with a working diagnosis of... Significant: P-value < 0.05; ΔNot significant: P-value > 0.05. ACS, acute coronary syndrome; CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; PE, pulmonary embolism.
Discussion and conclusion Several innovative educational methods were embraced by the MNSHA programme to deliver the course. An asynchronous approach to learning in a flipped classroom was utilized. The more traditional concept of mentoring was combined with these newer techniques in a modern, web-based environment.
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
212 European Journal of Emergency Medicine 2016, Vol 23 No 3
Needs analysis
From our brief survey, we concluded that a sizeable proportion of Dutch EPs do indeed perceive deficiencies in knowledge and skills in some of the most important topics within the specialty. We found that neither are the needs of EPs met fully during the 3-year training programme, nor are EPs currently able to effectively address their needs as part of their CPD. A discussion on the length of the Dutch EM training programme and perhaps improving the way it is delivered is outside the scope of this article. Our survey supported the prior perception that there is a need for better methods of delivering CPD to Dutch EPs. The majority of the topics that were most frequently requested were incorporated into the first module of the MNSHA programme. Asynchronous learning
Asynchronous learning is student-centred teaching, which targets the needs of the learner more specifically than traditional (synchronous) educational methods [3]. The elements of asynchronous learning incorporated into the MNSHA programme have intuitive advantages. EPs usually work shifts and are frequently unable to attend scheduled educational sessions. MNSHA participants can choose when they will study, within a rough guideline, so that over a period of time all participants arrive at the same learning outcomes. The participants’ diverse educational backgrounds are accommodated by asynchronous education, as they are free to access a vast array of educational material and can select what is appropriate to build on their current knowledge of a particular topic. To date, the majority of the literature on the use of asynchronous techniques pertains to education of medical students and junior medical doctors, not medical specialists. The literature on the use of asynchronous learning in EM is scarce [3,15]. A major disadvantage of asynchronous learning is that it relies on the learners’ abilities to identify their own knowledge gaps. When they are unable to do this effectively, the learners do not realize where their deficiencies lie and may therefore not study those areas (unconscious incompetence). Another disadvantage is that there is no immediate opportunity to discuss a topic with a group or ask questions, and there is no immediate feedback. These disadvantages are counterbalanced in our programme by the use of mentors, study groups and an online forum. Flipped classroom
Although there is little peer-reviewed literature supporting the benefits of the flipped classroom approach [4,9], this new approach to education offers several intuitive advantages. Participants have time to reflect on their study material and have the opportunity to rewind and repeat online lectures. Participants are able to control
when and where they access the educational material, and the education is learner driven as the responsibility for preclass preparation lies primarily with the participant [8]. Mentoring
Mentoring has been shown to be crucial to career development and success [1,12,16,17]. Effective mentoring in medicine leads to greater career satisfaction [17], and physicians with mentors have more confidence in their abilities [18]. Mentoring in EM is a poorly represented topic in the literature [11], especially with regard to mentoring of EPs themselves, despite the fact that mentoring is seen to play a critical role in all career stages [12]. There is as yet no published research assessing whether mentoring of physicians is correlated with improved patient outcomes. Participants survey
Our survey intended to evaluate whether the masterclass improved the knowledge and skills of the participants. It showed an improvement directly after the masterclass in the subjective belief of being up-to-date, in self-ranking of the ability to find and efficiently use resources, and in the confidence with every subject that was discussed in the masterclass. After 6 or 10 months, some of these selfrankings decreased, although they remained above premasterclass levels. These results are encouraging, but subject to several limitations. The main limitation of this survey is that it only looked at subjective improvements as rated by the participants. It was not practically feasible to perform a more objective testing of knowledge before and after the completion of the programme. Designing a study to assess for improvements in patient safety or clinical outcomes was also not feasible; hence, we instead chose to investigate subjective improvements in knowledge and confidence. A further limitation of this survey was the small number of participants, not all of whom returned all three surveys. Some bias may also have been caused by the fact that two groups where surveyed with different follow-up times (6 vs. 10 months after the masterclass). Conclusion
MNSHA combines innovative educational methods to deliver CPD to Dutch EPs. This combination of methods has not been previously described for a medical specialist CPD programme. The educational methods used by MNSHA seem intuitively advantageous to physicians, especially those in EM. They take previous knowledge and experience into consideration, and also allow physicians to access the educational material where and when they like. This is probably more suitable for EPs than traditional, synchronous educational strategies. Appropriate role models with specific knowledge and experience in EM are not yet widely available within the
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
Modular continuing professional development for EPs van den Berg et al. 213
Netherlands. A major component to the success of this course is the involvement of EPs from other, more mature EM systems as mentors to guide Dutch EPs through their learning process. Mentoring enables better application and retention of knowledge, as well as the development of consultant-level thinking. Our survey of participants showed some encouraging results, strongly suggesting an improvement in confidence. A more robust study would be required to better assess the outcomes of our programme. The MNSHA programme was developed specifically for EPs in the Netherlands. However, the authors believe that the combination of methods used could find wider application in medical education in other countries.
3
4 5
6
7 8 9
10
Acknowledgements A.P.vD, F.E.C.G. and S.M. have developed and are currently running the MNSHA programme, which this article describes. Conflicts of interest
11 12 13
14
There are no conflicts of interest.
15
References 1 EuSEM Taskforce on Curriculum. European curriculum for emergency medicine. 2009:25–4. Available at: http://www.eusem.org/cms/assets/1/ pdf/european_curriculum_for_em-aug09-djw.pdf. [Accessed 20 October 2013]. 2 Koning SW, Gaakeer MI, Veugelers R. Three-year emergency medicine training program in The Netherlands: first evaluation from the residents’ perspective. Int J Emerg Med 2013; 6:30.
16
17 18
Reiter DA, Lakoff DJ, Trueger NS, Shah KH. Individual interactive instruction: an innovative enhancement to resident education. Ann Emerg Med 2013; 61:110–113. Prober CG, Heath C. Lecture halls without lectures – a proposal for medical education. N Engl J Med 2012; 366:1657–1659. Sadosty AT, Goyal DG, Gene Hern H Jr, et al. Alternatives to the conference status quo: summary recommendations from the 2008 CORD Academic Assembly Conference Alternatives Workgroup. Acad Emerg Med 2009; 16 (Suppl 2):S25–S31. US Department of Education. Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. 2010. Available at: http://www2.ed.gov/rschstat/eval/tech/evidence-basedpractices/finalreport.pdf. [Accessed 15 October 2013]. Nickson CP, Cadogan MD. Free Open Access Medical education (FOAM) for the emergency physician. Emerg Med Australas 2014; 26:76–83. Abdullah MH. Self-directed learning. Available at: http://www.education.com/ reference/article/Ref_Self_Directed/mp. [Accessed 15 October 2013]. Educause Learning Initiative. 7 things you should know about flipped classrooms. 2012. Available at: http://www.educause.edu/library/resources. [Accessed 09 October 2013]. Martinez ME. What is metacognition? Phi Delta Kappan 2006; 87.9:696–699. Yeung M, Nuth J, Stiell IG. Mentoring in emergency medicine: the art and the evidence. CJEM 2010; 12:143–149. Garmel GM. Mentoring medical students in academic emergency medicine. Acad Emerg Med 2004; 11:1351–1357. Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract 2009; 18:1–9. Metcalfe J, Shimamura AP. Metacognition: knowing about knowing. Cambridge, MA: MIT Press; 1994. Burnette K, Ramundo M, Stevenson M, Beeson MS. Evaluation of a webbased asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad Emerg Med 2009; 16 Suppl 2: S46–S50. Stamm M, Buddeberg-Fischer B. The impact of mentoring during postgraduate training on doctors’ career success. Med Educ 2011; 45:488–496. Farrell SE, Digioia NM, Broderick KB, Coates WC. Mentoring for clinicianeducators. Acad Emerg Med 2004; 11:1346–1350. Ramanan RA, Phillips RS, Davis RB, Silen W, Reede JY. Mentoring in medicine: keys to satisfaction. Am J Med 2002; 112:336–341.
Copyright r 2016 Wolters Kluwer Health, Inc. All rights reserved.
