Research
Original Investigation
Taking a Fresh Look at the Skull Base in Otorhinolaryngology With Web-Based Simulation Student’s Interactive Skull-Base Trainer (SISTer) Peter Freiherr von Sass; Kathrin Scheckenbach, MD; Martin Wagenmann, MD; Thomas Klenzner, MD; Joerg Schipper, MD; Adam Chaker, MD
IMPORTANCE The increasing amount of medical knowledge and necessity for time-effective teaching and learning have given rise to emerging online, or e-learning, applications. The base of the skull is a challenging anatomic area in the otorhinolaryngology (ORL) department—for both students and lecturers. Technology-enhanced learning might be an expedient approach to benefit both learners and lecturers. OBJECTIVE To investigate and create for advanced medical students a self-assessed adaptive e-learning application for the skull base within our curriculum of otolaryngology at the University Medical Center of Heinrich Heine University, Düsseldorf, Germany. DESIGN, SETTING, AND PARTICIPANTS Pilot approach with prospective evaluation of a newly implemented web-based e-learning simulation. The e-learning application (Student’s Interactive Skull-Base Trainer) was made accessible as an elective course to a total of 269 enrolled medical students during the first 2 semesters after web launch. INTERVENTIONS Spatiotemporal independent e-learning application for the skull base. MAIN OUTCOMES AND MEASURES Self-assessed evaluation with focus on general acceptance and personal value as well as usage data analysis. RESULTS The application was well accepted by the learners. More than 80% of the participating students found the application to be a beneficial tool for enhancing their analytical and clinical problem-solving skills. Although the general matter of the skull base seemed to be of lesser interest, the concept of anchored instructions with the use of high-end, interactive, multimedia-based content was considered to be particularly suitable for this challenging topic. Most of the students would have appreciated an extension of optional e-learning modules. CONCLUSIONS AND RELEVANCE With this pilot approach we were able to implement a useful and now well-accepted tool for blended learning. We showed that it is possible to raise interest even in this very specialized subspecialty of ORL with overall individual learning benefit for the students. There is a demand for more e-learning and web-based simulation to support the existing curricula in a hybrid, blended way. Author Affiliations: Department of Otolaryngology, Universitätsklinikum Heinrich Heine Universität, Düsseldorf, Germany (von Sass, Scheckenbach, Wagenmann, Klenzner, Schipper); Department of Otolaryngology, Klinkum rechts der Isar, Technische Universität, München, Germany (Chaker).
JAMA Otolaryngol Head Neck Surg. 2015;141(2):154-159. doi:10.1001/jamaoto.2014.3041 Published online December 4, 2014. 154
Corresponding Author: Peter Freiherr von Sass, Hals-Nasen-OhrenKlinik des Universitätsklinikum Düsseldorf, Moorenstrasse 5, 40225 Düsseldorf, Germany (Peter.vonSass @med.uni-duesseldorf.de). (Reprinted) jamaotolaryngology.com
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
Student’s Interactive Skull-Base Trainer
Original Investigation Research
C
urricula in academic medicine and clinical teaching are currently undergoing substantial changes. Along with an exponential increase of medical expert knowledge, there are ongoing efforts to restructure curricula from an educational and didactic perspective.1-3 Educational goals, funding constraints, and teaching repertoire at hand in clinical teaching today have to be taken into consideration just as well as different individual learning types, previous knowledge, and personal interest. The aim is to enable medical students to develop (1) their diagnostic and analytical skills based on accurate anatomical and physiological knowledge, (2) clinical competencies, and (3) a solid foundation for mature clinical judgment. Widespread availability, use, and acceptance of web-based applications and the rise of technology-enhanced learning offer emerging blended- and online, or e-learning, opportunities.4,5 Otorhinolaryngology (ORL) is a closely intertwined clinical discipline based on the complex anatomy and physiologic characteristics of the head and neck, including the base of skull, with usually little representation in medical curricula and national medical license examinations. Often, students will achieve only basic-level cognitive learning objectives. The idea behind the Student’s interactive Skull-Base Trainer (SISTer) is indeed simple: creating a motivating hybrid learning environment that combines well-established lectures and seminar teaching methods with the emerging opportunities of web-based, individual, self-directed learning in the anatomically challenging area of the skull base. We hypothesized that the use of multidimensional and interactive display of anatomical structures embedded in clinical case histories from the university hospital is well suited for the complex base of skull and boosts self-assessed ORL learning outcome. The project was added to the regular curriculum, which also contains bedside teaching, lectures, seminars, tutorials, and study-guided learning. The aim of our project was to evaluate a didactic pilot elearning approach on this specialized and narrow subject to study the general acceptance and self-assessed learning outcome. Furthermore, we aimed to begin a dialogue on blended learning and web-based simulation with students to enhance quality of learning in the ORL curriculum, and to further implement—if positively evaluated—a blended learning concept.
Table 1. Overview of the 3 Case Studies Subject of Case Study
Screen Cards, No.
Exercises, No.
Inverted papilloma
18
11
Pleomorphic adenoma
17
5
Vestibular schwannoma
14
8
as well as the evaluation form were voluntary and anonymous. No compensation for participation was granted.
Learning Objectives The first objective was to develop analytical clinical skills on the skull base regarding recognition of abnormalities based on patient history and clinical findings and to further integrate these diagnostic findings into a more elaborate clinical diagnosis. Second, to encourage integration into interdisciplinary clinical pathways, students were guided through advanced therapeutic concepts offered as expert discussion within the application while the learning outcome was examined by the means of multiple-choice assessments in a selfdirected manner.
Didactic Concept We chose the didactic concept of anchored instructions to linearly guide the students throughout the whole process of diagnosis, treatment, and follow-up.5
Content Three interactive clinical cases on a total of 49 web-based screen cards were developed to depict clinical presentation, diagnostic procedures, and multimodal therapy, including follow-up care, based on anonymized patients from our university hospital. The cases present regular ORL diseases that further develop to complex interdisciplinary skull-base cases (Table 1). Students were able to register and access SISTer through a university link for http://www.Casus.net, a widely used e-learning platform in German universities. The availability of the new module was announced in the main ORL lecture on university and student union websites. The program was explicitly announced as not mandatory; however, students were informed that the content might be relevant for 10% of the ORL examination.
Self-Directed Learning Performance Assessments
Methods Prospective Pilot Study We performed a prospective, anonymous pilot study approach for a new web-based e-learning and simulation module in our ORL curriculum. Evaluation forms as well as electronically acquired user data were analyzed. The study was funded by the e-learning initiative HeinEcomp. The Faculty of Medicine of Heinrich Heine University, Düsseldorf, Germany, served as an equivalent to an institutional review board because the analysis concerned students (and not patients of the medical center) and authorized prospective evaluation and analysis. Participating students did not provide written informed consent as both the e-learning course jamaotolaryngology.com
Twenty-four of 49 screen cards contain exercises in which students are able to fill in their diagnostic interpretations as free text using an automated answer-recognition tool, multiplechoice, or multiple-selection questions, plus many others. After answering screen card questions, students receive immediate feedback comparing their answers with the most correct answer in text form as well as a percentage of correct answers and an author’s comment. At the end of each case study, students receive a summary of feedback on educational objectives as well as their personal performance statistics. For those students wanting to get even more deeply involved, additional expert commentaries are available to clarify more details of the disease or to explain further clinical course of actions and processes.
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
155
Research Original Investigation
Student’s Interactive Skull-Base Trainer
Figure 1. Examples of Screen Cards
Table 2. Demography and User Statistics Semester
A
Characteristic
1
2
135
134
Male
60
53
Female
75
81
104
109
Enrolled students, No.
Students logged in, No. Fully finished Received evaluation forms
89
100
48
None
ation data were irreversibly lost owing to technical issues. To date, we have partially adapted and included various changes in composition and content of the case studies fitting into a larger e-learning strategy, thus averting comparability.
Finalizing the e-Learning Module As seen in Table 2, more than 90% of the students who logged onto the e-learning platform finished the program completely, which we defined as visiting a minimum of 45 of the screen cards (92%). To do so, users had to answer virtually every interactive question in the program. We further logged the percentage of correct answers and the time spent on each card. The range of correct answers varied from 33% to 64%, with students spending on average 43 to 70 minutes on each case, which implies 2.8 to 3.9 minutes per card.
B
Providing Incentive for User Demand
Screenshot of 2 typical screen cards. A, Dichotomous composition of text and multimedia content. B, Multiple-choice question.
Layout and Navigation The dichotomous screen card layout enables the presentation of text on the left side and animated, interactive, or static multimedia content on the right side. We included anatomic images, sequences of clinical examinations, short videocasts of surgical and endoscopic procedures with commentary by experienced ORL surgeons, and pictures of histologic or complementary diagnostic findings (eg, hearing tests, magnetic resonance imaging, angiography, and computed tomographic sequences) (Figure 1). On many of these interactive screens, students are able to navigate through different imaging layers and perspectives, with anatomical text labels fading in or out on demand. Before clicking to the next screen, students have to answer questions that keep a record of their learning progress.
Results Completion of SISTer After Web Launch During the first 2 semesters following web launch, almost threequarters of all enrolled students (189 of 269) completed all 3 cases. More than 50% of the first-semester cohort submitted the evaluation form (48 of 89). In the subsequent semesters, we received only data on usage characteristics, while some electronic evalu156
As part of the evaluation, we further investigated the motivation for spending time and effort in the e-learning program. Ninety-six percent of students indicated that relevance for the final examination as one point of motivation, whereas curiosity (51%) and the expected advantage of e-learning (49%) ranged lower. A specific interest in ORL as a subject (29%) or interest in interdisciplinary problems (29%) both received fairly low priority (Figure 2). As part of the educational goals, we also investigated the impact on analytical and clinical problem-solving skills by evaluating self-assessments by the online participants. Most of the students (>80%) found SISTer a beneficial tool for enhancing those skills. About the same number of students (80%) indicated that the web-based 3-dimensional depictions helped them to relate the complex anatomy to the clinical case (Figure 3).
Good Acceptance of e-Learning on the Skull Base Electronic evaluation showed that general acceptance of our optional e-learning course was very good. Concerns that additional content could lead to an overcrowding of the tightly packed fourth-year curriculum were refuted by the students (86%). In fact, about half of the students (54%) indicated a desire for more in-class lectures. Moreover, even 82% of the students would have liked an extension of optional e-learning modules for self-directed hybrid learning. The skull base per se seems to be of lesser interest. Few students (24%) would welcome an elective in-class course on the subject (Figure 4). Interestingly, most students (73%) found the base of skull particularly well suited for e-learning.
JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
jamaotolaryngology.com
Student’s Interactive Skull-Base Trainer
Original Investigation Research
Enabling Students to Identify Learning Deficits by e-Learning We assessed whether the students found that SISTer blended well into the existing curriculum. The response from almost 80% was positive (Figure 5); the same percentage of students further stated that working with this e-learning module enhanced their understanding of previously unknown content. Even more (90%) (Figure 5) conceded that their personal knowledge deficiencies were revealed by our self-directed elearning application.
standing of context formerly unknown to the students, thus enabling better quality of teaching and more elaborate cognitive learning. Figure 2. Motivation of Students to Work With SISTer 96%
100
Discussion We saw broad acceptance for our web-based e-learning module, indicated by user characteristics and prospective standardized evaluation even though SISTer addresses a very specific field. In particular, we think that spatiotemporal independence and self-directed learning (as described by Ruiz et al6) in our web-based concept, together with cross-linking of complex anatomy and initially basic ORL cases that developed to specific complex entities, helped to process information into meaningful chunks. The motivation to acquire knowledge and make sense out of the presented material is an important factor and is also referred to as generative cognitive processing.7 We suggest that this enhanced the under-
Answers, %
75
51% 50 29%
49%
29%
25
0
General Interdisciplinary Relevance for Advantages of Interest in Problem Examination e-Learning ORL Question: Why did you work with these case studies?
Curiosity
Results of a multiple-selection question in percentages of 48 users (multiple answers possible). “Relevance for examination” was stated as one point of motivation to work with the e-learning application by almost all of the users, followed by “curiosity” and “advantages of e-learning.” ORL indicates otorhinolaryngology.
Figure 3. Self-assessment of Analytical and Clinical Problem-Solving Skills for 48 Users Fully applicable
Applicable
Less applicable
Not applicable
Interactive assessment of medical findings online enhanced my diagnostic skills Interactive demonstration of anatomic structures enhanced my diagnostic skills Web-based 3-dimensional depictions helped to relate clinical context and anatomy better 0
20
40
60
80
100
Results, %
Results of evaluation in percentages for analytical and problem-solving skills by means of multiple-choice questions. More than 80% of the learners rated the program as beneficial.
Figure 4. Acceptance of e-Learning and Specific Interest in the Subject for 48 Users Fully applicable
Applicable
Less applicable
Not applicable
Additional e-learning makes curricula to become even more busy I would appreciate more e-learning content to choose from and combine with the curriculum I wish for more in-class lectures
I wish for more e-learning Topic of skull base is in particular well suited for e-learning I wish for an elective course on base of skull 0
20
40
60
80
Answers, %
jamaotolaryngology.com
100
Answers of multiple-choice questions. There was a request for more e-learning, and the skull base was noted as particularly suitable as a subject by about three-quarters of the students.
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
157
Research Original Investigation
Student’s Interactive Skull-Base Trainer
Figure 5. Assessment of Individual Cross-Linking of Learning Fully applicable
Applicable
Less applicable
Not applicable
Working with the case studies complemented the in-class lectures well Working with the case studies enhanced my understanding of contexts formerly unknown Working with the case studies pointed out my shortcomings on the subject 0
20
40
60
80
100
Answers, %
Considering the very broad acceptance found in our investigation, we can state that we were able to create the necessary motivation in a narrow and specialized topic (with limited importance to most learners’ future professional development) by offering a novel and appealing resource for self-directed learning with adequate cognitive interactivity.
Relevance for Examinations as a Trigger for Acceptance As expected, relevance for the final examination was a trigger; however, students described the program as beneficial for their clinical problem-solving skills. Though the multimedia-based approach has been a controversial subject in the literature,8-10 we believe it is particularly accountable for this positive result; the 3-dimensional depictions were explicitly described as being helpful for linking anatomy and clinical context. Individual performance monitoring would have been feasible, but we explicitly designated the course as optional, offering only the incentive that 10% of the final examination might contain content of the e-learning application. In fact, with this 10% advantage for the final examination, we have found it difficult to explain the high interest of the students. Long-term usage data will be necessary to determine whether this was biased by the novelty factor of the application or mere coincidence of more highly motivated students. In a pilot project study of blended learning in Vienna, Austria, students did not benefit from e-learning.11 Students in the control group were more satisfied with traditional teaching methods than students in the e-learning group. The authors stated that medical students might be conservative and therefore might rely more on traditional teaching methods like lectures than on novel approaches.11 Although the design of our investigation did not allow comparison between different groups, we clearly interpret our results as an indication that optional, additional, and blended e-learning applications and simulations can be very well accepted. While only about 50% of participating students in our study would have liked more in-class lectures, the number wishing for more e-learning content exceeded 80% in our evaluation, therefore refuting the stereotype of overly conservative medical students. This positive response on e-learning is consistent with findings of many other investigations (eg, Khogali et al 4 showed that elearning on cardiovascular system was rated as valuable by 96% of the responding students). 158
Forty-eight users were asked multiple-choice questions on integration of the e-learning application into the existing curriculum. About 80% stated that the online contents complemented the lecture well and facilitated comprehension of the formerly unknown contents.
Evaluation by Students of the Benefit of SISTer In particular, the self-assessed, adaptive, and case-based approach and multidimensional imaging were considered to be useful for this challenging and difficult topic. The skull base, with its highly complex pathologic abnormalities, is hardly covered in medical curricula. The concept of anchored instructions with the use of high-end, interactive, multimedia-based content was evaluated as being particularly suited for this challenging topic. Students were able to acquire a deeper understanding of complex clinical presentation and anatomy in the ORL department and base of skull in an entertaining, multimedia-based, handson, online approach. Although not of personal interest for most, the blended approach and highly refurbished content activated the learners to meaningful learning as far as could be determined with our self-assessment questionnaire.
Study Limitations There is potential bias because of the pilot character of our study. Long-term user data are required, but owing to ongoing changes in our faculty curriculum are currently unavailable. Additional possible bias may be found in the selection of participants. We explicitly chose an elective course design. We kept usage and performance anonymous so as to receive an unaffected feedback from the students. It is therefore very possible that more ambitious students completed the application. This would be compatible with the observation that learners with better previous knowledge benefit more from multimedia-based e-learning.12 This view can, however, be challenged by the rather low percentage of correct answers on the screen cards. Just as easily, the poor result could be attributable to the new and difficult subject. Ninety percent of the students stated that the program identified gaps in their personal knowledge. Regarding the overall positive evaluation, we interpret this as motivating self-criticism rather than an indicator of a disproportionate requirement.
SISTer as a Helpful Pilot Project We electronically measured the time that users spent on each case study. The time spent averaged about 1 hour per case, totaling 3 hours. Even experienced scholars might find it difficult to offer a comprehensive overview about the challenging anatomy, diagnostic pitfalls, and multimodal, interdisciplinary approaches to medical students and newcomers when using standard educational methods such as lectures or seminars. Apart from the fact that the subject was barely covered,
JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
jamaotolaryngology.com
Student’s Interactive Skull-Base Trainer
Original Investigation Research
so far we were able to blend it into an existing curriculum in order to enhance basic ORL skills as well as specific expert knowledge with a reasonable amount of departmental effort. Depicting the same material in an in-class lecture would presumably take longer and lack the advantages of individually adapted and spatiotemporal independent learning. We presume that an in-class lecture would have lowered the general acceptance level obtainable from the rather restrained answers on the question about an elective skull base course.
ARTICLE INFORMATION Submitted for Publication: May 28, 2014; final revision received October 1, 2014; accepted October 16, 2014. Published Online: December 4, 2014. doi:10.1001/jamaoto.2014.3041. Author Contributions: Mr von Sass had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: All authors. Acquisition, analysis, or interpretation of data: von Sass, Chaker. Drafting of the manuscript: von Sass, Chaker. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: von Sass, Chaker. Obtained funding: Schipper, Chaker. Administrative, technical, or material support: von Sass, Scheckenbach, Klenzner, Wagenmann, Chaker. Study supervision: von Sass, Klenzner, Schipper, Chaker. Conflict of Interest Disclosures: Dr Wagenmann has given lectures sponsored by Allergopharma, ALK-Abello, Bencard, HAL Allergy, and Meda and has received grant support or consultant arrangements with Allergopharma, ALK-Abello, Bionorica, and HAL Allergy. Dr Chaker has given lectures sponsored by GSK and has received grant support or consultant arrangements with Allergopharma, ALK-Abello, Stallergenes, HAL Allergy, Mundipharma, Zeller AG, and Novartis Pharma AG. No other disclosures are reported. Funding/Support: Student’s interactive skull-base Trainer (SISTer) was funded by the e-learning initiative HeinEcomp of the medical faculty of Heinrich Heine University, Düsseldorf.
jamaotolaryngology.com
Conclusions The e-learning tool SISTer uses the resource e-learning in a didactic appropriate manner and complements hitherto existing classroom teaching. We think that it offers an additional value for medical students who learn cutting-edge diagnostic techniques and interdisciplinary clinical pathways in academic medicine.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Additional Contributions: Christian Babel, Thomas Baehring, and Beate Meinert (HeinEcomp), all of Medical Faculty of Heinrich Heine University, provided technical assistance. Bernd Turowski, MD, Department of Neuroradiology, and Daniel Hänggi, Department of Neurosurgery, Heinrich Heine University, provided support when setting up SISTer. Michael Hemming, Munich, Germany, critically reviewed the manuscript. Alexandra Hesse, and Pascal Berberat, MD, Medical Faculty of Technische Universität München, helped with lost literature and provided helpful discussions, respectively. No compensation was granted for assistance. REFERENCES 1. Schnabel KP, Boldt PD, Breuer G, et al. A consensus statement on practical skills in medical school: a position paper by the GMA Committee on Practical Skills. GMS Z Med Ausbild. 2011;28(4):Doc58. 2. Fegert JM, Obertacke U, Resch F, Hilzenbecher M. Die Qualität der Lehre nicht dem Zufall überlassen. Dtsch Arztebl. 2009;106(7):A290-A291. 3. Stausberg J, Kühnl A. E-learning in der medizin: Erweitertes Angebot. Dtsch Arztebl. 2013;110(37): A1692.
medical training: great opportunities, pitfalls and challenges. Med Teach. 2011;33(4):291-296. 6. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81(3):207-212. 7. Mayer RE. Applying the science of learning to medical education. Med Educ. 2010;44(6):543-549. 8. Levinson AJ, Weaver B, Garside S, McGinn H, Norman GR. Virtual reality and brain anatomy: a randomised trial of e-learning instructional designs. Med Educ. 2007;41(5):495-501. 9. Pereira JA, Pleguezuelos E, Merí A, Molina-Ros A, Molina-Tomás MC, Masdeu C. Effectiveness of using blended learning strategies for teaching and learning human anatomy. Med Educ. 2007;41(2): 189-195. 10. Garg AX, Norman GR, Eva KW, Spero L, Sharan S. Is there any real virtue of virtual reality? the minor role of multiple orientations in learning anatomy from computers. Acad Med. 2002;77(10)(suppl):S97-S99. 11. Grasl MC, Pokieser P, Gleiss A, et al. A new blended learning concept for medical students in otolaryngology. Arch Otolaryngol Head Neck Surg. 2012;138(4):358-366. 12. Tergan SO. Hypertext and hypermedia: conception, learning options, learning problems and perspectives. In: Issing LJ, Klimsa P, eds. Information and Teaching With Multimedia. Weinheim, Germany: Beltz PVU; 2002:99-111.
4. Khogali SE, Davies DA, Donnan PT, et al. Integration of e-learning resources into a medical school curriculum. Med Teach. 2011;33(4):311-318. 5. Dror I, Schmidt P, O’Connor L. A cognitive perspective on technology enhanced learning in
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
159
Original Investigation
Taking a Fresh Look at the Skull Base in Otorhinolaryngology With Web-Based Simulation Student’s Interactive Skull-Base Trainer (SISTer) Peter Freiherr von Sass; Kathrin Scheckenbach, MD; Martin Wagenmann, MD; Thomas Klenzner, MD; Joerg Schipper, MD; Adam Chaker, MD
IMPORTANCE The increasing amount of medical knowledge and necessity for time-effective teaching and learning have given rise to emerging online, or e-learning, applications. The base of the skull is a challenging anatomic area in the otorhinolaryngology (ORL) department—for both students and lecturers. Technology-enhanced learning might be an expedient approach to benefit both learners and lecturers. OBJECTIVE To investigate and create for advanced medical students a self-assessed adaptive e-learning application for the skull base within our curriculum of otolaryngology at the University Medical Center of Heinrich Heine University, Düsseldorf, Germany. DESIGN, SETTING, AND PARTICIPANTS Pilot approach with prospective evaluation of a newly implemented web-based e-learning simulation. The e-learning application (Student’s Interactive Skull-Base Trainer) was made accessible as an elective course to a total of 269 enrolled medical students during the first 2 semesters after web launch. INTERVENTIONS Spatiotemporal independent e-learning application for the skull base. MAIN OUTCOMES AND MEASURES Self-assessed evaluation with focus on general acceptance and personal value as well as usage data analysis. RESULTS The application was well accepted by the learners. More than 80% of the participating students found the application to be a beneficial tool for enhancing their analytical and clinical problem-solving skills. Although the general matter of the skull base seemed to be of lesser interest, the concept of anchored instructions with the use of high-end, interactive, multimedia-based content was considered to be particularly suitable for this challenging topic. Most of the students would have appreciated an extension of optional e-learning modules. CONCLUSIONS AND RELEVANCE With this pilot approach we were able to implement a useful and now well-accepted tool for blended learning. We showed that it is possible to raise interest even in this very specialized subspecialty of ORL with overall individual learning benefit for the students. There is a demand for more e-learning and web-based simulation to support the existing curricula in a hybrid, blended way. Author Affiliations: Department of Otolaryngology, Universitätsklinikum Heinrich Heine Universität, Düsseldorf, Germany (von Sass, Scheckenbach, Wagenmann, Klenzner, Schipper); Department of Otolaryngology, Klinkum rechts der Isar, Technische Universität, München, Germany (Chaker).
JAMA Otolaryngol Head Neck Surg. 2015;141(2):154-159. doi:10.1001/jamaoto.2014.3041 Published online December 4, 2014. 154
Corresponding Author: Peter Freiherr von Sass, Hals-Nasen-OhrenKlinik des Universitätsklinikum Düsseldorf, Moorenstrasse 5, 40225 Düsseldorf, Germany (Peter.vonSass @med.uni-duesseldorf.de). (Reprinted) jamaotolaryngology.com
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
Student’s Interactive Skull-Base Trainer
Original Investigation Research
C
urricula in academic medicine and clinical teaching are currently undergoing substantial changes. Along with an exponential increase of medical expert knowledge, there are ongoing efforts to restructure curricula from an educational and didactic perspective.1-3 Educational goals, funding constraints, and teaching repertoire at hand in clinical teaching today have to be taken into consideration just as well as different individual learning types, previous knowledge, and personal interest. The aim is to enable medical students to develop (1) their diagnostic and analytical skills based on accurate anatomical and physiological knowledge, (2) clinical competencies, and (3) a solid foundation for mature clinical judgment. Widespread availability, use, and acceptance of web-based applications and the rise of technology-enhanced learning offer emerging blended- and online, or e-learning, opportunities.4,5 Otorhinolaryngology (ORL) is a closely intertwined clinical discipline based on the complex anatomy and physiologic characteristics of the head and neck, including the base of skull, with usually little representation in medical curricula and national medical license examinations. Often, students will achieve only basic-level cognitive learning objectives. The idea behind the Student’s interactive Skull-Base Trainer (SISTer) is indeed simple: creating a motivating hybrid learning environment that combines well-established lectures and seminar teaching methods with the emerging opportunities of web-based, individual, self-directed learning in the anatomically challenging area of the skull base. We hypothesized that the use of multidimensional and interactive display of anatomical structures embedded in clinical case histories from the university hospital is well suited for the complex base of skull and boosts self-assessed ORL learning outcome. The project was added to the regular curriculum, which also contains bedside teaching, lectures, seminars, tutorials, and study-guided learning. The aim of our project was to evaluate a didactic pilot elearning approach on this specialized and narrow subject to study the general acceptance and self-assessed learning outcome. Furthermore, we aimed to begin a dialogue on blended learning and web-based simulation with students to enhance quality of learning in the ORL curriculum, and to further implement—if positively evaluated—a blended learning concept.
Table 1. Overview of the 3 Case Studies Subject of Case Study
Screen Cards, No.
Exercises, No.
Inverted papilloma
18
11
Pleomorphic adenoma
17
5
Vestibular schwannoma
14
8
as well as the evaluation form were voluntary and anonymous. No compensation for participation was granted.
Learning Objectives The first objective was to develop analytical clinical skills on the skull base regarding recognition of abnormalities based on patient history and clinical findings and to further integrate these diagnostic findings into a more elaborate clinical diagnosis. Second, to encourage integration into interdisciplinary clinical pathways, students were guided through advanced therapeutic concepts offered as expert discussion within the application while the learning outcome was examined by the means of multiple-choice assessments in a selfdirected manner.
Didactic Concept We chose the didactic concept of anchored instructions to linearly guide the students throughout the whole process of diagnosis, treatment, and follow-up.5
Content Three interactive clinical cases on a total of 49 web-based screen cards were developed to depict clinical presentation, diagnostic procedures, and multimodal therapy, including follow-up care, based on anonymized patients from our university hospital. The cases present regular ORL diseases that further develop to complex interdisciplinary skull-base cases (Table 1). Students were able to register and access SISTer through a university link for http://www.Casus.net, a widely used e-learning platform in German universities. The availability of the new module was announced in the main ORL lecture on university and student union websites. The program was explicitly announced as not mandatory; however, students were informed that the content might be relevant for 10% of the ORL examination.
Self-Directed Learning Performance Assessments
Methods Prospective Pilot Study We performed a prospective, anonymous pilot study approach for a new web-based e-learning and simulation module in our ORL curriculum. Evaluation forms as well as electronically acquired user data were analyzed. The study was funded by the e-learning initiative HeinEcomp. The Faculty of Medicine of Heinrich Heine University, Düsseldorf, Germany, served as an equivalent to an institutional review board because the analysis concerned students (and not patients of the medical center) and authorized prospective evaluation and analysis. Participating students did not provide written informed consent as both the e-learning course jamaotolaryngology.com
Twenty-four of 49 screen cards contain exercises in which students are able to fill in their diagnostic interpretations as free text using an automated answer-recognition tool, multiplechoice, or multiple-selection questions, plus many others. After answering screen card questions, students receive immediate feedback comparing their answers with the most correct answer in text form as well as a percentage of correct answers and an author’s comment. At the end of each case study, students receive a summary of feedback on educational objectives as well as their personal performance statistics. For those students wanting to get even more deeply involved, additional expert commentaries are available to clarify more details of the disease or to explain further clinical course of actions and processes.
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
155
Research Original Investigation
Student’s Interactive Skull-Base Trainer
Figure 1. Examples of Screen Cards
Table 2. Demography and User Statistics Semester
A
Characteristic
1
2
135
134
Male
60
53
Female
75
81
104
109
Enrolled students, No.
Students logged in, No. Fully finished Received evaluation forms
89
100
48
None
ation data were irreversibly lost owing to technical issues. To date, we have partially adapted and included various changes in composition and content of the case studies fitting into a larger e-learning strategy, thus averting comparability.
Finalizing the e-Learning Module As seen in Table 2, more than 90% of the students who logged onto the e-learning platform finished the program completely, which we defined as visiting a minimum of 45 of the screen cards (92%). To do so, users had to answer virtually every interactive question in the program. We further logged the percentage of correct answers and the time spent on each card. The range of correct answers varied from 33% to 64%, with students spending on average 43 to 70 minutes on each case, which implies 2.8 to 3.9 minutes per card.
B
Providing Incentive for User Demand
Screenshot of 2 typical screen cards. A, Dichotomous composition of text and multimedia content. B, Multiple-choice question.
Layout and Navigation The dichotomous screen card layout enables the presentation of text on the left side and animated, interactive, or static multimedia content on the right side. We included anatomic images, sequences of clinical examinations, short videocasts of surgical and endoscopic procedures with commentary by experienced ORL surgeons, and pictures of histologic or complementary diagnostic findings (eg, hearing tests, magnetic resonance imaging, angiography, and computed tomographic sequences) (Figure 1). On many of these interactive screens, students are able to navigate through different imaging layers and perspectives, with anatomical text labels fading in or out on demand. Before clicking to the next screen, students have to answer questions that keep a record of their learning progress.
Results Completion of SISTer After Web Launch During the first 2 semesters following web launch, almost threequarters of all enrolled students (189 of 269) completed all 3 cases. More than 50% of the first-semester cohort submitted the evaluation form (48 of 89). In the subsequent semesters, we received only data on usage characteristics, while some electronic evalu156
As part of the evaluation, we further investigated the motivation for spending time and effort in the e-learning program. Ninety-six percent of students indicated that relevance for the final examination as one point of motivation, whereas curiosity (51%) and the expected advantage of e-learning (49%) ranged lower. A specific interest in ORL as a subject (29%) or interest in interdisciplinary problems (29%) both received fairly low priority (Figure 2). As part of the educational goals, we also investigated the impact on analytical and clinical problem-solving skills by evaluating self-assessments by the online participants. Most of the students (>80%) found SISTer a beneficial tool for enhancing those skills. About the same number of students (80%) indicated that the web-based 3-dimensional depictions helped them to relate the complex anatomy to the clinical case (Figure 3).
Good Acceptance of e-Learning on the Skull Base Electronic evaluation showed that general acceptance of our optional e-learning course was very good. Concerns that additional content could lead to an overcrowding of the tightly packed fourth-year curriculum were refuted by the students (86%). In fact, about half of the students (54%) indicated a desire for more in-class lectures. Moreover, even 82% of the students would have liked an extension of optional e-learning modules for self-directed hybrid learning. The skull base per se seems to be of lesser interest. Few students (24%) would welcome an elective in-class course on the subject (Figure 4). Interestingly, most students (73%) found the base of skull particularly well suited for e-learning.
JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
jamaotolaryngology.com
Student’s Interactive Skull-Base Trainer
Original Investigation Research
Enabling Students to Identify Learning Deficits by e-Learning We assessed whether the students found that SISTer blended well into the existing curriculum. The response from almost 80% was positive (Figure 5); the same percentage of students further stated that working with this e-learning module enhanced their understanding of previously unknown content. Even more (90%) (Figure 5) conceded that their personal knowledge deficiencies were revealed by our self-directed elearning application.
standing of context formerly unknown to the students, thus enabling better quality of teaching and more elaborate cognitive learning. Figure 2. Motivation of Students to Work With SISTer 96%
100
Discussion We saw broad acceptance for our web-based e-learning module, indicated by user characteristics and prospective standardized evaluation even though SISTer addresses a very specific field. In particular, we think that spatiotemporal independence and self-directed learning (as described by Ruiz et al6) in our web-based concept, together with cross-linking of complex anatomy and initially basic ORL cases that developed to specific complex entities, helped to process information into meaningful chunks. The motivation to acquire knowledge and make sense out of the presented material is an important factor and is also referred to as generative cognitive processing.7 We suggest that this enhanced the under-
Answers, %
75
51% 50 29%
49%
29%
25
0
General Interdisciplinary Relevance for Advantages of Interest in Problem Examination e-Learning ORL Question: Why did you work with these case studies?
Curiosity
Results of a multiple-selection question in percentages of 48 users (multiple answers possible). “Relevance for examination” was stated as one point of motivation to work with the e-learning application by almost all of the users, followed by “curiosity” and “advantages of e-learning.” ORL indicates otorhinolaryngology.
Figure 3. Self-assessment of Analytical and Clinical Problem-Solving Skills for 48 Users Fully applicable
Applicable
Less applicable
Not applicable
Interactive assessment of medical findings online enhanced my diagnostic skills Interactive demonstration of anatomic structures enhanced my diagnostic skills Web-based 3-dimensional depictions helped to relate clinical context and anatomy better 0
20
40
60
80
100
Results, %
Results of evaluation in percentages for analytical and problem-solving skills by means of multiple-choice questions. More than 80% of the learners rated the program as beneficial.
Figure 4. Acceptance of e-Learning and Specific Interest in the Subject for 48 Users Fully applicable
Applicable
Less applicable
Not applicable
Additional e-learning makes curricula to become even more busy I would appreciate more e-learning content to choose from and combine with the curriculum I wish for more in-class lectures
I wish for more e-learning Topic of skull base is in particular well suited for e-learning I wish for an elective course on base of skull 0
20
40
60
80
Answers, %
jamaotolaryngology.com
100
Answers of multiple-choice questions. There was a request for more e-learning, and the skull base was noted as particularly suitable as a subject by about three-quarters of the students.
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
157
Research Original Investigation
Student’s Interactive Skull-Base Trainer
Figure 5. Assessment of Individual Cross-Linking of Learning Fully applicable
Applicable
Less applicable
Not applicable
Working with the case studies complemented the in-class lectures well Working with the case studies enhanced my understanding of contexts formerly unknown Working with the case studies pointed out my shortcomings on the subject 0
20
40
60
80
100
Answers, %
Considering the very broad acceptance found in our investigation, we can state that we were able to create the necessary motivation in a narrow and specialized topic (with limited importance to most learners’ future professional development) by offering a novel and appealing resource for self-directed learning with adequate cognitive interactivity.
Relevance for Examinations as a Trigger for Acceptance As expected, relevance for the final examination was a trigger; however, students described the program as beneficial for their clinical problem-solving skills. Though the multimedia-based approach has been a controversial subject in the literature,8-10 we believe it is particularly accountable for this positive result; the 3-dimensional depictions were explicitly described as being helpful for linking anatomy and clinical context. Individual performance monitoring would have been feasible, but we explicitly designated the course as optional, offering only the incentive that 10% of the final examination might contain content of the e-learning application. In fact, with this 10% advantage for the final examination, we have found it difficult to explain the high interest of the students. Long-term usage data will be necessary to determine whether this was biased by the novelty factor of the application or mere coincidence of more highly motivated students. In a pilot project study of blended learning in Vienna, Austria, students did not benefit from e-learning.11 Students in the control group were more satisfied with traditional teaching methods than students in the e-learning group. The authors stated that medical students might be conservative and therefore might rely more on traditional teaching methods like lectures than on novel approaches.11 Although the design of our investigation did not allow comparison between different groups, we clearly interpret our results as an indication that optional, additional, and blended e-learning applications and simulations can be very well accepted. While only about 50% of participating students in our study would have liked more in-class lectures, the number wishing for more e-learning content exceeded 80% in our evaluation, therefore refuting the stereotype of overly conservative medical students. This positive response on e-learning is consistent with findings of many other investigations (eg, Khogali et al 4 showed that elearning on cardiovascular system was rated as valuable by 96% of the responding students). 158
Forty-eight users were asked multiple-choice questions on integration of the e-learning application into the existing curriculum. About 80% stated that the online contents complemented the lecture well and facilitated comprehension of the formerly unknown contents.
Evaluation by Students of the Benefit of SISTer In particular, the self-assessed, adaptive, and case-based approach and multidimensional imaging were considered to be useful for this challenging and difficult topic. The skull base, with its highly complex pathologic abnormalities, is hardly covered in medical curricula. The concept of anchored instructions with the use of high-end, interactive, multimedia-based content was evaluated as being particularly suited for this challenging topic. Students were able to acquire a deeper understanding of complex clinical presentation and anatomy in the ORL department and base of skull in an entertaining, multimedia-based, handson, online approach. Although not of personal interest for most, the blended approach and highly refurbished content activated the learners to meaningful learning as far as could be determined with our self-assessment questionnaire.
Study Limitations There is potential bias because of the pilot character of our study. Long-term user data are required, but owing to ongoing changes in our faculty curriculum are currently unavailable. Additional possible bias may be found in the selection of participants. We explicitly chose an elective course design. We kept usage and performance anonymous so as to receive an unaffected feedback from the students. It is therefore very possible that more ambitious students completed the application. This would be compatible with the observation that learners with better previous knowledge benefit more from multimedia-based e-learning.12 This view can, however, be challenged by the rather low percentage of correct answers on the screen cards. Just as easily, the poor result could be attributable to the new and difficult subject. Ninety percent of the students stated that the program identified gaps in their personal knowledge. Regarding the overall positive evaluation, we interpret this as motivating self-criticism rather than an indicator of a disproportionate requirement.
SISTer as a Helpful Pilot Project We electronically measured the time that users spent on each case study. The time spent averaged about 1 hour per case, totaling 3 hours. Even experienced scholars might find it difficult to offer a comprehensive overview about the challenging anatomy, diagnostic pitfalls, and multimodal, interdisciplinary approaches to medical students and newcomers when using standard educational methods such as lectures or seminars. Apart from the fact that the subject was barely covered,
JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2 (Reprinted)
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
jamaotolaryngology.com
Student’s Interactive Skull-Base Trainer
Original Investigation Research
so far we were able to blend it into an existing curriculum in order to enhance basic ORL skills as well as specific expert knowledge with a reasonable amount of departmental effort. Depicting the same material in an in-class lecture would presumably take longer and lack the advantages of individually adapted and spatiotemporal independent learning. We presume that an in-class lecture would have lowered the general acceptance level obtainable from the rather restrained answers on the question about an elective skull base course.
ARTICLE INFORMATION Submitted for Publication: May 28, 2014; final revision received October 1, 2014; accepted October 16, 2014. Published Online: December 4, 2014. doi:10.1001/jamaoto.2014.3041. Author Contributions: Mr von Sass had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: All authors. Acquisition, analysis, or interpretation of data: von Sass, Chaker. Drafting of the manuscript: von Sass, Chaker. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: von Sass, Chaker. Obtained funding: Schipper, Chaker. Administrative, technical, or material support: von Sass, Scheckenbach, Klenzner, Wagenmann, Chaker. Study supervision: von Sass, Klenzner, Schipper, Chaker. Conflict of Interest Disclosures: Dr Wagenmann has given lectures sponsored by Allergopharma, ALK-Abello, Bencard, HAL Allergy, and Meda and has received grant support or consultant arrangements with Allergopharma, ALK-Abello, Bionorica, and HAL Allergy. Dr Chaker has given lectures sponsored by GSK and has received grant support or consultant arrangements with Allergopharma, ALK-Abello, Stallergenes, HAL Allergy, Mundipharma, Zeller AG, and Novartis Pharma AG. No other disclosures are reported. Funding/Support: Student’s interactive skull-base Trainer (SISTer) was funded by the e-learning initiative HeinEcomp of the medical faculty of Heinrich Heine University, Düsseldorf.
jamaotolaryngology.com
Conclusions The e-learning tool SISTer uses the resource e-learning in a didactic appropriate manner and complements hitherto existing classroom teaching. We think that it offers an additional value for medical students who learn cutting-edge diagnostic techniques and interdisciplinary clinical pathways in academic medicine.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Additional Contributions: Christian Babel, Thomas Baehring, and Beate Meinert (HeinEcomp), all of Medical Faculty of Heinrich Heine University, provided technical assistance. Bernd Turowski, MD, Department of Neuroradiology, and Daniel Hänggi, Department of Neurosurgery, Heinrich Heine University, provided support when setting up SISTer. Michael Hemming, Munich, Germany, critically reviewed the manuscript. Alexandra Hesse, and Pascal Berberat, MD, Medical Faculty of Technische Universität München, helped with lost literature and provided helpful discussions, respectively. No compensation was granted for assistance. REFERENCES 1. Schnabel KP, Boldt PD, Breuer G, et al. A consensus statement on practical skills in medical school: a position paper by the GMA Committee on Practical Skills. GMS Z Med Ausbild. 2011;28(4):Doc58. 2. Fegert JM, Obertacke U, Resch F, Hilzenbecher M. Die Qualität der Lehre nicht dem Zufall überlassen. Dtsch Arztebl. 2009;106(7):A290-A291. 3. Stausberg J, Kühnl A. E-learning in der medizin: Erweitertes Angebot. Dtsch Arztebl. 2013;110(37): A1692.
medical training: great opportunities, pitfalls and challenges. Med Teach. 2011;33(4):291-296. 6. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81(3):207-212. 7. Mayer RE. Applying the science of learning to medical education. Med Educ. 2010;44(6):543-549. 8. Levinson AJ, Weaver B, Garside S, McGinn H, Norman GR. Virtual reality and brain anatomy: a randomised trial of e-learning instructional designs. Med Educ. 2007;41(5):495-501. 9. Pereira JA, Pleguezuelos E, Merí A, Molina-Ros A, Molina-Tomás MC, Masdeu C. Effectiveness of using blended learning strategies for teaching and learning human anatomy. Med Educ. 2007;41(2): 189-195. 10. Garg AX, Norman GR, Eva KW, Spero L, Sharan S. Is there any real virtue of virtual reality? the minor role of multiple orientations in learning anatomy from computers. Acad Med. 2002;77(10)(suppl):S97-S99. 11. Grasl MC, Pokieser P, Gleiss A, et al. A new blended learning concept for medical students in otolaryngology. Arch Otolaryngol Head Neck Surg. 2012;138(4):358-366. 12. Tergan SO. Hypertext and hypermedia: conception, learning options, learning problems and perspectives. In: Issing LJ, Klimsa P, eds. Information and Teaching With Multimedia. Weinheim, Germany: Beltz PVU; 2002:99-111.
4. Khogali SE, Davies DA, Donnan PT, et al. Integration of e-learning resources into a medical school curriculum. Med Teach. 2011;33(4):311-318. 5. Dror I, Schmidt P, O’Connor L. A cognitive perspective on technology enhanced learning in
(Reprinted) JAMA Otolaryngology–Head & Neck Surgery February 2015 Volume 141, Number 2
Copyright 2015 American Medical Association. All rights reserved.
Downloaded From: http://archotol.jamanetwork.com/ by a New York University User on 05/23/2015
159
