2014; 36: 13–18
HOW WE. . .
How we make nephrology easier to learn: Computer-based modules at the point-of-care JEFFREY H. WILLIAM & GRACE C. HUANG Beth Israel Deaconess Medical Center, Boston, MA, USA
Background: Novel educational tools, such as case-based learning in a web-based module format, are an effective approach to teaching clinical concepts to medical trainees, especially if the situations are clinically relevant and the intervention is delivered at the point-of-care. Though studies have evaluated the effectiveness of point-of-care reference materials, limited literature addresses active web-based interventions designed for completion at the point-of-care. Aims: By taking advantage of existing technological resources and integrating an effective learning modality into the clinical environment, we can increase trainee understanding of high-yield topics in clinical nephrology. Methods: We designed interactive, case-based computer-based modules in Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities, with interwoven multiple-choice and free text questions with immediate feedback, supplemental practice questions, and enrichment material to be completed in the clinical environment. All medicine trainees at an urban, academic institution were invited to participate in a needs assessment, pre and post knowledge tests, and module completion. Results: Most trainees believed the modules were ‘‘very’’ or ‘‘extremely helpful’’ in understanding the selected topic and that they would likely change their clinical practice. Those who completed the modules performed better on a post-intervention knowledge assessment. Free-text feedback was overwhelmingly supportive of the modules. Conclusion: Our findings confirmed that a novel, simplified approach to renal content by making it readily applicable to a clinical context and available at the point-of-care improves trainee understanding of high-yield topics in nephrology.
Introduction
Practice points
This generation of medical trainees lives in a technology-rich, Internet age, with a need to disseminate information in an efficient and effective manner (Horwitz & Detsky 2011). A novel educational tool, such as case-based learning in a web-based module format, has been an effective approach to teaching a variety of clinical concepts to medical trainees, especially if the situations are highly clinically relevant and the learner has a vested interest in the new material (Westmoreland et al. 2010; Lee et al. 2011; Truncali et al. 2011; Sangvai et al. 2012). Furthermore, enhancing the relevance of a topic for a medical trainee is accomplished by delivering the educational intervention at the point-of-care. Using a real clinical context to integrate otherwise ‘‘complex’’ pathophysiology into case-based learning helps trainees retain knowledge better. When offered a choice of how a teaching tool such as this should be delivered, most nephrology fellows and trainees would state that they prefer online educational material for learning (Desai et al. 2011). The interest in nephrology among medical students and residents has dropped substantially over the past decade. Despite the increasing subspecialization of Internal Medicine residents in the United States, nephrology is just one of only two medical subspecialties that have seen a declining number of fellowship applications since 2002. A recent survey of medical students exploring nephrology as a career choice
. Trainees in medicine learn more effectively through high-yield and relevant educational interventions at the point-of-care . Better understanding of subspecialty topics increases interest in the field . Novel educational tools must focus on utilizing preexisting technologies that are already relied upon by trainees . Simplifying the approach and limiting the amount of information in an educational intervention can enhance long-term learning
20 13
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Abstract
showed that they felt topics in a traditional renal pathophysiology course were ‘‘too complex [and] lacked relevance’’ (Rosner et al. 2009). While nephrology fellows report that favorable exposures to the field of nephrology and exemplary mentors influenced their decision to apply for a fellowship, many students and residents state that their own exposures to the field are minimal and frequently negative, citing the high burden of illness of dialysis patients and complicated content (McMahon et al. 2012; Jhaveri et al. 2013). As a result, the American Society of Nephrology (ASN) formed the Task Force on Increasing Interest in Nephrology Careers. Their
Correspondence: Jeffrey H. William, Division of Nephrology, Department of Medicine, Beth Israel Deaconess Medical Center, 185 Pilgrim Road, Farr 8, Boston, MA 02215, USA. Tel: (617) 632-9880; fax: (617) 632-9890; email: [email protected] ISSN 0142–159X print/ISSN 1466–187X online/14/10013–6 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2013.847912
13
J. H. William & G. C. Huang
recommendations included the development of creative educational rotations, increased use of social media, and creating novel educational tools to make nephrology more attractive to a trainee that might be considering a different field (Parker et al. 2011). To meet these challenges, educators need to take advantage of the resources we already use on a daily basis, creating new curricula that address poorly understood topics in nephrology through computer-based learning in order to adapt best teaching practices to today’s learners (Kohan 2011). Though numerous studies have evaluated the effectiveness of various existing point-of-care reference materials (Scott et al. 2000; Farrell 2008; Phua & Lim 2008; Banzi et al. 2010), there is limited literature that addresses novel active web-based interventions designed for delivery at the point-of-care.
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
What we did Our goal was to promote active learning of nephrology through the development of computer-based modules at the point-of-care that address high-yield, relevant topics in clinical nephrology, with the aim of achieving a more long-lasting understanding of these historically poorly understood clinical entities.
Module design We designed three interactive computer-based modules in Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities using an authoring tool in the Harvard Medical School (HMS) student portal website, accessible at any computer with an Internet connection. The modules were case-based, centered about a patient presentation, with embedded multiplechoice questions and multimedia, along with the opportunities to provide free text answers and to receive explanations for each answer choice. Basic concepts in pathophysiology essential to understanding the topic were interwoven within each case, with reference to an ‘‘Appendix’’ section that trainees were instructed to complete for enrichment purposes following completion of the module. For appropriate topics, we also included practice questions as supplemental material to solidify the new learning approach once the module was complete. These modules were designed to last no longer than 10 min and with the intention of completion while working in a clinical environment, though no time limit or access restrictions by location were imposed on any user.
Needs assessment We gave all trainees in the Internal Medicine Residency program at the Beth Israel Deaconess Medical Center, a large urban academic institution, the opportunity to partake in the intervention. We disseminated by e-mail a survey that constituted a needs assessment of trainees, capturing information about how trainees approach new clinical questions (i.e. admission of a patient on home peritoneal dialysis), soliciting any perceived knowledge deficiencies within nephrology, and elucidating preferred learning styles among trainees. The survey was completed by 100 Internal Medicine trainees, PGY-1 through PGY-3 (Table 1). Most trainees 14
(90%) identified UpToDateÕ , described by its authors as an ‘‘evidence-based clinical decision support resource authored by physicians to help healthcare practitioners make decisions at the point of care’’, (‘‘UpToDate’’ 2013) as their preferred resource for learning new information about a poorly understood topic. Almost 60% believed that it ‘‘almost always’’ resulted in answering their question, but 43% of respondents reported that it did not result in long-term retention of the clinical concept. Only 8% of trainees reported that they were willing to spend more than 15 min looking up a new topic while actively taking care of patients ‘‘on the wards.’’ The majority of respondents would only devote 30 min or less to learning more about a topic ‘‘off the wards’’ or at home. Related to learning styles, respondents were more likely to state that repetition and clinical application were more effective means of learning new clinical material than reading the same information from their preferred resource. They also believed that understanding the background pathophysiology was helpful in longer-term understanding of medical concepts. Specifically regarding nephrology content, nearly half of respondents believed that renal topics are more difficult to understand than topics in other specialties, citing confusing pathophysiology and difficulty in applying the concepts to clinical care. The most frequently chosen module was ‘‘Principles of Dialysis’’, completed by 39 trainees, with ‘‘Hyponatremia’’ and ‘‘Acid–Base abnormalities’’ following closely behind (Table 2). Many respondents completed the module within the hospital setting, but very rarely while actively admitting new patients. When included in the modules, the practice question and appendix sections were unanimously rated as ‘‘very’’ or ‘‘extremely helpful’’, despite usually requiring greater than 10 min to complete, within the context of the module itself. Numerous free-text comments specifically cited the helpfulness of the practice questions as well as the simplified approach to each topic (Box 1). Subjectively, most trainees (up to 96%) felt that the modules were ‘‘very’’ or ‘‘extremely helpful’’ in understanding the topic and indicated they would be likely to change their clinical practice in treating a patient with this condition. Most trainees expressed interest in developing modules like these for topics in other specialties as well (52–64%).
Evaluation: Knowledge tests and module feedback Prior to completing any modules, we asked participants to complete a challenging pre-intervention knowledge test, consisting of a case with multiple-choice questions about clinical judgment that integrated the topics of each module. Each question represented an important learning objective that would be addressed within each of the three modules. The case simulated a common clinical situation that medicine trainees are exposed to on a regular basis. As in clinical practice, many questions included multiple correct answer choices; receiving credit for a specific question required all correct choices and none of the incorrect choices, to be selected. Following each module, a post-module survey solicited data on trainee usage (i.e. time to completion, environment completed in), satisfaction, and free-text feedback for future improvement. Months after completing these
Computer-based modules at the point-of-care
Table 1. Baseline characteristics.
Year of study
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
PGY-1 Preferred resource for learning new material in clinical setting UpToDateÕ Small medical reference Non-medical search engine (i.e. Google) PubMed Time spent reading about new topic in clinical setting 1–5 min 6–10 min 11–15 min 415 min Time spent reading about new topic at home/outside of clinical setting 1–15 min 16–30 min 31–45 min 445 min I do not read more at home Subjective understanding of topic 1 ¼ "I do not understand this topic very well" 4 ¼ "I understand this topic very well" Hyponatremia (mean, SD) Principles of Dialysis (mean, SD) Acid–Base abnormalities (mean, SD) Are Renal topics more difficult to understand than other specialties? Yes No Most effective modality for learning renal topics in past Lectures in medical school or residency noon conferences Small groups (i.e. med school pathophysiology course, attending rounds) Self-learning (i.e. UpToDateÕ , PubMed, or other online modules) Individual interactions with nephrologists (i.e. elective time, one-on-one sessions, or time spent with Renal fellows consulting on your patient) Time spent on Renal consultation service 51 week 1 week 1–2 weeks 42 weeks Self-reported learning style Learn it the first time ¼ 1 vs. Require repetition ¼ 4 (mean, SD) Read and remember ¼ 1 vs. Relevant to patient care ¼ 4 (mean, SD) Remember algorithms ¼ 1 vs. Need to look up algorithms ¼ 4 (mean, SD) Work through problems individually ¼ 1 vs. with colleagues ¼ 4 (mean, SD) Learn concepts best with accompanying pathophysiology ¼ 1 vs. without pathophysiology ¼ 4 (mean, SD)
PGY-2
PGY-3
40 3 3 1
34 0 2 0
16 1 0 0
14 19 12 2
5 14 12 5
6 7 3 1
24 13 1 1 8
15 12 3 1 5
10 2 0 2 3
2.8 (0.8) 1.7 (0.6) 2.8 (0.8)
3.2 (0.5) 2.1 (0.7) 2.9 (0.6)
3.1 (0.6) 2.1 (0.7) 2.9 (0.7)
22 25
19 17
6 11
10 16 8 13
2 12 9 13
1 2 7 7
39 4 2 2
13 8 12 3
8 4 5 0
3.1 3.2 3.1 2.3 1.6
(0.8) (0.8) (0.8) (0.9) (0.7)
3.3 3.4 2.9 2.1 1.6
(0.8) (0.7) (1.0) (1.0) (0.8)
3.1 3.4 3.1 2.4 2.1
(0.8) (0.7) (1.0) (0.9) (0.9)
Baseline characteristics were reported in the needs assessment, where respondents were provided with a case of a new admitted patient on home peritoneal dialysis and asked to answer each question in the context of the case. Questions also addressed attitudes and exposure to nephrology, learning environments, and learning styles.
surveys, we sent an identical post-intervention knowledge test to all trainees. The pre and post-intervention knowledge tests were completed by 72 (46%) and 38 (24%) of trainees, respectively. There were 26 unique pairs of data, and t-tests analyses did not reveal significant differences between pre- and post-intervention knowledge tests. However, in unpaired comparisons, trainees’ performance improved, on average, by 10.6% ( p ¼ 0.05).
What’s next Our findings suggested that targeted, clinically-oriented, computer-based teaching of common topics in nephrology may result in improved understanding, but may also serve to clarify
clinical entities that are often difficult to grasp for trainees, as they self-identified in the needs assessment. Trainees identified a very high utilization of electronic resources to research new medical knowledge and apply it to their patients, but this occurs under great time pressure and with questionable longer-term learning. While actively caring for patients, our trainees did not feel they could devote more than 15 min to learning new clinical material and applying these concepts. Therefore, these computerbased modules were designed to be completed in a timesensitive clinical environment, focusing on high-yield patient presentations in nephrology, a medical specialty that a relatively large percentage of all respondents feel is more difficult to grasp than others. Though the module utilization rate was relatively low, users of the modules unanimously commented on their helpfulness and the
15
J. H. William & G. C. Huang
Table 2. Module surveys.
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Module topic Hyponatremia (n ¼ 23)
Acid–Base abnormalities (n ¼ 22)
Principles of Dialysis (n ¼ 39)
1 3 6
2 3 5
3 5 10
13 0
11 0
18 3
0 3 11 9
0 2 4 16
0 14 17 8
0 3 16 4
0 1 9 12
0 8 22 9
4
1
4
11 2 6
13 3 5
17 8 10
7 15 1 0
7 14 1 0
7 23 9 0
12 11
14 8
22 17
Where and when did you complete the module? On the wards, ICU, or nightfloat (on-call) On the wards/ICU, but not on-call In the hospital, but not on a rotation with call (i.e. elective, ambulatory block, or research) At home Other Length of time to complete module 1–5 min 6–10 min 11–15 min 415 min Subjective rating of helpfulness in understanding the topics Not helpful Somewhat helpful Very helpful Extremely helpful Preference of environment in which to complete module On the wards, ICU, or nightfloat During any elective or ambulatory time (in the hospital) Specifically before, during or after a Renal elective At home/not in the hospital Will you change your practice after completing this module? Definitely Probably Probably not Definitely not Are you willing to help design your own module in a specialty of your choice? Yes No
At the completion of each module, the respondent was asked to fill out a survey to report feedback of their experience. This survey was designed to record when and where the module was completed, how long it took, and how helpful it was. Participants were also asked to provide suggestions to improve module design by reporting their individual preferences about where a module should be completed and if they would be willing to design a similar module in another specialty.
Box 1. Selected module feedback. Principles of dialysis ‘‘This module was phenomenal, and I very much appreciated the ‘‘return to basics.’’ Often times it feels like we’re supposed to know this kind of stuff already, so being able to address it in this fashion is very helpful’’ ‘‘The most clear explanation of dialysis I’ve ever seen.’’ ‘‘Very easy to use, and the information included was pertinent to my practice.’’ ‘‘Well designed, good use of graphics, nice having ‘‘active’’ components (questions embedded)’’ Hyponatremia ‘‘Very good simple approach for dealing with a potentially very difficult topic’’ ‘‘Fantastic hyponatremia module! Learned a lot and will be using this framework more. Thank you!’’ ‘‘I enjoy how there’s personal ‘‘feedback’’ built in through the questions and ability to answer them.’’ Acid–base abnormalities ‘‘These are great! The best focused practice that I have had on acid–base status since starting residency’’ ‘‘Great practice questions to reinforce the information.’’ ‘‘Very high yield, practice centered approach’’ ‘‘Acid–base model is a great way to take a stepwise approach to a common and important medical topic.’’
likelihood that the approaches employed within each topic would likely change their future practice. Time was a key factor affecting module completion. While the modules were originally designed to be completed at the point-of-care in less than 10 min, many participants felt more comfortable completing them while off-service and took more time for completion. Though it may not be realistic to have trainees complete learning tools that require active 16
participation with integrated practice questions while concurrently admitting patients on a call night, about half of the trainees involved in this study believed they would still be most useful and reasonable to complete in a less busy clinical environment. This is consistent with the well-supported notion that adult learners acquire knowledge best when they can apply their newly acquired knowledge in real-time and see the result of their actions. Trainees especially appreciated the
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Computer-based modules at the point-of-care
opportunity to practice and apply the simplified approaches to hyponatremia and acid–base abnormalities. Despite the extra time they required to complete a module, additional practice questions were frequently requested in user feedback. Most users also completed the appendix sections containing pathophysiologic explanations of the topic, which may have contributed to the increased amount of time it took most trainees to complete the modules (greater than 15 min), but also to longer-term knowledge acquisition. These results reflect a recent meta-analysis of a heterogeneous pool of internet-based learning interventions showing that knowledge, skills, behavioral, and satisfaction outcomes are generally favorable when compared to no intervention, but are essentially equivalent to alternative teaching methods using multimedia (Cook et al. 2008). Through a systematic review, it was shown that increased feedback and interactivity may increase the time of a learning intervention, but also strengthen knowledge outcomes (Cook et al. 2010). Medical residents consistently state that time is the biggest factor in participating in additional learning tools while in a clinical setting. With new work hour restrictions in place for interns and an increased burden on completing clinical work falling to supervising residents, all trainees are feeling more pressured for time. Any learning tools meant for the clinical setting must focus on the delicate balance of time versus effective, durable learning. There were a number of limitations to this study that may inform future similar educational interventions. To make the modules as accessible as possible to trainees, we did not require them to log into a network or system. Therefore, we could not keep track of individual user behavior (e.g. number of times accessed) or performance on the questions within the module. Instead, successful completion of a separate postmodule survey (with a link and instructions provided upon completion of the module) was our only means of confirming completion of the modules and obtaining feedback. The amount of surveys and assessments involved in this study, in addition to the time-dependent reasons stated above, may have presented an obstacle to trainees completing the modules themselves. As one respondent put it, ‘‘the preintervention knowledge assessment took longer than the module!’’ However, the design of the knowledge assessment also mirrored the clinical setting. By presenting the material in a challenging, case-based format, with integration of multiple renal topics at one time, it simulated the complex reality of patient presentations. Questions were specifically targeted toward each module topic, with the intent of making the knowledge tests challenging enough to detect an improvement in performance after the intervention. Our pre-post study design focused on knowledge gain after completing the modules and did not provide a comparison with a control group of non-users or against another form of teaching. Finally, our study occurred at a single academic institution and our findings may differ from other sites.
Conclusion Case-based learning in a computer-based module format has already been proven effective in delivering educational
content to medical trainees in a variety of clinical scenarios and across specialties. Based on the existing literature, the specific design of our computer-based modules for point-ofcare delivery is unique and was well-received by the study participants. These three modules focused on high-yield topics in nephrology (Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities) and were designed to be completed at the point-of-care, incorporating active learning methods including multiple-choice and practice questions along the way to improve their clinical relevance and promote long-term knowledge acquisition. Though designed as self-learning modules, they may also be used as a teaching tool for daily rounds or provided as mandatory resources prior to the start of a nephrology elective to encourage higher level learning during their limited time on service. Given the interest of residents in designing modules like this for other high-yield medical topics, a potential library of the modules could be made available within any specialty for similar use and additional feedback can be incorporated. Though the integration of pathophysiology and clinical entities through a simplified approach is quite clear in nephrology, the use of this style of learning tool might inform teaching in other specialties in medicine. The development of educational innovations within nephrology, such as these modules, will continue to be essential in inspiring trainees to not only feel more comfortable applying these concepts to patient care but also to inform their career choice. Our findings show that simplifying the approach to renal content that is commonly perceived as confusing, increasing its applicability to a clinical context, and making it readily available at the point-of-care can improve trainee understanding of the most commonly encountered topics in nephrology.
Acknowledgements We would like to thank Dr C. Christopher Smith for his help in recruiting participants and his overall support of the work. We would also like to acknowledge Drs Stewart Lecker and Mihran Naljayan for their help in confirming the accuracy of the module content. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.
Notes on Contributors JEFFREY H. WILLIAM, M.D., is a Clinical Fellow in Nephrology in the Department of Medicine at Beth Israel Deaconess Medical Center and Harvard Medical School. GRACE C. HUANG, M.D., is an Associate Professor of Medicine at the Carl J. Shapiro Institute for Education and Research at Beth Israel Deaconess Medical Center and Harvard Medical School.
References Banzi R, Liberati A, Moschetti I, Tagliabue L. 2010. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res 12(3):1–16.
17
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
J. H. William & G. C. Huang
Cook DA, Levinson AJ, Garside S. 2010. Time and learning efficiency in Internet-based learning: A systematic review and meta-analysis. Adv Health Sci Educ: Theory Practice 15(5):755–770. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. 2008. Internet-based learning in the health professions: A meta-analysis. J Am Med Assoc 300(10):1181–1196. Desai T, Stankeyeva D, Chapman A, Bailey L. 2011. Nephrology fellows show consistent use of, and improved knowledge from, a nephrologistprogrammed teaching instrument. J Nephrol 24(3):345–350. Farrell A. 2008. An evaluation of the five most used evidence based bedside information tools in Canadian health libraries. Evid Based Libr Inf Pract 3(2):3–17. Horwitz LI, Detsky AS. 2011. Physician communication in the 21st century: To talk or to text? J Am Med Assoc 305(11):1128–1129. Jhaveri KD, Sparks MA, Shah HH, Khan S, Chawla A, Desai T, Kohan DE. 2013. Why not nephrology? A survey of US internal medicine subspecialty fellows. Am J Kidney Dis 61(4):540–546. Kohan DE. 2011. Training the next generation of nephrologists. Clin J Am Soc Nephrol 6(11):2564–2566. Lee CA, Chang A, Chou CL, Boscardin C, Hauer KE. 2011. Standardized patient-narrated web-based learning modules improve students’ communication skills on a high-stakes clinical skills examination. J Gen Intern Med 26(11):1374–1377. McMahon GM, Thomas L, Tucker JK, Lin J. 2012. Factors in career choice among US nephrologists. Clin J Am Soc Nephrol 7(11):1786–1792. Parker MG, Ibrahim T, Shaffer R, Rosner MH, Molitoris BA. 2011. The future nephrology workforce: Will there be one? Clin J Am Soc Nephrol 6(6):1501–1506.
18
Phua J, Lim TK. 2008. How residents and interns utilise and perceive the personal digital assistant and UpToDate. BMC Med Educ 8:39. Rosner M, Parker M, Kohan D. (2009). Nephrology as a career choice: A survey of medical students. [Abstract]. J Am Soc Nephrol 20:767A: SA-PO2867. Sangvai S, Mahan JD, Lewis KO, Pudlo N, Suresh S, McKenzie LB. 2012. The impact of an interactive Web-based module on residents’ knowledge and clinical practice of injury prevention. Clin Pediatr (Philadelphia). 51(2):165–174. Scott I, Heyworth R, Fairweather P. 2000. The use of evidence-based medicine in the practice of consultant physicians. New Zeal J Med 30:319–326. Truncali A, Lee JD, Ark TK, Gillespie C, Triola M, Hanley K, Kalet AL. 2011. Teaching physicians to address unhealthy alcohol use: A randomized controlled trial assessing the effect of a Web-based module on medical student performance. J Subs Abuse Treat 40(2): 203–213. UpToDate: Evidence-Based Clinical Decision Support at the Point of Care. (2013). [Accessed 4 September 2013] Available from www.uptodate. com/home. Westmoreland GR, Counsell SR, Tu W, Wu J, Litzelman DK. 2010. Webbased training in geriatrics for medical residents: A randomized controlled trial using standardized patients to assess outcomes. J Am Geriatr Soc 58(6):1163–1169.
HOW WE. . .
How we make nephrology easier to learn: Computer-based modules at the point-of-care JEFFREY H. WILLIAM & GRACE C. HUANG Beth Israel Deaconess Medical Center, Boston, MA, USA
Background: Novel educational tools, such as case-based learning in a web-based module format, are an effective approach to teaching clinical concepts to medical trainees, especially if the situations are clinically relevant and the intervention is delivered at the point-of-care. Though studies have evaluated the effectiveness of point-of-care reference materials, limited literature addresses active web-based interventions designed for completion at the point-of-care. Aims: By taking advantage of existing technological resources and integrating an effective learning modality into the clinical environment, we can increase trainee understanding of high-yield topics in clinical nephrology. Methods: We designed interactive, case-based computer-based modules in Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities, with interwoven multiple-choice and free text questions with immediate feedback, supplemental practice questions, and enrichment material to be completed in the clinical environment. All medicine trainees at an urban, academic institution were invited to participate in a needs assessment, pre and post knowledge tests, and module completion. Results: Most trainees believed the modules were ‘‘very’’ or ‘‘extremely helpful’’ in understanding the selected topic and that they would likely change their clinical practice. Those who completed the modules performed better on a post-intervention knowledge assessment. Free-text feedback was overwhelmingly supportive of the modules. Conclusion: Our findings confirmed that a novel, simplified approach to renal content by making it readily applicable to a clinical context and available at the point-of-care improves trainee understanding of high-yield topics in nephrology.
Introduction
Practice points
This generation of medical trainees lives in a technology-rich, Internet age, with a need to disseminate information in an efficient and effective manner (Horwitz & Detsky 2011). A novel educational tool, such as case-based learning in a web-based module format, has been an effective approach to teaching a variety of clinical concepts to medical trainees, especially if the situations are highly clinically relevant and the learner has a vested interest in the new material (Westmoreland et al. 2010; Lee et al. 2011; Truncali et al. 2011; Sangvai et al. 2012). Furthermore, enhancing the relevance of a topic for a medical trainee is accomplished by delivering the educational intervention at the point-of-care. Using a real clinical context to integrate otherwise ‘‘complex’’ pathophysiology into case-based learning helps trainees retain knowledge better. When offered a choice of how a teaching tool such as this should be delivered, most nephrology fellows and trainees would state that they prefer online educational material for learning (Desai et al. 2011). The interest in nephrology among medical students and residents has dropped substantially over the past decade. Despite the increasing subspecialization of Internal Medicine residents in the United States, nephrology is just one of only two medical subspecialties that have seen a declining number of fellowship applications since 2002. A recent survey of medical students exploring nephrology as a career choice
. Trainees in medicine learn more effectively through high-yield and relevant educational interventions at the point-of-care . Better understanding of subspecialty topics increases interest in the field . Novel educational tools must focus on utilizing preexisting technologies that are already relied upon by trainees . Simplifying the approach and limiting the amount of information in an educational intervention can enhance long-term learning
20 13
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Abstract
showed that they felt topics in a traditional renal pathophysiology course were ‘‘too complex [and] lacked relevance’’ (Rosner et al. 2009). While nephrology fellows report that favorable exposures to the field of nephrology and exemplary mentors influenced their decision to apply for a fellowship, many students and residents state that their own exposures to the field are minimal and frequently negative, citing the high burden of illness of dialysis patients and complicated content (McMahon et al. 2012; Jhaveri et al. 2013). As a result, the American Society of Nephrology (ASN) formed the Task Force on Increasing Interest in Nephrology Careers. Their
Correspondence: Jeffrey H. William, Division of Nephrology, Department of Medicine, Beth Israel Deaconess Medical Center, 185 Pilgrim Road, Farr 8, Boston, MA 02215, USA. Tel: (617) 632-9880; fax: (617) 632-9890; email: [email protected] ISSN 0142–159X print/ISSN 1466–187X online/14/10013–6 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2013.847912
13
J. H. William & G. C. Huang
recommendations included the development of creative educational rotations, increased use of social media, and creating novel educational tools to make nephrology more attractive to a trainee that might be considering a different field (Parker et al. 2011). To meet these challenges, educators need to take advantage of the resources we already use on a daily basis, creating new curricula that address poorly understood topics in nephrology through computer-based learning in order to adapt best teaching practices to today’s learners (Kohan 2011). Though numerous studies have evaluated the effectiveness of various existing point-of-care reference materials (Scott et al. 2000; Farrell 2008; Phua & Lim 2008; Banzi et al. 2010), there is limited literature that addresses novel active web-based interventions designed for delivery at the point-of-care.
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
What we did Our goal was to promote active learning of nephrology through the development of computer-based modules at the point-of-care that address high-yield, relevant topics in clinical nephrology, with the aim of achieving a more long-lasting understanding of these historically poorly understood clinical entities.
Module design We designed three interactive computer-based modules in Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities using an authoring tool in the Harvard Medical School (HMS) student portal website, accessible at any computer with an Internet connection. The modules were case-based, centered about a patient presentation, with embedded multiplechoice questions and multimedia, along with the opportunities to provide free text answers and to receive explanations for each answer choice. Basic concepts in pathophysiology essential to understanding the topic were interwoven within each case, with reference to an ‘‘Appendix’’ section that trainees were instructed to complete for enrichment purposes following completion of the module. For appropriate topics, we also included practice questions as supplemental material to solidify the new learning approach once the module was complete. These modules were designed to last no longer than 10 min and with the intention of completion while working in a clinical environment, though no time limit or access restrictions by location were imposed on any user.
Needs assessment We gave all trainees in the Internal Medicine Residency program at the Beth Israel Deaconess Medical Center, a large urban academic institution, the opportunity to partake in the intervention. We disseminated by e-mail a survey that constituted a needs assessment of trainees, capturing information about how trainees approach new clinical questions (i.e. admission of a patient on home peritoneal dialysis), soliciting any perceived knowledge deficiencies within nephrology, and elucidating preferred learning styles among trainees. The survey was completed by 100 Internal Medicine trainees, PGY-1 through PGY-3 (Table 1). Most trainees 14
(90%) identified UpToDateÕ , described by its authors as an ‘‘evidence-based clinical decision support resource authored by physicians to help healthcare practitioners make decisions at the point of care’’, (‘‘UpToDate’’ 2013) as their preferred resource for learning new information about a poorly understood topic. Almost 60% believed that it ‘‘almost always’’ resulted in answering their question, but 43% of respondents reported that it did not result in long-term retention of the clinical concept. Only 8% of trainees reported that they were willing to spend more than 15 min looking up a new topic while actively taking care of patients ‘‘on the wards.’’ The majority of respondents would only devote 30 min or less to learning more about a topic ‘‘off the wards’’ or at home. Related to learning styles, respondents were more likely to state that repetition and clinical application were more effective means of learning new clinical material than reading the same information from their preferred resource. They also believed that understanding the background pathophysiology was helpful in longer-term understanding of medical concepts. Specifically regarding nephrology content, nearly half of respondents believed that renal topics are more difficult to understand than topics in other specialties, citing confusing pathophysiology and difficulty in applying the concepts to clinical care. The most frequently chosen module was ‘‘Principles of Dialysis’’, completed by 39 trainees, with ‘‘Hyponatremia’’ and ‘‘Acid–Base abnormalities’’ following closely behind (Table 2). Many respondents completed the module within the hospital setting, but very rarely while actively admitting new patients. When included in the modules, the practice question and appendix sections were unanimously rated as ‘‘very’’ or ‘‘extremely helpful’’, despite usually requiring greater than 10 min to complete, within the context of the module itself. Numerous free-text comments specifically cited the helpfulness of the practice questions as well as the simplified approach to each topic (Box 1). Subjectively, most trainees (up to 96%) felt that the modules were ‘‘very’’ or ‘‘extremely helpful’’ in understanding the topic and indicated they would be likely to change their clinical practice in treating a patient with this condition. Most trainees expressed interest in developing modules like these for topics in other specialties as well (52–64%).
Evaluation: Knowledge tests and module feedback Prior to completing any modules, we asked participants to complete a challenging pre-intervention knowledge test, consisting of a case with multiple-choice questions about clinical judgment that integrated the topics of each module. Each question represented an important learning objective that would be addressed within each of the three modules. The case simulated a common clinical situation that medicine trainees are exposed to on a regular basis. As in clinical practice, many questions included multiple correct answer choices; receiving credit for a specific question required all correct choices and none of the incorrect choices, to be selected. Following each module, a post-module survey solicited data on trainee usage (i.e. time to completion, environment completed in), satisfaction, and free-text feedback for future improvement. Months after completing these
Computer-based modules at the point-of-care
Table 1. Baseline characteristics.
Year of study
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
PGY-1 Preferred resource for learning new material in clinical setting UpToDateÕ Small medical reference Non-medical search engine (i.e. Google) PubMed Time spent reading about new topic in clinical setting 1–5 min 6–10 min 11–15 min 415 min Time spent reading about new topic at home/outside of clinical setting 1–15 min 16–30 min 31–45 min 445 min I do not read more at home Subjective understanding of topic 1 ¼ "I do not understand this topic very well" 4 ¼ "I understand this topic very well" Hyponatremia (mean, SD) Principles of Dialysis (mean, SD) Acid–Base abnormalities (mean, SD) Are Renal topics more difficult to understand than other specialties? Yes No Most effective modality for learning renal topics in past Lectures in medical school or residency noon conferences Small groups (i.e. med school pathophysiology course, attending rounds) Self-learning (i.e. UpToDateÕ , PubMed, or other online modules) Individual interactions with nephrologists (i.e. elective time, one-on-one sessions, or time spent with Renal fellows consulting on your patient) Time spent on Renal consultation service 51 week 1 week 1–2 weeks 42 weeks Self-reported learning style Learn it the first time ¼ 1 vs. Require repetition ¼ 4 (mean, SD) Read and remember ¼ 1 vs. Relevant to patient care ¼ 4 (mean, SD) Remember algorithms ¼ 1 vs. Need to look up algorithms ¼ 4 (mean, SD) Work through problems individually ¼ 1 vs. with colleagues ¼ 4 (mean, SD) Learn concepts best with accompanying pathophysiology ¼ 1 vs. without pathophysiology ¼ 4 (mean, SD)
PGY-2
PGY-3
40 3 3 1
34 0 2 0
16 1 0 0
14 19 12 2
5 14 12 5
6 7 3 1
24 13 1 1 8
15 12 3 1 5
10 2 0 2 3
2.8 (0.8) 1.7 (0.6) 2.8 (0.8)
3.2 (0.5) 2.1 (0.7) 2.9 (0.6)
3.1 (0.6) 2.1 (0.7) 2.9 (0.7)
22 25
19 17
6 11
10 16 8 13
2 12 9 13
1 2 7 7
39 4 2 2
13 8 12 3
8 4 5 0
3.1 3.2 3.1 2.3 1.6
(0.8) (0.8) (0.8) (0.9) (0.7)
3.3 3.4 2.9 2.1 1.6
(0.8) (0.7) (1.0) (1.0) (0.8)
3.1 3.4 3.1 2.4 2.1
(0.8) (0.7) (1.0) (0.9) (0.9)
Baseline characteristics were reported in the needs assessment, where respondents were provided with a case of a new admitted patient on home peritoneal dialysis and asked to answer each question in the context of the case. Questions also addressed attitudes and exposure to nephrology, learning environments, and learning styles.
surveys, we sent an identical post-intervention knowledge test to all trainees. The pre and post-intervention knowledge tests were completed by 72 (46%) and 38 (24%) of trainees, respectively. There were 26 unique pairs of data, and t-tests analyses did not reveal significant differences between pre- and post-intervention knowledge tests. However, in unpaired comparisons, trainees’ performance improved, on average, by 10.6% ( p ¼ 0.05).
What’s next Our findings suggested that targeted, clinically-oriented, computer-based teaching of common topics in nephrology may result in improved understanding, but may also serve to clarify
clinical entities that are often difficult to grasp for trainees, as they self-identified in the needs assessment. Trainees identified a very high utilization of electronic resources to research new medical knowledge and apply it to their patients, but this occurs under great time pressure and with questionable longer-term learning. While actively caring for patients, our trainees did not feel they could devote more than 15 min to learning new clinical material and applying these concepts. Therefore, these computerbased modules were designed to be completed in a timesensitive clinical environment, focusing on high-yield patient presentations in nephrology, a medical specialty that a relatively large percentage of all respondents feel is more difficult to grasp than others. Though the module utilization rate was relatively low, users of the modules unanimously commented on their helpfulness and the
15
J. H. William & G. C. Huang
Table 2. Module surveys.
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Module topic Hyponatremia (n ¼ 23)
Acid–Base abnormalities (n ¼ 22)
Principles of Dialysis (n ¼ 39)
1 3 6
2 3 5
3 5 10
13 0
11 0
18 3
0 3 11 9
0 2 4 16
0 14 17 8
0 3 16 4
0 1 9 12
0 8 22 9
4
1
4
11 2 6
13 3 5
17 8 10
7 15 1 0
7 14 1 0
7 23 9 0
12 11
14 8
22 17
Where and when did you complete the module? On the wards, ICU, or nightfloat (on-call) On the wards/ICU, but not on-call In the hospital, but not on a rotation with call (i.e. elective, ambulatory block, or research) At home Other Length of time to complete module 1–5 min 6–10 min 11–15 min 415 min Subjective rating of helpfulness in understanding the topics Not helpful Somewhat helpful Very helpful Extremely helpful Preference of environment in which to complete module On the wards, ICU, or nightfloat During any elective or ambulatory time (in the hospital) Specifically before, during or after a Renal elective At home/not in the hospital Will you change your practice after completing this module? Definitely Probably Probably not Definitely not Are you willing to help design your own module in a specialty of your choice? Yes No
At the completion of each module, the respondent was asked to fill out a survey to report feedback of their experience. This survey was designed to record when and where the module was completed, how long it took, and how helpful it was. Participants were also asked to provide suggestions to improve module design by reporting their individual preferences about where a module should be completed and if they would be willing to design a similar module in another specialty.
Box 1. Selected module feedback. Principles of dialysis ‘‘This module was phenomenal, and I very much appreciated the ‘‘return to basics.’’ Often times it feels like we’re supposed to know this kind of stuff already, so being able to address it in this fashion is very helpful’’ ‘‘The most clear explanation of dialysis I’ve ever seen.’’ ‘‘Very easy to use, and the information included was pertinent to my practice.’’ ‘‘Well designed, good use of graphics, nice having ‘‘active’’ components (questions embedded)’’ Hyponatremia ‘‘Very good simple approach for dealing with a potentially very difficult topic’’ ‘‘Fantastic hyponatremia module! Learned a lot and will be using this framework more. Thank you!’’ ‘‘I enjoy how there’s personal ‘‘feedback’’ built in through the questions and ability to answer them.’’ Acid–base abnormalities ‘‘These are great! The best focused practice that I have had on acid–base status since starting residency’’ ‘‘Great practice questions to reinforce the information.’’ ‘‘Very high yield, practice centered approach’’ ‘‘Acid–base model is a great way to take a stepwise approach to a common and important medical topic.’’
likelihood that the approaches employed within each topic would likely change their future practice. Time was a key factor affecting module completion. While the modules were originally designed to be completed at the point-of-care in less than 10 min, many participants felt more comfortable completing them while off-service and took more time for completion. Though it may not be realistic to have trainees complete learning tools that require active 16
participation with integrated practice questions while concurrently admitting patients on a call night, about half of the trainees involved in this study believed they would still be most useful and reasonable to complete in a less busy clinical environment. This is consistent with the well-supported notion that adult learners acquire knowledge best when they can apply their newly acquired knowledge in real-time and see the result of their actions. Trainees especially appreciated the
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
Computer-based modules at the point-of-care
opportunity to practice and apply the simplified approaches to hyponatremia and acid–base abnormalities. Despite the extra time they required to complete a module, additional practice questions were frequently requested in user feedback. Most users also completed the appendix sections containing pathophysiologic explanations of the topic, which may have contributed to the increased amount of time it took most trainees to complete the modules (greater than 15 min), but also to longer-term knowledge acquisition. These results reflect a recent meta-analysis of a heterogeneous pool of internet-based learning interventions showing that knowledge, skills, behavioral, and satisfaction outcomes are generally favorable when compared to no intervention, but are essentially equivalent to alternative teaching methods using multimedia (Cook et al. 2008). Through a systematic review, it was shown that increased feedback and interactivity may increase the time of a learning intervention, but also strengthen knowledge outcomes (Cook et al. 2010). Medical residents consistently state that time is the biggest factor in participating in additional learning tools while in a clinical setting. With new work hour restrictions in place for interns and an increased burden on completing clinical work falling to supervising residents, all trainees are feeling more pressured for time. Any learning tools meant for the clinical setting must focus on the delicate balance of time versus effective, durable learning. There were a number of limitations to this study that may inform future similar educational interventions. To make the modules as accessible as possible to trainees, we did not require them to log into a network or system. Therefore, we could not keep track of individual user behavior (e.g. number of times accessed) or performance on the questions within the module. Instead, successful completion of a separate postmodule survey (with a link and instructions provided upon completion of the module) was our only means of confirming completion of the modules and obtaining feedback. The amount of surveys and assessments involved in this study, in addition to the time-dependent reasons stated above, may have presented an obstacle to trainees completing the modules themselves. As one respondent put it, ‘‘the preintervention knowledge assessment took longer than the module!’’ However, the design of the knowledge assessment also mirrored the clinical setting. By presenting the material in a challenging, case-based format, with integration of multiple renal topics at one time, it simulated the complex reality of patient presentations. Questions were specifically targeted toward each module topic, with the intent of making the knowledge tests challenging enough to detect an improvement in performance after the intervention. Our pre-post study design focused on knowledge gain after completing the modules and did not provide a comparison with a control group of non-users or against another form of teaching. Finally, our study occurred at a single academic institution and our findings may differ from other sites.
Conclusion Case-based learning in a computer-based module format has already been proven effective in delivering educational
content to medical trainees in a variety of clinical scenarios and across specialties. Based on the existing literature, the specific design of our computer-based modules for point-ofcare delivery is unique and was well-received by the study participants. These three modules focused on high-yield topics in nephrology (Principles of Dialysis, Hyponatremia, and Acid–Base abnormalities) and were designed to be completed at the point-of-care, incorporating active learning methods including multiple-choice and practice questions along the way to improve their clinical relevance and promote long-term knowledge acquisition. Though designed as self-learning modules, they may also be used as a teaching tool for daily rounds or provided as mandatory resources prior to the start of a nephrology elective to encourage higher level learning during their limited time on service. Given the interest of residents in designing modules like this for other high-yield medical topics, a potential library of the modules could be made available within any specialty for similar use and additional feedback can be incorporated. Though the integration of pathophysiology and clinical entities through a simplified approach is quite clear in nephrology, the use of this style of learning tool might inform teaching in other specialties in medicine. The development of educational innovations within nephrology, such as these modules, will continue to be essential in inspiring trainees to not only feel more comfortable applying these concepts to patient care but also to inform their career choice. Our findings show that simplifying the approach to renal content that is commonly perceived as confusing, increasing its applicability to a clinical context, and making it readily available at the point-of-care can improve trainee understanding of the most commonly encountered topics in nephrology.
Acknowledgements We would like to thank Dr C. Christopher Smith for his help in recruiting participants and his overall support of the work. We would also like to acknowledge Drs Stewart Lecker and Mihran Naljayan for their help in confirming the accuracy of the module content. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.
Notes on Contributors JEFFREY H. WILLIAM, M.D., is a Clinical Fellow in Nephrology in the Department of Medicine at Beth Israel Deaconess Medical Center and Harvard Medical School. GRACE C. HUANG, M.D., is an Associate Professor of Medicine at the Carl J. Shapiro Institute for Education and Research at Beth Israel Deaconess Medical Center and Harvard Medical School.
References Banzi R, Liberati A, Moschetti I, Tagliabue L. 2010. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res 12(3):1–16.
17
Med Teach Downloaded from informahealthcare.com by Selcuk Universitesi on 01/24/15 For personal use only.
J. H. William & G. C. Huang
Cook DA, Levinson AJ, Garside S. 2010. Time and learning efficiency in Internet-based learning: A systematic review and meta-analysis. Adv Health Sci Educ: Theory Practice 15(5):755–770. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. 2008. Internet-based learning in the health professions: A meta-analysis. J Am Med Assoc 300(10):1181–1196. Desai T, Stankeyeva D, Chapman A, Bailey L. 2011. Nephrology fellows show consistent use of, and improved knowledge from, a nephrologistprogrammed teaching instrument. J Nephrol 24(3):345–350. Farrell A. 2008. An evaluation of the five most used evidence based bedside information tools in Canadian health libraries. Evid Based Libr Inf Pract 3(2):3–17. Horwitz LI, Detsky AS. 2011. Physician communication in the 21st century: To talk or to text? J Am Med Assoc 305(11):1128–1129. Jhaveri KD, Sparks MA, Shah HH, Khan S, Chawla A, Desai T, Kohan DE. 2013. Why not nephrology? A survey of US internal medicine subspecialty fellows. Am J Kidney Dis 61(4):540–546. Kohan DE. 2011. Training the next generation of nephrologists. Clin J Am Soc Nephrol 6(11):2564–2566. Lee CA, Chang A, Chou CL, Boscardin C, Hauer KE. 2011. Standardized patient-narrated web-based learning modules improve students’ communication skills on a high-stakes clinical skills examination. J Gen Intern Med 26(11):1374–1377. McMahon GM, Thomas L, Tucker JK, Lin J. 2012. Factors in career choice among US nephrologists. Clin J Am Soc Nephrol 7(11):1786–1792. Parker MG, Ibrahim T, Shaffer R, Rosner MH, Molitoris BA. 2011. The future nephrology workforce: Will there be one? Clin J Am Soc Nephrol 6(6):1501–1506.
18
Phua J, Lim TK. 2008. How residents and interns utilise and perceive the personal digital assistant and UpToDate. BMC Med Educ 8:39. Rosner M, Parker M, Kohan D. (2009). Nephrology as a career choice: A survey of medical students. [Abstract]. J Am Soc Nephrol 20:767A: SA-PO2867. Sangvai S, Mahan JD, Lewis KO, Pudlo N, Suresh S, McKenzie LB. 2012. The impact of an interactive Web-based module on residents’ knowledge and clinical practice of injury prevention. Clin Pediatr (Philadelphia). 51(2):165–174. Scott I, Heyworth R, Fairweather P. 2000. The use of evidence-based medicine in the practice of consultant physicians. New Zeal J Med 30:319–326. Truncali A, Lee JD, Ark TK, Gillespie C, Triola M, Hanley K, Kalet AL. 2011. Teaching physicians to address unhealthy alcohol use: A randomized controlled trial assessing the effect of a Web-based module on medical student performance. J Subs Abuse Treat 40(2): 203–213. UpToDate: Evidence-Based Clinical Decision Support at the Point of Care. (2013). [Accessed 4 September 2013] Available from www.uptodate. com/home. Westmoreland GR, Counsell SR, Tu W, Wu J, Litzelman DK. 2010. Webbased training in geriatrics for medical residents: A randomized controlled trial using standardized patients to assess outcomes. J Am Geriatr Soc 58(6):1163–1169.
