Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
VECTOR-BORNE AND ZOONOTIC DISEASES Volume 14, Number 6, 2014 ª Mary Ann Liebert, Inc. DOI: 10.1089/vbz.2013.1305
Clinician Outreach to Improve the Quality of Rabies Postexposure Prophylaxis Administration: Maryland’s Experience, 2010–2011 Kis Robertson1 and Katherine Feldman 2
Abstract
Errors in the administration of rabies postexposure prophylaxis (PEP) are known to occur despite guidelines promulgated by the Advisory Committee on Immunization Practices (ACIP). In addition to elevating a patient’s PEP failure risk, errors in administration increase medical expenses. A record review conducted at a local health department in Maryland revealed that *8% of reported PEPs deviated from ACIP guidelines. Physicians surveyed at a local hospital also demonstrated knowledge gaps concerning use of rabies immune globulin and special indications for the five-dose PEP series. In response, we implemented an intervention to raise clinician awareness and adherence to recommended protocols. Activities included development of a patient fact sheet, hospital-based poster, and online training course targeted to a diverse group of healthcare providers. We also explored the use of hospital informatics in delivering PEP guidance. Clinician outreach by using a multipronged approach might increase adherence to recommended protocols. Key Words:
Rabies—Postexposure prophylaxis—Physicians—Medical errors—Awareness—Education.
Introduction
R
abies is an acute, progressive encephalomyelitis that develops in humans during active infection with a lyssavirus. The most epidemiologically important lyssavirus is the rabies virus, which causes an estimated 55,000 deaths annually worldwide (Knobel et al. 2005). Transmission typically occurs when infectious saliva contaminates a wound through an animal bite. Although rabies is usually fatal, postexposure prophylaxis (PEP) is highly effective in preventing the onset of this disease (World Health Organization 2005). The Advisory Committee on Immunization Practices (ACIP) promulgates recommendations for human rabies prophylaxis in the United States. In addition to describing exposure risk considerations, ACIP advises how to administer rabies PEP biologics. Recommended vaccines and dose schedules, vaccine injection sites, and use of rabies immune globulin (RIG) are described in the guidelines. Evidence indicates a need to improve the quality of rabies PEP administration in the United States. Although rare, rabies among PEP recipients is a documented phenomenon, with case management often associated with PEP that 1 2
departs from recommended protocols. Such deviations typically involve deficient wound cleansing and RIG infiltration (Fescharek et al. 1991, Wilde 2007). Despite the absence of reported PEP failures in the United States since cell culture vaccines have been marketed, nonadherence to ACIP guidelines is common. One retrospective analysis of 63 PEP recipients treated at university emergency departments (EDs) determined that 41% had records revealing instructions for RIG that conflicted with the guidelines ( Jerrard 2004). Another investigation determined that approximately 6% of patients treated in EDs did not receive PEP when indicated, whereas 40% of PEPs were administered unnecessarily (Moran et al. 2000). Other reports attest to similar findings (Harrigan 1996, Auslander and Kaelin 1997, Pearson 2000, Conti et al. 2002). Excessive medical costs are another consequence of misadministered PEP. Clinicians commonly administer extra doses of biologics when correcting PEP errors—a strategy that produces additional prescription expenses, hospital fees, and numerous indirect costs (Shwiff et al. 2007). Testing to verify seroconversion is a more economical alternative; however, even this option consumes resources (Sterner and Sun 2004). The overuse of medical services also warrants
U.S. Department of Agriculture, Washington, DC. Maryland Department of Health and Mental Hygiene, Baltimore, Maryland.
454
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
RABIES CLINICIAN OUTREACH IN MARYLAND
acknowledgment as a problem arising from indiscriminate rabies exposure risk assessments (Krebs et al. 1998). Analysis has shown that when PEP is administered in situations with < 0.7% probability of rabies transmission, relative costs begin to exceed the value of human lives potentially saved (Dhankar et al. 2008). Thus, nonjudicious use adversely impacts PEP cost effectiveness on a societal level. The Maryland Department of Health and Mental Hygiene (DHMH) identified PEP misadministration as a concern worth addressing through an educational campaign. The following report describes the activities that guided and comprised this effort. Materials and Methods Needs assessment
Prior to starting the campaign, we conducted a needs assessment to investigate PEP misadminstration in Maryland. For the first phase, we reviewed PEP records reported during 2009 to the Montgomery County Department of Health and Human Services (MDHHS) and described instances in which PEP was given in a manner discrepant with ACIP protocol. MDHHS was selected because of anecdotal evidence indicating patterns of PEP errors in the county. Records examined included forms forwarded by providers detailing a recipient’s exposure source, dose volumes, and administration dates. Health department notes and medical records were also reviewed when available. These documents were useful in establishing whether PEP conformed to ACIP guidelines because they included details pertaining to injection sites, RIG infiltration, and the circumstances of the exposure. For the purposes of this review, PEP was considered misadministered if records indicated a biologic had been administered in contravention with ACIP, regardless of patient outcome. Deviations from the recommended vaccination schedule were not considered misadministrations because of the difficulty in determining whether these occurrences resulted from clinician error rather than patient noncompliance. For the second part of our needs assessment, we asked infectious disease fellows and faculty at a local university teaching hospital to complete a knowledge-based questionnaire both before and after receiving a lecture about rabies. The 1-h lecture, which was presented by a rabies subject matter expert from DHMH, featured case-based presentations explaining rabies exposure risk considerations and PEP administration. The identical pretests and posttests comprised 10 multiple choice questions about sources of human rabies, relevant considerations when assessing rabies risk, indications for RIG use, acceptable vaccine injection sites, and indications for a five-dose PEP vaccine regimen. Test takers represented a convenience sample of 10 out of approximately 40 attendees. Performance on the pretest was considered an indicator of baseline knowledge. Average preand posttest scores were compared using a nonpaired t-test to evaluate lecture effectiveness in raising awareness. Intervention strategy
System-based models for preventing medical errors have gained prominence during recent years, as public attention regarding health care quality has increased (Kryriacou and Coben 2000, Woodward et al. 2010). Human, environmental,
455
and institutional factors interact in the health care setting, and although each can contribute to system failures, each factor can also be viewed as a potential vehicle for prevention. Associated with the concept of interdependence is the idea that safeguards can and should be enacted well in advance of the point at which medical care is delivered (Reason 2000). Positioning interventions at multiple nodes in a system also creates a failsafe through redundancy. To accomplish our goal in this intervention, we targeted nodes in the point-of-care system that we considered to be meaningful control points in the prevention of rabies PEP misadministration. These nodes included patients, the clinic environment, hospital informatics, and health care providers. Activities incorporated into our education campaign included development of a patient fact sheet, hospital-based poster, and online training module for clinicians. We also explored delivery of PEP guidance using computerized physician order entry (CPOE) systems. CPOEs were identified as a potential tool in our intervention because these systems are positioned to provide pharmacy guidance after the decision has been made to administer PEP. Moreover, a CPOE can minimize clinician guesswork by generating patient-tailored instructions. To explore the feasibility of using this technology as a conduit for PEP guidance, we contacted pharmacists at five hospitals in Montgomery County and inquired about the use of CPOEs in their EDs. For the online training course, we collaborated with the Centers for Disease Control and Prevention (CDC) to develop a module accessible at no cost through the Internet and incentivized with continuing education (CE) credits for physicians, nurses, pharmacists, and veterinarians. The 2008 and 2010 ACIP guidelines on human rabies prevention, the 2010 CDC Yellow Book, and the Compendium on Animal Rabies Prevention and Control were sources from which course content was derived (Centers for Disease Control and Prevention 2008, 2009, 2010, National Association of State Public Health Veterinarians 2011). Case-based instruction was pursued because it was considered the most efficient way to increase competency in patient evaluation for rabies exposure (Drexel et al. 2011, Tamler et al. 2011). While the course was in development, we also delivered in-service trainings to approximately 70 ED clinicians from three hospitals in Montgomery County, and used these sessions as an opportunity to learn more about knowledge gaps among our target population. ED clinicians and selected state public health veterinarians from across the country were asked to review a draft version of the poster. Feedback elicited from these groups was incorporated into the poster before completion. Results Needs assessment
Among 100 PEP records reviewed, four (4%) exhibited evidence of PEP misadministration. Of the four PEP recipients associated with these records, two (50%) received vaccine injected in the gluteus (instead of the deltoid as recommended by ACIP); one (25%) did not receive RIG during PEP initiation (the record indicated that the physician believed that it was unnecessary and had to be advised by MDHHS to administer RIG); and one (25%) had a wound that was not infiltrated with RIG. Records also indicated that two of the four patients had received a corrective dose of vaccine, with
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
456
ROBERTSON AND FELDMAN
postvaccination serologic testing also performed for one of these patients. In addition to these reports, records indicated that PEP had been initiated in four patients, although the biting animals were dogs available for testing or observation and presented a low rabies risk. On the basis of these findings, we determined that PEP administered either incorrectly and unnecessarily occurred on a relatively frequent basis, possibly involving 8% of all PEP administrations in Montgomery County. Misuse or inappropriate withholding of RIG and misplacement of rabies vaccine were both observed, indicating that inadequacies in clinician practices extended to multiple PEP components. The frequency of suboptimal PEP is likely to be substantially > 8% because documentation that would permit detection of ACIP deviations was inconsistently available among PEP records that we reviewed. Results of the pre- and posttraining tests of infectious disease physicians to assess rabies awareness are reported in Table 1. Before the lecture, respondents on average answered only 43% of pretest questions correctly. Weakest performance (with no respondents answering correctly) was observed on question 8, which required knowledge of correct RIG and vaccine use. After the lecture, the average posttest score was 74.4%, constituting a 73% improvement, compared with pretest performance (95% confidence interval [CI] = 40.0–106.0%; p = 0.0002). In contrast with what had been observed regarding the pretest, approximately half (56%) of the respondents correctly answered question 8 on the posttest. Question 5 demonstrated the largest improvement in performance; only 10% of pretest responses correctly identified a situation in which RIG is inadvisable (i.e., when a patient has been previously vaccinated against rabies), compared with 80% of posttest responses. From these results, we reached the following conclusions: Substantive knowledge gaps appeared to exist among the medical community regarding rabies, and case-based instruction was effective in raising clinician awareness of ACIP-recommended practices, at least on a short-term basis. Implications for disease prevention and the economical use
of PEP were also inferred. Our findings were determined by a small, nonrepresentative sample of physicians, which limits the generalizability of our conclusions to the population. In addition, given the small convenience sample (with one loss-to-followup), it is also possible that respondents differed significantly from unsurveyed attendees with respect to knowledge. Thus, our results may have under- or overestimated true knowledge within this group. Despite these limitations, we considered our observations as evidence for the need to educate healthcare providers about rabies PEP. Patient fact sheet
To leverage patient knowledge in our intervention, we developed a patient fact sheet focused on rabies PEP, with the intended audience being persons seeking health information prior to medical visitation for possible rabies virus exposure. The fact sheet informs patients of what to expect when receiving rabies PEP, so that the patient has the awareness necessary to intervene if (and before) RIG or rabies vaccine are given in an incorrect manner. Content stresses how RIG should be administered into wounds, how much RIG should be given, where anatomically rabies vaccine should be injected, and probably most importantly, where it should not be injected. With this focus on the procedural aspects of PEP, our fact sheet differs from the rabies-specific Vaccine Information Sheet published by the CDC, which provides a more general overview of rabies PEP. Given how frequently the internet is used by the public for health information (Diaz et al. 2002), posting the fact sheet online was seen as the best means of connecting Maryland patients to PEP guidance (http://phpa.dhmh.maryland.gov/ OIDEOR/CZVBD/Shared%20Documents/Rabies_PEP_Fact sheet.pdf). The electronic format of the fact sheet also lends itself to e-mail transmission, which is a particularly useful communication method when residents contact the state or local health department with questions about PEP. Because ease of comprehension is another determinant of accessibility,
Table 1. Knowledge Demonstrated by Clinicians at a Local University Hospital before and after a Lecture on Rabies (n = 10) No.(%) respondents exhibiting knowledge Question no. 1 2 3 4 5 6 7 8 9 10
Knowledge indicator
Pretest
Posttest*
Bats are a leading source of human rabies in the United States. Apparent health of a bat is least relevant when assessing rabies risk when compared with other considerations. Dog rabies is rare in the United States Exposure circumstances, dog’s vaccination status, local animal epidemiology, and apparent health of a dog are all relevant when assessing rabies risk. RIG should not be administered to previously vaccinated patients. Wound severity is not a relevant factor when deciding to infiltrate with RIG. The deltoid is the only acceptable site for rabies vaccine injection in adults. Wounds should be infiltrated with RIG, with vaccine given distant from RIG but not in the gluteus. RIG can be administered £ 7 days after PEP initiation HIV-positive patients should receive a 5-dose PEP vaccination course.
6 (60) 4 (40)
6 (67) 8 (89)
9 (90) 8 (80)
8 (89) 9 (100)
1 (10) 3 (30) 1 (1) 0
8 5 5 5
9 (90) 2 (20)
7 (78) 7 (78)
RIG, rabies immune globulin; PEP, post exposure prophylaxis; HIV, human immunodeficiency virus. *Sample size for post-test was 9 because one respondent was lost to follow-up.
(89) (56) (56) (56)
RABIES CLINICIAN OUTREACH IN MARYLAND
plain language and basic, nonmedical terms are used in the fact sheet as often as possible to minimize barriers to understanding that can alienate patients with limited health literacy.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
Poster for health care providers
Figure 1 illustrates the poster that was developed for Maryland health care facilities where rabies PEP is routinely
457
administered. A description of rabies PEP, indications for its use, and recommended PEP regimens as determined by the patient’s vaccination history are summarized in the poster. Additionally, the poster lists important do’s and don’ts when administering PEP. By drawing the reader’s attention to these instructions, we sought to promote clinician adherence to recommended protocols and awareness of problematic practices.
FIG. 1. Poster for Maryland health care providers summarizing rabies postexposure prophylaxis guidelines. (Reprinted with permission from the Maryland Department of Health and Mental Hygiene.)
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
458
The poster was mailed or delivered to EDs and urgent care centers statewide; the rabies coordinators in all 24 of Maryland’s local health departments (LHDs) distributed the poster to the facilities in their jurisdiction. The poster is also available on the DHMH Internet site to encourage further dissemination both within and outside Maryland (http://phpa.dhmh.mary land.gov/OIDEOR/CZVBD/Shared%20Documents/Rabies% 20poster%20new%202.pdf). PEP guidance by using hospital informatics
When pharmacists at five hospitals in Montgomery County were contacted and asked about the use of CPOEs in their EDs, four facilities indicated using such a system (the single differing hospital reporting that although a CPOE was used elsewhere in the hospital, the ED relied on an alternative ordering method). Among four EDs with CPOEs in use, none was programmed to provide rabies PEP guidance. Pharmacists at three of these EDs indicated that as members of larger hospital networks, their facility lacked privileges that would permit modifying their system locally. We concluded that for these three sites, piloting the use of CPOEs in delivering rabies PEP guidance might be hindered by organizational or institutional barriers; therefore, further engagement on this topic was not pursued. Of five EDs contacted, only one ED reported having administrator privileges to their CPOE system. When contacted by DHMH, information technology (IT) pharmacists at this hospital (hospital A) expressed interest in the idea of programming their system with rabies PEP guidance. They subsequently agreed to collaborate with DHMH in designing a suitable algorithm.
ROBERTSON AND FELDMAN
Figure 2 illustrates the conceptual framework for the algorithm that was developed. After an order for rabies PEP is initiated by the clinician through CPOE, the algorithm, as conceived, would prompt the clinician to answer questions about the patient. On the basis of how the prompts are answered, the resulting instructions would inform the clinician how to administer rabies biologics. These instructions would also alert the clinician to practices that should be avoided and further reinforce adherence to recommended protocols. Approximately 2 months after this discussion, hospital A launched a new CPOE for ED use. Initially, it was hoped that this upgrade would allow IT pharmacists to program PEP decision support into the new system; however, attempts to accomplish this have been unsuccessful. The main stumbling block identified by the hospital is that despite having administrator privileges, the new system is not amenable to extensive programming changes because it uses preconfigured templates as a means of simplifying the ordering process. Clinician training
In addition to providing a basic overview on rabies, the online course we developed presented five case scenarios that were designed to simulate situations routinely confronted by ED clinicians and public health practitioners. Topics expounded upon included the epidemiology of rabies in Maryland and the United States, laboratory methods used for rabies diagnosis, and the strategy of conducting 10-day observation to rule out rabies exposure from dogs, cats, and ferrets. The mechanics of administering rabies PEP were also covered in detail. Attention was given to explaining the functions of RIG
FIG. 2. Conceptual framework for an algorithm describing rabies postexposure prophylaxis (PEP). RIG, rabies immune globulin; IM, intramuscularly.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
RABIES CLINICIAN OUTREACH IN MARYLAND
and rabies vaccine and relating these functions to recommended practices. Indications for the five-dose PEP vaccine regimen, the do’s and don’ts of PEP administration, and appropriate actions to take in response to a PEP error were among other concepts discussed. After the course was approved by CDC for CE credits and posted to the DHMH rabies Internet site, it was publicly released (http://phpa.dhmh.maryland.gov/training/SitePages/ rabies.aspx) and promoted through multiple outlets, including CDC’s Morbidity and Mortality Weekly Report, professional listservs, news bulletins, social media, and nonprofit organizations. Among the approximately 2300 participants who have provided evaluation feedback as of the date of this report, 94% either agreed (46%) or strongly agreed (48%) the course addressed a gap in their knowledge or skills. In addition, 98% either agreed (40%) or strongly agreed (58%) that they were capable of applying knowledge gained from taking the course to their work. Discussion
Through the experience of implementing this intervention, we acquired a greater understanding of the problems and opportunities that influence educational outreach to the medical community. Rabies poses a special set of challenges. In the United States, human exposure potential to the rabies virus is ample because of the burden of disease in wildlife, yet only one to three human cases are reported annually (Blanton 2011). The cause of this seeming incongruence is undetermined; however, one effect it might have on certain providers is a limited appreciation for aligning rabies PEP administration with established recommendations. A multipronged strategy aimed at keeping clinicians informed about rabies might be useful in combating knowledge gaps that often occur with rare diseases. In this intervention, we attempted to use IT as a means of providing PEP instructions to clinicians, but our efforts were impeded by structural barriers associated with the technology. Although CPOEs were in use in almost all EDs that we surveyed, these facilities were limited in their ability to program additional decision support into their systems. Nevertheless, the positive interest expressed by hospital administrators at one facility indicates that future efforts might be successful if facilities that use customizable systems are engaged. Workaround strategies that bypass existing hurdles might also be a way to deliver guidance in an acceptable manner. An important catalyst for CPOE adoption is federally funded incentives that come attached to the meaningful use of electronic health records (Blummenthal 2009, Centers for Medicare & Medicaid Services 2011). As more hospitals use CPOEs to prevent medical errors, conditions might encourage the full potential of these systems to be actualized. The next step in this project is evaluating the effectiveness of our intervention. Although our goal was to improve the quality of rabies PEP administration in Maryland, available methods for measuring this outcome have limitations, the most important being the lack of sensitivity. Despite the fact that rabies PEP is reportable by Maryland LHDs, PEP that is administered without the knowledge of LHDs might go unrecognized, and providers are not required to disclose information that permits detection of ACIP deviations. Thus,
459
changes in clinician competency measured as a proxy for changes in practice might be a warranted approach. Acknowledgments
We thank Cindy Edwards from MDDHS, and Sara Critchley, Paul Melstrom, and Sergio Recuenco (content expert) from CDC for serving on the training course planning committee, as well as Karen Michaels from Suburban Hospital–Johns Hopkins Medicine, David Blythe, and Lucy Wilson from DHMH, and Charles Rupprecht from CDC for serving as course reviewers. Author Disclosure Statement
The authors have no competing financial interests. References
Auslander M, Kaelin C. Rabies postexposure prophylaxis survey—Kentucky, 1994. Emerg Infect Dis 1997; 3:199–202. Blanton JD, Palmer D, Dyer J, Rupprecht CE. Rabies surveillance in the United States during 2010. J Am Vet Med Assoc 2011; 239:773–783. Blummenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009; 360:1477–1479. Centers for Disease Control and Prevention. Human rabies prevention—United States, 2008: Recommendations of the Advisory Committee on Immunization Practices. MMWR Recommend Rep 2008; 57(No. RR-3):1–28. Centers for Disease Control and Prevention. Health information for international travel 2010. Atlanta, GA: US Department of Health and Human Services, Public Health Service; 2009. Available at http://wwwnc.cdc.gov/travel/page/yellowbook2012-home.htm Accessed August 22, 2012. Centers for Disease Control and Prevention. Use of a reduced (4-dose) vaccine schedule for postexposure prophylaxis to prevent human rabies: Recommendations of the Advisory Committee on Immunization Practices. MMWR Recommend Rep 2010; 59(No. RR-2):1–12. Centers for Medicare & Medicaid Services, US Department of Health and Human Services. Electronic health record incentive programs. Baltimore, MD: US Department of Health and Human Services, Centers for Medicare & Medicaid Services. 2011. Available at https://www.cms.gov/ehrincentiveprograms Accessed August 22, 2012. Conti L, Wiersma R, Hopkins R. Evaluation of state-provided postexposure prophylaxis against rabies in Florida. South Med J 2002; 95:225–230. Dhankar P, Vaidya SA, Fishbien DB, Meltzer MI. Cost effectiveness of rabies post exposure prophylaxis in the United States. Vaccine 2008; 5:4251–4255. Diaz JA, Griffith RA, Ng JJ, Reinert SE, et al. Patients’ use of the Internet for medical information. J Gen Intern Med 2002; 17:180–185. Drexel C, Jacobson A, Hanania NA, Whitfeld B, et al. Measuring the impact of a live, case-based, multiformat, interactive continuing medical education program on improving clinician knowledge and competency in evidence-based COPD care. Int J Chron Obstruct Pulmon Dis 2011; 6: 297–307. Fescharek R, Schwarz S, Quast U, Gandhi N, et al. Postexposure rabies prophylaxis: When the guidelines are not respected. Vaccine 1991; 9:868–872. Gautret P, Parola P. Rabies vaccination for international travelers. Vaccine 2012; 30:126–133.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
460
Harrigan RA, Kauffman FH. Postexposure rabies prophylaxis in an urban emergency department. J Emerg Med 1996; 14: 287–292. Jerrard DA. The use of rabies immune globulin by emergency physicians. J Emerg Med 2004; 27:15–19. Knobel DL, Cleaveland S, Coleman PG, Fevre EM, et al. Reevaluating the burden of rabies in Africa and Asia. Bull World Health Organ 2005; 83:360–368. Krebs JW, Long-Marin SC, Childs JE. Causes, costs, and estimates of rabies postexposure prophylaxis treatments in the United States. J Public Health Manag Pract 1998; 4:56–62. Kyriacou DN, Coben JH. Errors in emergency medicine: Research strategies. Acad Emerg Med 2000; 7:1201–1203. Moran GJ, Talan DA, Mower W, Newdow M, et al. Appropriateness of rabies postexposure prophylaxis in treatment for animal exposures: Emergency ID Net Study Group. JAMA 2000; 284:1001–1007. Pearson L. Medical errors hit home. Nurse Pract 2000; 25:11–13. National Association of State Public Health Veterinarians. Compendium of animal rabies prevention and control, 2011. MMWR Recommend Rep 2011; 60(No. RR-6):1–17. Reason J. Human error: Models and management. Br Med J 2000; 320:768–770. Shwiff SA, Sterner RT, Jay MT, Parikh S, et al. Direct and indirect costs of rabies exposure: A retrospective study in southern California (1998–2002). J Wildlife Dis 2007; 43:251–257.
ROBERTSON AND FELDMAN
Sterner RT, Sun B. Relative factor costs of wildlife rabies impacts in the U.S. USDA National Wildlife Research Center– Staff Publications. 2004. Available at http://digitalcommons .unl.edu/cgi/viewcontent.cgi?article = 1385&context = icwdm_ usdanwrc Accessed August 22, 2012. Tamler R, Green D, Skamagas M, Breen TL, et al. Effect of case-based training for medical residents on inpatient glycemia. Diabetes Care 2011; 31:1738–1740. Wilde H. Failures of post-exposure rabies prophylaxis. Vaccine 2007; 25:7605–7609. Woodward HI, Mytton OT, Lemer C, Yardley IE, et al. What have we learned about interventions to reduce medical errors? Annu Rev Public Health 2010; 21:479–497. World Health Organization. WHO expert consultation on rabies. World Health Organ Tech Ser 2005; 931:1–88.
Address correspondence to: Kis Robertson U.S. Department of Agriculture 1400 Independence Avenue, SW Stop 3766, PP 3, 9-210B Washington, DC 20250 E-mail: [email protected]
VECTOR-BORNE AND ZOONOTIC DISEASES Volume 14, Number 6, 2014 ª Mary Ann Liebert, Inc. DOI: 10.1089/vbz.2013.1305
Clinician Outreach to Improve the Quality of Rabies Postexposure Prophylaxis Administration: Maryland’s Experience, 2010–2011 Kis Robertson1 and Katherine Feldman 2
Abstract
Errors in the administration of rabies postexposure prophylaxis (PEP) are known to occur despite guidelines promulgated by the Advisory Committee on Immunization Practices (ACIP). In addition to elevating a patient’s PEP failure risk, errors in administration increase medical expenses. A record review conducted at a local health department in Maryland revealed that *8% of reported PEPs deviated from ACIP guidelines. Physicians surveyed at a local hospital also demonstrated knowledge gaps concerning use of rabies immune globulin and special indications for the five-dose PEP series. In response, we implemented an intervention to raise clinician awareness and adherence to recommended protocols. Activities included development of a patient fact sheet, hospital-based poster, and online training course targeted to a diverse group of healthcare providers. We also explored the use of hospital informatics in delivering PEP guidance. Clinician outreach by using a multipronged approach might increase adherence to recommended protocols. Key Words:
Rabies—Postexposure prophylaxis—Physicians—Medical errors—Awareness—Education.
Introduction
R
abies is an acute, progressive encephalomyelitis that develops in humans during active infection with a lyssavirus. The most epidemiologically important lyssavirus is the rabies virus, which causes an estimated 55,000 deaths annually worldwide (Knobel et al. 2005). Transmission typically occurs when infectious saliva contaminates a wound through an animal bite. Although rabies is usually fatal, postexposure prophylaxis (PEP) is highly effective in preventing the onset of this disease (World Health Organization 2005). The Advisory Committee on Immunization Practices (ACIP) promulgates recommendations for human rabies prophylaxis in the United States. In addition to describing exposure risk considerations, ACIP advises how to administer rabies PEP biologics. Recommended vaccines and dose schedules, vaccine injection sites, and use of rabies immune globulin (RIG) are described in the guidelines. Evidence indicates a need to improve the quality of rabies PEP administration in the United States. Although rare, rabies among PEP recipients is a documented phenomenon, with case management often associated with PEP that 1 2
departs from recommended protocols. Such deviations typically involve deficient wound cleansing and RIG infiltration (Fescharek et al. 1991, Wilde 2007). Despite the absence of reported PEP failures in the United States since cell culture vaccines have been marketed, nonadherence to ACIP guidelines is common. One retrospective analysis of 63 PEP recipients treated at university emergency departments (EDs) determined that 41% had records revealing instructions for RIG that conflicted with the guidelines ( Jerrard 2004). Another investigation determined that approximately 6% of patients treated in EDs did not receive PEP when indicated, whereas 40% of PEPs were administered unnecessarily (Moran et al. 2000). Other reports attest to similar findings (Harrigan 1996, Auslander and Kaelin 1997, Pearson 2000, Conti et al. 2002). Excessive medical costs are another consequence of misadministered PEP. Clinicians commonly administer extra doses of biologics when correcting PEP errors—a strategy that produces additional prescription expenses, hospital fees, and numerous indirect costs (Shwiff et al. 2007). Testing to verify seroconversion is a more economical alternative; however, even this option consumes resources (Sterner and Sun 2004). The overuse of medical services also warrants
U.S. Department of Agriculture, Washington, DC. Maryland Department of Health and Mental Hygiene, Baltimore, Maryland.
454
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
RABIES CLINICIAN OUTREACH IN MARYLAND
acknowledgment as a problem arising from indiscriminate rabies exposure risk assessments (Krebs et al. 1998). Analysis has shown that when PEP is administered in situations with < 0.7% probability of rabies transmission, relative costs begin to exceed the value of human lives potentially saved (Dhankar et al. 2008). Thus, nonjudicious use adversely impacts PEP cost effectiveness on a societal level. The Maryland Department of Health and Mental Hygiene (DHMH) identified PEP misadministration as a concern worth addressing through an educational campaign. The following report describes the activities that guided and comprised this effort. Materials and Methods Needs assessment
Prior to starting the campaign, we conducted a needs assessment to investigate PEP misadminstration in Maryland. For the first phase, we reviewed PEP records reported during 2009 to the Montgomery County Department of Health and Human Services (MDHHS) and described instances in which PEP was given in a manner discrepant with ACIP protocol. MDHHS was selected because of anecdotal evidence indicating patterns of PEP errors in the county. Records examined included forms forwarded by providers detailing a recipient’s exposure source, dose volumes, and administration dates. Health department notes and medical records were also reviewed when available. These documents were useful in establishing whether PEP conformed to ACIP guidelines because they included details pertaining to injection sites, RIG infiltration, and the circumstances of the exposure. For the purposes of this review, PEP was considered misadministered if records indicated a biologic had been administered in contravention with ACIP, regardless of patient outcome. Deviations from the recommended vaccination schedule were not considered misadministrations because of the difficulty in determining whether these occurrences resulted from clinician error rather than patient noncompliance. For the second part of our needs assessment, we asked infectious disease fellows and faculty at a local university teaching hospital to complete a knowledge-based questionnaire both before and after receiving a lecture about rabies. The 1-h lecture, which was presented by a rabies subject matter expert from DHMH, featured case-based presentations explaining rabies exposure risk considerations and PEP administration. The identical pretests and posttests comprised 10 multiple choice questions about sources of human rabies, relevant considerations when assessing rabies risk, indications for RIG use, acceptable vaccine injection sites, and indications for a five-dose PEP vaccine regimen. Test takers represented a convenience sample of 10 out of approximately 40 attendees. Performance on the pretest was considered an indicator of baseline knowledge. Average preand posttest scores were compared using a nonpaired t-test to evaluate lecture effectiveness in raising awareness. Intervention strategy
System-based models for preventing medical errors have gained prominence during recent years, as public attention regarding health care quality has increased (Kryriacou and Coben 2000, Woodward et al. 2010). Human, environmental,
455
and institutional factors interact in the health care setting, and although each can contribute to system failures, each factor can also be viewed as a potential vehicle for prevention. Associated with the concept of interdependence is the idea that safeguards can and should be enacted well in advance of the point at which medical care is delivered (Reason 2000). Positioning interventions at multiple nodes in a system also creates a failsafe through redundancy. To accomplish our goal in this intervention, we targeted nodes in the point-of-care system that we considered to be meaningful control points in the prevention of rabies PEP misadministration. These nodes included patients, the clinic environment, hospital informatics, and health care providers. Activities incorporated into our education campaign included development of a patient fact sheet, hospital-based poster, and online training module for clinicians. We also explored delivery of PEP guidance using computerized physician order entry (CPOE) systems. CPOEs were identified as a potential tool in our intervention because these systems are positioned to provide pharmacy guidance after the decision has been made to administer PEP. Moreover, a CPOE can minimize clinician guesswork by generating patient-tailored instructions. To explore the feasibility of using this technology as a conduit for PEP guidance, we contacted pharmacists at five hospitals in Montgomery County and inquired about the use of CPOEs in their EDs. For the online training course, we collaborated with the Centers for Disease Control and Prevention (CDC) to develop a module accessible at no cost through the Internet and incentivized with continuing education (CE) credits for physicians, nurses, pharmacists, and veterinarians. The 2008 and 2010 ACIP guidelines on human rabies prevention, the 2010 CDC Yellow Book, and the Compendium on Animal Rabies Prevention and Control were sources from which course content was derived (Centers for Disease Control and Prevention 2008, 2009, 2010, National Association of State Public Health Veterinarians 2011). Case-based instruction was pursued because it was considered the most efficient way to increase competency in patient evaluation for rabies exposure (Drexel et al. 2011, Tamler et al. 2011). While the course was in development, we also delivered in-service trainings to approximately 70 ED clinicians from three hospitals in Montgomery County, and used these sessions as an opportunity to learn more about knowledge gaps among our target population. ED clinicians and selected state public health veterinarians from across the country were asked to review a draft version of the poster. Feedback elicited from these groups was incorporated into the poster before completion. Results Needs assessment
Among 100 PEP records reviewed, four (4%) exhibited evidence of PEP misadministration. Of the four PEP recipients associated with these records, two (50%) received vaccine injected in the gluteus (instead of the deltoid as recommended by ACIP); one (25%) did not receive RIG during PEP initiation (the record indicated that the physician believed that it was unnecessary and had to be advised by MDHHS to administer RIG); and one (25%) had a wound that was not infiltrated with RIG. Records also indicated that two of the four patients had received a corrective dose of vaccine, with
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
456
ROBERTSON AND FELDMAN
postvaccination serologic testing also performed for one of these patients. In addition to these reports, records indicated that PEP had been initiated in four patients, although the biting animals were dogs available for testing or observation and presented a low rabies risk. On the basis of these findings, we determined that PEP administered either incorrectly and unnecessarily occurred on a relatively frequent basis, possibly involving 8% of all PEP administrations in Montgomery County. Misuse or inappropriate withholding of RIG and misplacement of rabies vaccine were both observed, indicating that inadequacies in clinician practices extended to multiple PEP components. The frequency of suboptimal PEP is likely to be substantially > 8% because documentation that would permit detection of ACIP deviations was inconsistently available among PEP records that we reviewed. Results of the pre- and posttraining tests of infectious disease physicians to assess rabies awareness are reported in Table 1. Before the lecture, respondents on average answered only 43% of pretest questions correctly. Weakest performance (with no respondents answering correctly) was observed on question 8, which required knowledge of correct RIG and vaccine use. After the lecture, the average posttest score was 74.4%, constituting a 73% improvement, compared with pretest performance (95% confidence interval [CI] = 40.0–106.0%; p = 0.0002). In contrast with what had been observed regarding the pretest, approximately half (56%) of the respondents correctly answered question 8 on the posttest. Question 5 demonstrated the largest improvement in performance; only 10% of pretest responses correctly identified a situation in which RIG is inadvisable (i.e., when a patient has been previously vaccinated against rabies), compared with 80% of posttest responses. From these results, we reached the following conclusions: Substantive knowledge gaps appeared to exist among the medical community regarding rabies, and case-based instruction was effective in raising clinician awareness of ACIP-recommended practices, at least on a short-term basis. Implications for disease prevention and the economical use
of PEP were also inferred. Our findings were determined by a small, nonrepresentative sample of physicians, which limits the generalizability of our conclusions to the population. In addition, given the small convenience sample (with one loss-to-followup), it is also possible that respondents differed significantly from unsurveyed attendees with respect to knowledge. Thus, our results may have under- or overestimated true knowledge within this group. Despite these limitations, we considered our observations as evidence for the need to educate healthcare providers about rabies PEP. Patient fact sheet
To leverage patient knowledge in our intervention, we developed a patient fact sheet focused on rabies PEP, with the intended audience being persons seeking health information prior to medical visitation for possible rabies virus exposure. The fact sheet informs patients of what to expect when receiving rabies PEP, so that the patient has the awareness necessary to intervene if (and before) RIG or rabies vaccine are given in an incorrect manner. Content stresses how RIG should be administered into wounds, how much RIG should be given, where anatomically rabies vaccine should be injected, and probably most importantly, where it should not be injected. With this focus on the procedural aspects of PEP, our fact sheet differs from the rabies-specific Vaccine Information Sheet published by the CDC, which provides a more general overview of rabies PEP. Given how frequently the internet is used by the public for health information (Diaz et al. 2002), posting the fact sheet online was seen as the best means of connecting Maryland patients to PEP guidance (http://phpa.dhmh.maryland.gov/ OIDEOR/CZVBD/Shared%20Documents/Rabies_PEP_Fact sheet.pdf). The electronic format of the fact sheet also lends itself to e-mail transmission, which is a particularly useful communication method when residents contact the state or local health department with questions about PEP. Because ease of comprehension is another determinant of accessibility,
Table 1. Knowledge Demonstrated by Clinicians at a Local University Hospital before and after a Lecture on Rabies (n = 10) No.(%) respondents exhibiting knowledge Question no. 1 2 3 4 5 6 7 8 9 10
Knowledge indicator
Pretest
Posttest*
Bats are a leading source of human rabies in the United States. Apparent health of a bat is least relevant when assessing rabies risk when compared with other considerations. Dog rabies is rare in the United States Exposure circumstances, dog’s vaccination status, local animal epidemiology, and apparent health of a dog are all relevant when assessing rabies risk. RIG should not be administered to previously vaccinated patients. Wound severity is not a relevant factor when deciding to infiltrate with RIG. The deltoid is the only acceptable site for rabies vaccine injection in adults. Wounds should be infiltrated with RIG, with vaccine given distant from RIG but not in the gluteus. RIG can be administered £ 7 days after PEP initiation HIV-positive patients should receive a 5-dose PEP vaccination course.
6 (60) 4 (40)
6 (67) 8 (89)
9 (90) 8 (80)
8 (89) 9 (100)
1 (10) 3 (30) 1 (1) 0
8 5 5 5
9 (90) 2 (20)
7 (78) 7 (78)
RIG, rabies immune globulin; PEP, post exposure prophylaxis; HIV, human immunodeficiency virus. *Sample size for post-test was 9 because one respondent was lost to follow-up.
(89) (56) (56) (56)
RABIES CLINICIAN OUTREACH IN MARYLAND
plain language and basic, nonmedical terms are used in the fact sheet as often as possible to minimize barriers to understanding that can alienate patients with limited health literacy.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
Poster for health care providers
Figure 1 illustrates the poster that was developed for Maryland health care facilities where rabies PEP is routinely
457
administered. A description of rabies PEP, indications for its use, and recommended PEP regimens as determined by the patient’s vaccination history are summarized in the poster. Additionally, the poster lists important do’s and don’ts when administering PEP. By drawing the reader’s attention to these instructions, we sought to promote clinician adherence to recommended protocols and awareness of problematic practices.
FIG. 1. Poster for Maryland health care providers summarizing rabies postexposure prophylaxis guidelines. (Reprinted with permission from the Maryland Department of Health and Mental Hygiene.)
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
458
The poster was mailed or delivered to EDs and urgent care centers statewide; the rabies coordinators in all 24 of Maryland’s local health departments (LHDs) distributed the poster to the facilities in their jurisdiction. The poster is also available on the DHMH Internet site to encourage further dissemination both within and outside Maryland (http://phpa.dhmh.mary land.gov/OIDEOR/CZVBD/Shared%20Documents/Rabies% 20poster%20new%202.pdf). PEP guidance by using hospital informatics
When pharmacists at five hospitals in Montgomery County were contacted and asked about the use of CPOEs in their EDs, four facilities indicated using such a system (the single differing hospital reporting that although a CPOE was used elsewhere in the hospital, the ED relied on an alternative ordering method). Among four EDs with CPOEs in use, none was programmed to provide rabies PEP guidance. Pharmacists at three of these EDs indicated that as members of larger hospital networks, their facility lacked privileges that would permit modifying their system locally. We concluded that for these three sites, piloting the use of CPOEs in delivering rabies PEP guidance might be hindered by organizational or institutional barriers; therefore, further engagement on this topic was not pursued. Of five EDs contacted, only one ED reported having administrator privileges to their CPOE system. When contacted by DHMH, information technology (IT) pharmacists at this hospital (hospital A) expressed interest in the idea of programming their system with rabies PEP guidance. They subsequently agreed to collaborate with DHMH in designing a suitable algorithm.
ROBERTSON AND FELDMAN
Figure 2 illustrates the conceptual framework for the algorithm that was developed. After an order for rabies PEP is initiated by the clinician through CPOE, the algorithm, as conceived, would prompt the clinician to answer questions about the patient. On the basis of how the prompts are answered, the resulting instructions would inform the clinician how to administer rabies biologics. These instructions would also alert the clinician to practices that should be avoided and further reinforce adherence to recommended protocols. Approximately 2 months after this discussion, hospital A launched a new CPOE for ED use. Initially, it was hoped that this upgrade would allow IT pharmacists to program PEP decision support into the new system; however, attempts to accomplish this have been unsuccessful. The main stumbling block identified by the hospital is that despite having administrator privileges, the new system is not amenable to extensive programming changes because it uses preconfigured templates as a means of simplifying the ordering process. Clinician training
In addition to providing a basic overview on rabies, the online course we developed presented five case scenarios that were designed to simulate situations routinely confronted by ED clinicians and public health practitioners. Topics expounded upon included the epidemiology of rabies in Maryland and the United States, laboratory methods used for rabies diagnosis, and the strategy of conducting 10-day observation to rule out rabies exposure from dogs, cats, and ferrets. The mechanics of administering rabies PEP were also covered in detail. Attention was given to explaining the functions of RIG
FIG. 2. Conceptual framework for an algorithm describing rabies postexposure prophylaxis (PEP). RIG, rabies immune globulin; IM, intramuscularly.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
RABIES CLINICIAN OUTREACH IN MARYLAND
and rabies vaccine and relating these functions to recommended practices. Indications for the five-dose PEP vaccine regimen, the do’s and don’ts of PEP administration, and appropriate actions to take in response to a PEP error were among other concepts discussed. After the course was approved by CDC for CE credits and posted to the DHMH rabies Internet site, it was publicly released (http://phpa.dhmh.maryland.gov/training/SitePages/ rabies.aspx) and promoted through multiple outlets, including CDC’s Morbidity and Mortality Weekly Report, professional listservs, news bulletins, social media, and nonprofit organizations. Among the approximately 2300 participants who have provided evaluation feedback as of the date of this report, 94% either agreed (46%) or strongly agreed (48%) the course addressed a gap in their knowledge or skills. In addition, 98% either agreed (40%) or strongly agreed (58%) that they were capable of applying knowledge gained from taking the course to their work. Discussion
Through the experience of implementing this intervention, we acquired a greater understanding of the problems and opportunities that influence educational outreach to the medical community. Rabies poses a special set of challenges. In the United States, human exposure potential to the rabies virus is ample because of the burden of disease in wildlife, yet only one to three human cases are reported annually (Blanton 2011). The cause of this seeming incongruence is undetermined; however, one effect it might have on certain providers is a limited appreciation for aligning rabies PEP administration with established recommendations. A multipronged strategy aimed at keeping clinicians informed about rabies might be useful in combating knowledge gaps that often occur with rare diseases. In this intervention, we attempted to use IT as a means of providing PEP instructions to clinicians, but our efforts were impeded by structural barriers associated with the technology. Although CPOEs were in use in almost all EDs that we surveyed, these facilities were limited in their ability to program additional decision support into their systems. Nevertheless, the positive interest expressed by hospital administrators at one facility indicates that future efforts might be successful if facilities that use customizable systems are engaged. Workaround strategies that bypass existing hurdles might also be a way to deliver guidance in an acceptable manner. An important catalyst for CPOE adoption is federally funded incentives that come attached to the meaningful use of electronic health records (Blummenthal 2009, Centers for Medicare & Medicaid Services 2011). As more hospitals use CPOEs to prevent medical errors, conditions might encourage the full potential of these systems to be actualized. The next step in this project is evaluating the effectiveness of our intervention. Although our goal was to improve the quality of rabies PEP administration in Maryland, available methods for measuring this outcome have limitations, the most important being the lack of sensitivity. Despite the fact that rabies PEP is reportable by Maryland LHDs, PEP that is administered without the knowledge of LHDs might go unrecognized, and providers are not required to disclose information that permits detection of ACIP deviations. Thus,
459
changes in clinician competency measured as a proxy for changes in practice might be a warranted approach. Acknowledgments
We thank Cindy Edwards from MDDHS, and Sara Critchley, Paul Melstrom, and Sergio Recuenco (content expert) from CDC for serving on the training course planning committee, as well as Karen Michaels from Suburban Hospital–Johns Hopkins Medicine, David Blythe, and Lucy Wilson from DHMH, and Charles Rupprecht from CDC for serving as course reviewers. Author Disclosure Statement
The authors have no competing financial interests. References
Auslander M, Kaelin C. Rabies postexposure prophylaxis survey—Kentucky, 1994. Emerg Infect Dis 1997; 3:199–202. Blanton JD, Palmer D, Dyer J, Rupprecht CE. Rabies surveillance in the United States during 2010. J Am Vet Med Assoc 2011; 239:773–783. Blummenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009; 360:1477–1479. Centers for Disease Control and Prevention. Human rabies prevention—United States, 2008: Recommendations of the Advisory Committee on Immunization Practices. MMWR Recommend Rep 2008; 57(No. RR-3):1–28. Centers for Disease Control and Prevention. Health information for international travel 2010. Atlanta, GA: US Department of Health and Human Services, Public Health Service; 2009. Available at http://wwwnc.cdc.gov/travel/page/yellowbook2012-home.htm Accessed August 22, 2012. Centers for Disease Control and Prevention. Use of a reduced (4-dose) vaccine schedule for postexposure prophylaxis to prevent human rabies: Recommendations of the Advisory Committee on Immunization Practices. MMWR Recommend Rep 2010; 59(No. RR-2):1–12. Centers for Medicare & Medicaid Services, US Department of Health and Human Services. Electronic health record incentive programs. Baltimore, MD: US Department of Health and Human Services, Centers for Medicare & Medicaid Services. 2011. Available at https://www.cms.gov/ehrincentiveprograms Accessed August 22, 2012. Conti L, Wiersma R, Hopkins R. Evaluation of state-provided postexposure prophylaxis against rabies in Florida. South Med J 2002; 95:225–230. Dhankar P, Vaidya SA, Fishbien DB, Meltzer MI. Cost effectiveness of rabies post exposure prophylaxis in the United States. Vaccine 2008; 5:4251–4255. Diaz JA, Griffith RA, Ng JJ, Reinert SE, et al. Patients’ use of the Internet for medical information. J Gen Intern Med 2002; 17:180–185. Drexel C, Jacobson A, Hanania NA, Whitfeld B, et al. Measuring the impact of a live, case-based, multiformat, interactive continuing medical education program on improving clinician knowledge and competency in evidence-based COPD care. Int J Chron Obstruct Pulmon Dis 2011; 6: 297–307. Fescharek R, Schwarz S, Quast U, Gandhi N, et al. Postexposure rabies prophylaxis: When the guidelines are not respected. Vaccine 1991; 9:868–872. Gautret P, Parola P. Rabies vaccination for international travelers. Vaccine 2012; 30:126–133.
Vector-Borne and Zoonotic Diseases 2014.14:454-460. Downloaded from online.liebertpub.com by Uc Davis Libraries University of California Davis on 01/09/15. For personal use only.
460
Harrigan RA, Kauffman FH. Postexposure rabies prophylaxis in an urban emergency department. J Emerg Med 1996; 14: 287–292. Jerrard DA. The use of rabies immune globulin by emergency physicians. J Emerg Med 2004; 27:15–19. Knobel DL, Cleaveland S, Coleman PG, Fevre EM, et al. Reevaluating the burden of rabies in Africa and Asia. Bull World Health Organ 2005; 83:360–368. Krebs JW, Long-Marin SC, Childs JE. Causes, costs, and estimates of rabies postexposure prophylaxis treatments in the United States. J Public Health Manag Pract 1998; 4:56–62. Kyriacou DN, Coben JH. Errors in emergency medicine: Research strategies. Acad Emerg Med 2000; 7:1201–1203. Moran GJ, Talan DA, Mower W, Newdow M, et al. Appropriateness of rabies postexposure prophylaxis in treatment for animal exposures: Emergency ID Net Study Group. JAMA 2000; 284:1001–1007. Pearson L. Medical errors hit home. Nurse Pract 2000; 25:11–13. National Association of State Public Health Veterinarians. Compendium of animal rabies prevention and control, 2011. MMWR Recommend Rep 2011; 60(No. RR-6):1–17. Reason J. Human error: Models and management. Br Med J 2000; 320:768–770. Shwiff SA, Sterner RT, Jay MT, Parikh S, et al. Direct and indirect costs of rabies exposure: A retrospective study in southern California (1998–2002). J Wildlife Dis 2007; 43:251–257.
ROBERTSON AND FELDMAN
Sterner RT, Sun B. Relative factor costs of wildlife rabies impacts in the U.S. USDA National Wildlife Research Center– Staff Publications. 2004. Available at http://digitalcommons .unl.edu/cgi/viewcontent.cgi?article = 1385&context = icwdm_ usdanwrc Accessed August 22, 2012. Tamler R, Green D, Skamagas M, Breen TL, et al. Effect of case-based training for medical residents on inpatient glycemia. Diabetes Care 2011; 31:1738–1740. Wilde H. Failures of post-exposure rabies prophylaxis. Vaccine 2007; 25:7605–7609. Woodward HI, Mytton OT, Lemer C, Yardley IE, et al. What have we learned about interventions to reduce medical errors? Annu Rev Public Health 2010; 21:479–497. World Health Organization. WHO expert consultation on rabies. World Health Organ Tech Ser 2005; 931:1–88.
Address correspondence to: Kis Robertson U.S. Department of Agriculture 1400 Independence Avenue, SW Stop 3766, PP 3, 9-210B Washington, DC 20250 E-mail: [email protected]
