ORIGINAL ARTICLE
Ambulatory Computerized Prescribing and Preventable Adverse Drug Events Joseph Marcus Overhage, MD, PhD,* Tejal K. Gandhi, MD, MPH,†‡ Carol Hope, PharmD, MS,§ Andrew C. Seger, PharmD,† Michael D. Murray, PharmD, MPH,||¶ E. John Orav, PhD,** and David W. Bates, MD, MSc†‡ Background: Adverse drug events (ADEs) represent a significant cause of injury in the ambulatory care setting. Computerized physician order entry reduces rates of serious medication errors that can lead to ADEs in the inpatient setting, but few studies have evaluated whether computerized prescribing in the ambulatory setting reduces preventable ADE rates in ambulatory care. Objective: To determine the rates of preventable ADEs before and after the implementation of computerized prescribing with basic clinical decision support for ordering medications. Design: Before-after study of ADE rates in practices implementing computer order entry. Participants: Adult patients seeking care in primary care practices at academic medical centers in Boston, Massachusetts (n = 41,819), and Indianapolis, Indiana (n = 9128). Main Measures: We attempted to standardize the medication-related decision support knowledge base provided at the 2 sites, although the electronic records and presentation layers used at the 2 sites differed. The primary outcome was preventable ADEs identified based on structured results or symptoms defined by extracting symptom concepts from provider notes; potential ADEs were a secondary outcome. Results: Computerized prescribing did not significantly change the rate of preventable ADEs at either site. Compared with Boston practices, the rate of potential ADEs was more than seven-fold greater at Indianapolis (6.4/10,000 patient-months vs. 49.5/10,000 patient-months, P < 0.001). Computerized prescribing was associated with a 56% decrease in the potential ADE rate at Indianapolis (49.5 to 21.9/10,000 patient-months, P < 0.001) but a 104% increase at Boston (6.4 to 13.0/10,000 patientmonths, P < 0.001). Preventable ADEs that occurred after computerized prescribing was implemented were due to patient education issues, physicians ignoring feedback from CDSS, and incomplete computerized knowledge base was incomplete (34%, 33%, and 33% in Indianapolis and 44%, 28%, and 28% in Boston). Conclusions: The implementation of computerized prescribing in the ambulatory setting was not associated with any change in preventable ADEs but was associated with a decrease in potential ADEs at Indianapolis but an increase at Boston, although the absolute rate of ADEs was much lower in Boston. From the *Siemens Health Services, Malvern, PA;, †Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital; ‡Harvard Medical School, Boston, MA; §National Library of Medicine VA Medical Informatics Fellow, Salt Lake City, UT; ||Purdue University College of Pharmacy, Indianapolis, IN; ¶Regenstrief Institute, Indianapolis, IN; and **Harvard School of Public Health, Boston, MA. Correspondence: Joseph Marcus Overhage, MD, PhD, Chief Medical Information Officer, Siemens Health Services, 50 Valley Stream Parkway, MC B9K Malvern, PA 19355 (e‐mail: [email protected]). The authors disclose no conflict of interest. The study was supported by grant U18 HS11169 from the Agency for Healthcare Research and Quality. The agency did not have any role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; nor preparation, review, or approval of the manuscript. Clinical Trials Registration: www.clinicaltrials.gov Improving Safety by Basic Computerized Outpatient Prescribing 0008–44. Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Key Words: adverse drug events, ambulatory care, comparative study, decision support systems, medical records systems, computerized (J Patient Saf 2016;12: 69–74)
BACKGROUND More than 4 billion outpatient prescriptions are filled in the United States each year; this has increased 1% per year over the last 5 years.1 Indeed, 74% of ambulatory visits result in initiation or continuation of a drug.2 Although drugs provide many benefits, even appropriately-prescribed drugs have the potential to cause adverse drug events (ADEs), which are harms associated with drug administration. Potential ADEs are errors in medication prescribing, transcribing, administering, and monitoring. Preventable ADEs are due to errors the provider could have avoided; such as through improved prescribing accuracy (e.g., correct dose). Adverse drug events are common in ambulatory patients, with annual estimates ranging from 3% to 38%,3–6 which translates to 8.8 million ADEs annually in the United States.7 Errors in medication prescribing or monitoring account for approximately 28% of ADEs; most are preventable.8 Reviews of the potential value of health information technology (HIT) to reduce the frequency of ADEs have identified computerized prescribing, including provider order entry (CPOE), as an important tool for reducing them, mostly in the inpatient setting.9–12 Computerized prescribing refers to systems that automate the ordering process, ensuring legible, complete, and more standardized orders, including prescriptions. These prescriptions will eliminate some medication errors and resulting ADEs, but clinical decision support systems (CDSS) incorporated into computerized prescribing could eliminate even more ADEs. Basic clinical decision support capabilities include dosage, route, and frequency suggestions. Advanced clinical decision support capabilities include allergy checking, drug-drug, drug-laboratory, and drug-diagnosis interactions along with corollary orders and other medication management guidelines.13 Studies have also demonstrated CPOE can reduce potential ADE rates (some of which result in ADEs) in the inpatient setting.14 The limited evidence currently available in the ambulatory care setting suggests that computerized prescribing reduces medication errors and potential ADEs but not preventable ADEs.15–19 In July of 2010, CMS issued rules describing the meaningful use HIT, including implementation of e-prescribing, largely on the belief that these systems will improve the quality and safety of clinical care and approximately 83% of providers now have the ability to order prescriptions electronically.16 The purpose of this study was to assess the impact of basic computerized prescribing on preventable and potential ADEs in the ambulatory setting.
Setting
METHODS
The study was conducted in 2 ambulatory primary care practices, one in Boston, Massachusetts, and the other in Indianapolis, www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
69
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
Indiana. Boston study sites included 11 ambulatory practices affiliated with Brigham and Women’s Hospital, an urban tertiary academic medical center. These practices included hospital and community-based clinics and community health centers, using an electronic record called the longitudinal medical record (LMR) supported by the Brigham Integrated Computing System.20 The Indianapolis study sites included 6 ambulatory practices from IU Medical Group-Primary Care, a large primary care practice affiliated with the Indiana University School of Medicine. These sites used the Medical Gopher electronic medical record.21 The Gopher stores data in a retrievable format within the Regenstrief Medical Record System.22 Patients at both sites were cared for by faculty and house staff supervised by that faculty. Patients eligible for this study were 18 years of age or greater with ambulatory visits at the participating primary care practices during the study periods. Indiana University’s institutional review board and the Brigham and Women’s Human Research Committee approved the study.
Study Design The study was a before-after design with each measurement period lasting 6 months. In Boston, the preintervention period lasted from January 1, 2000, to through June 30, 2000, and the postintervention period lasted from January 1, 2001, through June 30, 2001; in Indianapolis, the preintervention period lasted from January 1, 1997, through June 30, 1997, and the postintervention period from January 1, 2001, through June 30, 2001. We selected ambulatory care practices that had implemented computerized prescribing within the previous 5 years. During the control period (before computerized prescribing implementation), providers wrote prescriptions by hand on prescription blanks in Boston and patient-specific computer-generated prescription forms in Indianapolis. All providers at all sites for both practices were included in the study. We chose to study 2 sites with different prescribing systems because this reflects how HIT will be developed and deployed in the United States for at least the near future: multiple vendors (or home-grown systems), each with different features.
Boston Providers at Brigham and Women’s Hospital affiliated practices use the LMR as their medical record, which includes ambulatory electronic prescribing.23 Clinicians ordered many medications using electronic prescribing and printed prescriptions for the patient. Prescriptions for acute medications were frequently written by hand. The LMR applies clinical decision support logic to medication orders at the time of ordering, presenting messages to the clinician to help guide the ordering process. All data contained in the Brigham and Women’s EMR are available to the LMR for decision support purposes (e.g., patient age, sex, medical problems, medications, laboratory data, and allergies).
Indianapolis The Medical Gopher is the Regenstrief Institute’s clinical workstation providing comprehensive computerized ordering and clinical documentation, workflow, patient education, and other functions.24 Importantly, the Gopher provides a fully developed clinical decision support system. Over time, the system was implemented in a number of ambulatory practices including several primary care multispecialty practices (internal medicine, pediatrics, and obstetrics/gynecology), a number of specialty practices, and the largest, busiest emergency department and trauma center in Indianapolis.
70
www.journalpatientsafety.com
Intervention During the intervention period, providers in Indianapolis generated prescriptions for all of their patients and providers in Boston generated nearly all prescriptions for chronic medications and some for acute medications using the computerized prescribing systems in place at their site. Many patients also received care from providers, including specialists, in other practices that did not use the computerized prescribing systems. The computerized prescribing intervention period represented a major systems change in the practices and included several features expected to reduce the frequency of preventable and potential ADEs (Fig. 1).23,24 The application provided physicians with a menu of medications from the formulary, default doses, and a range of potential doses for each medication. Physicians were required to enter dosage, route, and frequency for all orders. Also, computerized prescribing ensured all orders were legible and included the prescribing physician’s signature. For a number of medications, the system displayed relevant laboratory results (e.g., potassium levels when the provider ordered furosemide) at the time of ordering. Both systems provided drug-allergy, drug-laboratory, and drug-drug interaction checks. The system in Indianapolis provided some consequent orders related to drug-laboratory checking, which are orders that should follow from other orders (e.g., anticonvulsant levels when selected anticonvulsants were ordered). The system in Boston provided default dosing and some therapeutic duplication alerting. We standardized the medication-related basic decision support to the extent possible across the 2 sites, although the presentation layers differed.
End Points We defined an ADE as harm associated with the administration of a drug and preventable ADEs were those due to a medication error (ME) that the provider could have entirely avoided. Our primary end points were the number of preventable ADEs 25 (Fig. 2). The reviewers based their decisions about whether an ADE was preventable on the physician's presumed knowledge at the time they prescribed the drug. If insufficient information was available, the reviewers assumed that the physician's decision was correct. Our secondary end points were the number of potential ADEs, which we defined a potential ADE as an error in the medication use process, including prescribing, transcribing, administering, and monitoring steps.27
ADE Detection We previously described methods to determine the incidence of ADEs and provide only a brief overview here.28 We drew on rules used in Regenstrief Institute’s and Brigham and Women’s Hospital’s previous inpatient and outpatient ADE studies to develop the 122 specific criteria used in our study. These criteria fell into 2 broad categories: those based on structured results such as coded laboratory tests and those based on symptoms mentioned in a provider’s notes. We expressed these criteria as database queries used to create the list of cases or candidate ADEs, which we call signals. After identifying these signals, we removed all duplicates and randomly sampled signals for review. Data analysts imported the randomly selected signals into a database. Pharmacists and nurses blinded to whether the event occurred with or without the use of computerized prescribing reviewed and adjudicated ADEs and potential ADEs, in consultation with physicians as necessary. The same pool of reviewers evaluated events from before and after intervention periods. The ability of our approach to capture medication errors not linked to patient harm was limited. © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Computerized Prescribing and Preventable ADEs
FIGURE 1. A comparison of the functions available at the time of the study in Boston and Indianapolis. Computerized prescribing systems using Bell's framework.25 Full circles indicate the functional capability was robustly present, half circles that the functional capability partially exists, and empty circles indicate that the functional capability was missing. An NA indicates that the functional capability was not applicable.
ADE Classification Reviewers used all available data to classify potential and preventable ADE severity as (fatal, life-threatening, serious, or significant) and the cause of preventable ADEs, which we classified as patient education issues, providers ignoring CDSS feedback, or CDSS knowledge base issues.29
separately. We measured events per 10,000 patient-months to account for variation over time in the number of patients seen. Because the implementation of computerized prescribing changed how physicians documented care and provided decision support to improve monitoring, both of which could result in detecting
Sample Size Estimates In previous work, we found 864 ADEs in 88,514 patient visits, of which, 329 (38%) were judged preventable.30 Our primary end point of preventable ADEs represented a rate of 329 preventable ADEs per 88,514 visits (0.37%). To demonstrate a 50% reduction in this rate, which we expected based on reductions seen in inpatient environments, with a 90% power and a 2-sided type I error of 5%, we needed 17,000 patient visits in both the intervention and control arms of the study. Although we did not have a measure of the correlation between ADEs among visits to the same physician, such a correlation would reduce the power of the study; therefore, we inflated the sample sizes by 20% to compensate for correlated responses. This brought the sample size to 20,400 visits per arm.
Analysis Because of differences in patient population and computerized prescribing systems at the 2 sites, we analyzed them © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 2. Diagrammatic representation of the relationship between adverse drug events (ADEs), medical errors, and preventable ADEs.26 www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
71
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
TABLE 1. Patient Characteristics Characteristics
Boston
Indianapolis
Sex (% male) Age in years (mean ± SD) Race % Black % White
28.1 48.1 ± 16.7
30.6 46.7 ± 15.5
22 59
40† 53
†
P < 0.05.
more ADEs, we also analyzed the change in rates with adjustments for the number of nonpreventable ADEs (the number of preventable ADEs subtracted from the total number of total ADEs), which we would not expect to be affected by implementation of computerized prescribing. Assuming that the number of events followed a Poisson distribution and tested for a change in events rates between the before and after periods by using a normal approximation of the Poisson. For our primary analyses, we assumed, as a null hypothesis, that there would be no change in the event rate over time. For analysis of secondary outcomes, we estimated the relative change in ADEs over the same period, assumed that this change reflected natural temporal trends, and used this estimated change as the null hypothesis.
105% increase in both preventable ADEs and (significantly) potential ADEs in Boston. The intervention was not associated with a significant change in symptom-based preventable ADEs (Table 4) in Indianapolis or Boston, or potential ADEs in Boston, but was associated with a significant 311% increase in potential ADEs in Indianapolis. There were no significant changes in serious or life-threatening events at either site. The most common cause of preventable ADEs was patient education issues (34% in Indianapolis and 44% in Boston), leading patients to take the wrong dose or to fail to undergo follow-up testing in some cases. A smaller proportion of preventable ADEs were due to physicians ignoring feedback from CDSS (33% in Indianapolis, and 28% in Boston). Finally, in a similar number of cases, the computerized knowledge base was incomplete, such as failing to include drug-age checking (33% in Indianapolis and 28% in Boston). Because the order entry system prompted providers to monitor certain laboratory tests to improve patient safety, increased surveillance could have increased the number of ADEs detected. We observed a 20% increase in the rate of laboratory monitoring tests performed. At the Boston site, the number of adverse events increased, but the preventable adverse events did not significantly change (Tables 2, 3, and 4). In addition, in Indianapolis, computerized prescribing implementation dramatically increased the quantity of electronic documentation available, which could have increased the number of symptom ADEs detected.
RESULTS
DISCUSSION
We studied 88 physicians in 6 primary care practices in Indiana and 140 physicians in 11 practices in Boston. There were 9128 patients in Indianapolis and 41,819 patients in Boston in the preintervention period. In the postintervention period, there were 9882 patients in Indianapolis and 42,180 patients in Boston. Patient age and sex were similar between study sites, except that there were a higher proportion of black patients in Indianapolis (Table 1). As shown in Table 2, total measured ADEs increased significantly by 112% in Indianapolis but did not change significantly in Boston (19%). Computerized prescribing was associated with a nonsignificant reduction in total preventable ADEs by 25% in Indianapolis and 2% in Boston. Compared with Boston practices, the rate of potential ADEs was more than 7-fold greater in Indianapolis (6.4/10,000 patient-months vs. 49.5/10,000 patientmonths, P < 0.001). Computerized prescribing was associated with a 56% decrease in the potential ADE rate in Indianapolis (49.5 to 21.9/10,000 patient-months, P < 0.001) but a 104% increase in Boston (6.4 to 13.0/10,000 patient-months, P < 0.001). Adverse drug events based on structured results are shown in Table 3. In this structured data subset, the intervention was associated with a significant reduction in preventable ADEs by 85% and potential ADEs by 71% in Indianapolis but was associated with a
Computerized prescribing with basic decision support to order medications was associated with a reduced rate of preventable and potential ADEs in ambulatory primary care practices associated with one medical center but not another. Absolute rates were much higher in Indianapolis than in Boston, even postintervention. The same pattern was evident for serious and life-threatening events. Some possible explanations for the differences in the ADE rate, including factors related to the providers, patient populations, and computerized prescribing systems. Provider differences seem unlikely to account for the findings because venues in both cities were associated with tertiary care university medical centers. We found differences in patient populations in terms of race, with Indianapolis sites having a higher percent of black patients. Significant racial differences in health literacy31 contribute to medication errors (such as taking incorrect doses),32,33 these may account for a portion of the preintervention difference in education-related preventable ADEs. Interestingly, the number of preventable errors attributed to patient education was lower in Indianapolis than in Boston potentially because the computerized prescribing system in Indianapolis provided richer patient education tools and coordination of care facilities. More likely significant is heterogeneity between the 2 computerized prescribing systems and the implementation at the sites. Although similar in many ways, there were
TABLE 2. ADEs, Preventable ADEs, and Potential ADEs, Mean and Standard Deviation, Adjusted Per 10,000 Patient-Months of Exposure Preintervention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
36.3 (5.38) 5.5 (1.16) 6.4 (0.60)
44.8 (5.85) 5.4 (1.19) 13.0 (1.03)†
51.5 (5.48) 12.2 (2.56) 49.5 (8.76)
109.3 (7.08)† 9.1 (2.19) 21.9 (3.20)†
†
P < 0.05.
72
www.journalpatientsafety.com
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Computerized Prescribing and Preventable ADEs
TABLE 3. ADEs, Preventable ADEs, and Potential ADEs Identified from Structured Data Such as Laboratory Results, Allergies, and Medication Orders, Mean and Standard Deviation, Adjusted Per 10,000 Patient-Months of Exposure Preintrevention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
6.46 (1.28) 1.08 (0.29) 6.34 (0.60)
4.74 (0.63) 2.21 (1.03) 13.0 (1.03)†
18.1 (2.01) 6.57 (1.48) 47.5 (6.39)
28.2 (2.53) 1.01 (0.455)† 13.7 (2.19)†
†
P < 0.05.
important differences. First, although we tried to standardize the decision support offered , the system in Indianapolis had a different user interface and offered more decision support at the time of the study than did the Boston system. Important differences included safety alerts based on drug-choice errors (such as allergies), dosage calculation, patient education materials, and corollary orders (such as monitoring tests). Such differences might account for the failure of the intervention in Boston to be associated with a reduction in total ADEs and preventable ADEs, while facilitating such improvement in Indianapolis. Second, providers in Boston frequently chose not to use the system when writing prescriptions for drugs the patient would take for a limited time, such as an antibiotic. Our findings represent a lower bound of the proportion of ADEs that computerized prescribing can prevent. Some patients saw providers, such as specialists, who did not use computerized prescribing; as such, our study missed opportunities to avoid ADEs related to medications prescribed by these providers. Furthermore, our computerized prescribing systems included basic, but not advanced, decision support, such as pregnancy- and agebased alerting and renal function based dosing. Based on our analysis of causes of preventable ADEs during the postcomputerized prescribing period, we would expect advanced decision support (eliminating the knowledge base deficiencies) to prevent onethird of the preventable errors that occurred during the intervention period further reducing the number of in preventable ADEs. Finally, detection of ADEs in the postintervention period might have improved from increased drug therapy monitoring by the computerized prescribing systems.34,35 Although our study provides evidence that using computerized prescribing is associated with a reduction in preventable ADEs, it also highlights the importance of a multifaceted approach to improving patient safety. In our study, the most common causes of preventable ADEs were patient education and providers ignoring feedback from the CDSS. Although both of these factors may be partially addressed through improved system design and implementation—through more artfully crafted delivery of CDS feedback to providers for example—the level of patient engagement
and the culture of safety depend heavily on factors other than computerized prescribing implementation. More than one-half of preventable ADEs in the ambulatory setting result from errors in prescribing.5,6,36 Despite available evidence, less than 25% of physicians in a national survey rated “increasing the use of computers to order drugs and medical tests” as “very effective” for reducing potential ADEs.37 This suggests that physician’s lack “outcomes expectancy” for using computerized prescribing to order medications. Ideally, developers should match computerized prescribing systems functions with the types of errors made when manual prescribing processes are used.38 Even without advanced clinical decision support, computerized prescribing addresses potential sources of error in the prescribing process by creating legible, complete prescriptions. Adding clinical decision support can eliminate other sources of error. The study had a number of limitations, including factors related to the specific practices studied, the systems, and the study design. To improve generalizability, we studied practices with wellestablished computerized prescribing systems in 2 geographically distinct practices. These practices have affiliations with academic medical centers but are likely representative of a growing number of private primary care practices implementing computerized prescribing in that they were striving to provide care efficiently and with high, consistent quality. Nonetheless, the 2 computerized prescribing systems were “home-grown;” neither is commercially available, limiting generalizability. It would have been ideal to randomize half of the practices to computerized prescribing while maintaining the remainder on a paper system. This was not possible due to the complexity of computerized prescribing implementation in large organizations; therefore, we relied on a before-after design. Our results also show that the process of implementation is a real world issue and may present differences between systems in their ability to prevent potential or actual patient harm. Because the before-after comparisons showed improvement at one site, a temporal trend could have accounted for some of the differences. However, we found no evidence for an underlying temporal trend, and nonpreventable ADE rates remained the same.
TABLE 4. ADEs, Preventable ADEs, and Potential ADEs Identified from Symptom-Based Review of Clinical Notes, Mean and Standard Deviation, Adjusted per 10,000 Patient-Months of Exposure Preintervention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
29.9 (5.22) 4.42 (1.16) 0.04 (0.048)
40.1 (5.81) 3.20 (0.97) 0 (0)
33.4 (5.11) 5.56 (0.20) 2.01 (1.19)
81.1 (6.58)† 8.10 (2.02) 8.26 (2.36)†
†
P < 0.05.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
73
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
Our study provides evidence that well-designed computerized prescribing with basic decision support and patient education materials is associated with a reduction in the number of potential and preventable ADEs in ambulatory settings. Because, at least in the near future, the United States’ HIT infrastructure will be heterogeneous (no single electronic health record), each vendor (or developer of home-grown system) must incorporate evidence-based user interaction, user interface, and decision support in the effort to reduce medication errors. We expect reductions in potential and preventable ADEs to translate into important improvements in morbidity, mortality, and cost. ACKNOWLEDGMENT The authors thank the data managers, research assistants, and others who performed important chart review and data management tasks that allowed this work to be completed. REFERENCES 1. IMS Institute for Healthcare Informatics. The use of Medicines in the United States: Review of 2011. Available at: http://www.imshealth.com/ ims/Global/Content/Insights/IMS%20Institute%20for%20Healthcare% 20Informatics/IHII_Medicines_in_U.S_Report_2011.pdf. Accessed October 20, 2012. 2. Schappert SM, Rechtsteiner EA. “Ambulatory medical care utilization estimates for 2007”. Vital Health Stat. 2011;13:1–38. 3. Hutchinson TA, Flegel KM, Kramer MS, et al. “Frequency, severity and risk factors for adverse drug reactions in adult out-patients: a prospective study”. J Chronic Dis. 1986;39:533–542. 4. Gurwitz JH, Field TS, Harrold LR, et al. “Incidence and preventability of adverse drug events among older persons in the ambulatory setting”. JAMA. 2003;289:1107–1116. 5. Gandhi TK, Weingart SN, Borus J, et al. “Adverse drug events in ambulatory care”. N Engl J Med. 2003;348:1556–1564. 6. Hanlon JT, Schmader KE, Koronkowski MJ, et al. “Adverse drug events in high risk older outpatients”. J Am Geriatr Soc. 1997;45:945–948. 7. Johnston D, Pan E, Walker J, et al. Patient Safety in the Physician's Office: Assessing the value of ambulatory CPOE. In: Center for Information Technology Leadership. Oakland, CA: California HealthCare Foundation. 8. Bates DW, Cullen DJ, Laird N, et al. “Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group”. JAMA. 1995;274:29–34. 9. Bates DW. “Using information technology to reduce rates of medication errors in hospitals”. BMJ. 2000;320:788–791. 10. Bates DW, Cohen M, Leape LL, et al. “Reducing the frequency of errors in medicine using information technology”. J Am Med Inform Assoc. 2001;8:299–308. 11. Wolfstadt JI, Gurwitz JH, Field TS, et al. “The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review”. J Gen Intern Med. 2008;23:451–458.
16. Hsiao C-J, Hing E. Use and Characteristics of Electronic Health Record Systems Among Office-Based Physician Practices: United States, 2001–2013. NCHS Data Brief, No. 143. Hyattsville, MD: National Center for Health Statistics; 2014. 17. Steele AW, Eisert S, Witter J, et al. “The effect of automated alerts on provider ordering behavior in an outpatient setting”. PLoS Med. 2005;2:e255. 18. Gandhi TK, Weingart SN, Seger AC, et al. “Impact of Basic Computerized Prescribing on Outpatient Medication Errors and Adverse Drug Events”. J Am Med Inform Assoc. 9(Suppl 6):S48–S49. 19. Kaushal R, Kern LM, Barron Y, et al. “Electronic prescribing improves medication safety in community-based office practices”. J Gen Intern Med. 2010;25:530–536. 20. Teich JM, Glaser JP, Beckley RF, et al. „The Brigham integrated computing system (BICS): advanced clinical systems in an academic hospital environment”. Int J Med Inform. 1999;54:197–208. 21. McDonald CJ, Tierney WM. “The Medical Gopher–a microcomputer system to help find, organize and decide about patient data”. West J Med. 1986;145:823–829. 22. McDonald CJ, Overhage JM, Tierny WM, et al. “The Regenstrief Medical Record System: a quarter century experience”. Int J Med Inform. 1999;54:225–253. 23. Abramson EL, Barron Y, Quaresimo J, et al. “Electronic prescribing within an electronic health record reduces ambulatory prescribing errors”. Jt Comm J Qual Patient Saf. 2011;37:470–478. 24. Spurr CD, Wang SJ, Kuperman GJ, et al. Confirming and Delivering the Benefits of an Ambulatory Electronic Medical Record for an Integrated Delivery System. Boston, MA: TEPR; 2001. 25. Bell DS, Cretin S, Marken RS, et al. “A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities”. J Am Med Inform Assoc. 2004;11:60–70. 26. Gandhi TK, Seger DL, Bates DW. “Identifying drug safety issues: from research to practice.” Int J Qual Health Care. 2000;12:69–76. 27. Overhage JM, Lukes A. “Practical, reliable, comprehensive method for characterizing pharmacists' clinical activities”. Am J Health Syst Pharm. 1999;56:2444–2450. 28. Hope C, Overhage JM, Seger A, et al. “A tiered approach is more cost effective than traditional pharmacist-based review for classifying computer-detected signals as adverse drug events”. J Biomed Inform. 2003;36:92–98. 29. Marimoto T, Gandhi TK, Seger AC, et al. “Adverse drug events and medication errors: detection and classification methods”. Qual Saf Health Care. 2004;13:306–314. 30. Honigman B, Lee J, Rothschild J, et al. “Using computerized data to identify adverse drug events in outpatients”. J Am Med Inform Assoc. 2001;8:254–266. 31. The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. U.S. Department of Education, 2006. Available at: http://www.eric.ed.gov/PDFS/ED493284.pdf. 32. Baker DW, Parker RM, Williams MV, et al. The health care experience of patients with low literacy. Arch Fam Med. 1996;5:329–334. 33. Davis TC, Wolf MS, Bass PF 3rd, et al. “Literacy and miunderstanding prescription drug labels”. Ann Intern Med. 2006;145:887–894.
12. Shamliyan TA, Duval S, Du J, et al. “Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors”. Health Serv Res. 2008;43(1 Pt 1):32–53.
34. Kahn MG, Ranade D. “The impact of electronic medical records data sources on an adverse drug event quality measure”. J Am Med Inform Assoc. 2010;17:185–191.
13. Overhage JM, Tierney WM, Zhou XH, et al. “A randomized trial of “corollary orders” to prevent errors of omission”. J Am Med Inform Assoc. 1997;4:364–375.
35. Nebeker JR, Hoffman JM, Weir CR, et al. “High rates of adverse drug events in a highly computerized hospital”. Arch Intern Med. 2005;165:1111–1116.
14. King WJ, Paice N, Rangrej J, et al. “The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients”. Pediatrics. 2003;112(3 Pt 1):506–509. 15. Devine EB, Hansen RN, Wilson-Norton JL, et al. “The impact of computerized provider order entry on medication errors in a multispecialty group practice”. J Am Med Inform Assoc. 2010;17:78–84.
74
www.journalpatientsafety.com
36. Burnum JF. “Letter: Preventability of adverse drug reactions”. Ann Intern Med. 1976;85:80–81. 37. Rosen AB, Blendon RJ, DesRoches CM, et al. “Physicians' views of interventions to reduce medical errors: does evidence of effectiveness matter?”. Acad Med. 2005;80:189–192. 38. Simon HA. The Science of Design: Creating the Artificial in the Science of the Artificial. 3rd ed. Cambridge, MA: MIT Press; 1996:111–138.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
Ambulatory Computerized Prescribing and Preventable Adverse Drug Events Joseph Marcus Overhage, MD, PhD,* Tejal K. Gandhi, MD, MPH,†‡ Carol Hope, PharmD, MS,§ Andrew C. Seger, PharmD,† Michael D. Murray, PharmD, MPH,||¶ E. John Orav, PhD,** and David W. Bates, MD, MSc†‡ Background: Adverse drug events (ADEs) represent a significant cause of injury in the ambulatory care setting. Computerized physician order entry reduces rates of serious medication errors that can lead to ADEs in the inpatient setting, but few studies have evaluated whether computerized prescribing in the ambulatory setting reduces preventable ADE rates in ambulatory care. Objective: To determine the rates of preventable ADEs before and after the implementation of computerized prescribing with basic clinical decision support for ordering medications. Design: Before-after study of ADE rates in practices implementing computer order entry. Participants: Adult patients seeking care in primary care practices at academic medical centers in Boston, Massachusetts (n = 41,819), and Indianapolis, Indiana (n = 9128). Main Measures: We attempted to standardize the medication-related decision support knowledge base provided at the 2 sites, although the electronic records and presentation layers used at the 2 sites differed. The primary outcome was preventable ADEs identified based on structured results or symptoms defined by extracting symptom concepts from provider notes; potential ADEs were a secondary outcome. Results: Computerized prescribing did not significantly change the rate of preventable ADEs at either site. Compared with Boston practices, the rate of potential ADEs was more than seven-fold greater at Indianapolis (6.4/10,000 patient-months vs. 49.5/10,000 patient-months, P < 0.001). Computerized prescribing was associated with a 56% decrease in the potential ADE rate at Indianapolis (49.5 to 21.9/10,000 patient-months, P < 0.001) but a 104% increase at Boston (6.4 to 13.0/10,000 patientmonths, P < 0.001). Preventable ADEs that occurred after computerized prescribing was implemented were due to patient education issues, physicians ignoring feedback from CDSS, and incomplete computerized knowledge base was incomplete (34%, 33%, and 33% in Indianapolis and 44%, 28%, and 28% in Boston). Conclusions: The implementation of computerized prescribing in the ambulatory setting was not associated with any change in preventable ADEs but was associated with a decrease in potential ADEs at Indianapolis but an increase at Boston, although the absolute rate of ADEs was much lower in Boston. From the *Siemens Health Services, Malvern, PA;, †Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital; ‡Harvard Medical School, Boston, MA; §National Library of Medicine VA Medical Informatics Fellow, Salt Lake City, UT; ||Purdue University College of Pharmacy, Indianapolis, IN; ¶Regenstrief Institute, Indianapolis, IN; and **Harvard School of Public Health, Boston, MA. Correspondence: Joseph Marcus Overhage, MD, PhD, Chief Medical Information Officer, Siemens Health Services, 50 Valley Stream Parkway, MC B9K Malvern, PA 19355 (e‐mail: [email protected]). The authors disclose no conflict of interest. The study was supported by grant U18 HS11169 from the Agency for Healthcare Research and Quality. The agency did not have any role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; nor preparation, review, or approval of the manuscript. Clinical Trials Registration: www.clinicaltrials.gov Improving Safety by Basic Computerized Outpatient Prescribing 0008–44. Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Key Words: adverse drug events, ambulatory care, comparative study, decision support systems, medical records systems, computerized (J Patient Saf 2016;12: 69–74)
BACKGROUND More than 4 billion outpatient prescriptions are filled in the United States each year; this has increased 1% per year over the last 5 years.1 Indeed, 74% of ambulatory visits result in initiation or continuation of a drug.2 Although drugs provide many benefits, even appropriately-prescribed drugs have the potential to cause adverse drug events (ADEs), which are harms associated with drug administration. Potential ADEs are errors in medication prescribing, transcribing, administering, and monitoring. Preventable ADEs are due to errors the provider could have avoided; such as through improved prescribing accuracy (e.g., correct dose). Adverse drug events are common in ambulatory patients, with annual estimates ranging from 3% to 38%,3–6 which translates to 8.8 million ADEs annually in the United States.7 Errors in medication prescribing or monitoring account for approximately 28% of ADEs; most are preventable.8 Reviews of the potential value of health information technology (HIT) to reduce the frequency of ADEs have identified computerized prescribing, including provider order entry (CPOE), as an important tool for reducing them, mostly in the inpatient setting.9–12 Computerized prescribing refers to systems that automate the ordering process, ensuring legible, complete, and more standardized orders, including prescriptions. These prescriptions will eliminate some medication errors and resulting ADEs, but clinical decision support systems (CDSS) incorporated into computerized prescribing could eliminate even more ADEs. Basic clinical decision support capabilities include dosage, route, and frequency suggestions. Advanced clinical decision support capabilities include allergy checking, drug-drug, drug-laboratory, and drug-diagnosis interactions along with corollary orders and other medication management guidelines.13 Studies have also demonstrated CPOE can reduce potential ADE rates (some of which result in ADEs) in the inpatient setting.14 The limited evidence currently available in the ambulatory care setting suggests that computerized prescribing reduces medication errors and potential ADEs but not preventable ADEs.15–19 In July of 2010, CMS issued rules describing the meaningful use HIT, including implementation of e-prescribing, largely on the belief that these systems will improve the quality and safety of clinical care and approximately 83% of providers now have the ability to order prescriptions electronically.16 The purpose of this study was to assess the impact of basic computerized prescribing on preventable and potential ADEs in the ambulatory setting.
Setting
METHODS
The study was conducted in 2 ambulatory primary care practices, one in Boston, Massachusetts, and the other in Indianapolis, www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
69
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
Indiana. Boston study sites included 11 ambulatory practices affiliated with Brigham and Women’s Hospital, an urban tertiary academic medical center. These practices included hospital and community-based clinics and community health centers, using an electronic record called the longitudinal medical record (LMR) supported by the Brigham Integrated Computing System.20 The Indianapolis study sites included 6 ambulatory practices from IU Medical Group-Primary Care, a large primary care practice affiliated with the Indiana University School of Medicine. These sites used the Medical Gopher electronic medical record.21 The Gopher stores data in a retrievable format within the Regenstrief Medical Record System.22 Patients at both sites were cared for by faculty and house staff supervised by that faculty. Patients eligible for this study were 18 years of age or greater with ambulatory visits at the participating primary care practices during the study periods. Indiana University’s institutional review board and the Brigham and Women’s Human Research Committee approved the study.
Study Design The study was a before-after design with each measurement period lasting 6 months. In Boston, the preintervention period lasted from January 1, 2000, to through June 30, 2000, and the postintervention period lasted from January 1, 2001, through June 30, 2001; in Indianapolis, the preintervention period lasted from January 1, 1997, through June 30, 1997, and the postintervention period from January 1, 2001, through June 30, 2001. We selected ambulatory care practices that had implemented computerized prescribing within the previous 5 years. During the control period (before computerized prescribing implementation), providers wrote prescriptions by hand on prescription blanks in Boston and patient-specific computer-generated prescription forms in Indianapolis. All providers at all sites for both practices were included in the study. We chose to study 2 sites with different prescribing systems because this reflects how HIT will be developed and deployed in the United States for at least the near future: multiple vendors (or home-grown systems), each with different features.
Boston Providers at Brigham and Women’s Hospital affiliated practices use the LMR as their medical record, which includes ambulatory electronic prescribing.23 Clinicians ordered many medications using electronic prescribing and printed prescriptions for the patient. Prescriptions for acute medications were frequently written by hand. The LMR applies clinical decision support logic to medication orders at the time of ordering, presenting messages to the clinician to help guide the ordering process. All data contained in the Brigham and Women’s EMR are available to the LMR for decision support purposes (e.g., patient age, sex, medical problems, medications, laboratory data, and allergies).
Indianapolis The Medical Gopher is the Regenstrief Institute’s clinical workstation providing comprehensive computerized ordering and clinical documentation, workflow, patient education, and other functions.24 Importantly, the Gopher provides a fully developed clinical decision support system. Over time, the system was implemented in a number of ambulatory practices including several primary care multispecialty practices (internal medicine, pediatrics, and obstetrics/gynecology), a number of specialty practices, and the largest, busiest emergency department and trauma center in Indianapolis.
70
www.journalpatientsafety.com
Intervention During the intervention period, providers in Indianapolis generated prescriptions for all of their patients and providers in Boston generated nearly all prescriptions for chronic medications and some for acute medications using the computerized prescribing systems in place at their site. Many patients also received care from providers, including specialists, in other practices that did not use the computerized prescribing systems. The computerized prescribing intervention period represented a major systems change in the practices and included several features expected to reduce the frequency of preventable and potential ADEs (Fig. 1).23,24 The application provided physicians with a menu of medications from the formulary, default doses, and a range of potential doses for each medication. Physicians were required to enter dosage, route, and frequency for all orders. Also, computerized prescribing ensured all orders were legible and included the prescribing physician’s signature. For a number of medications, the system displayed relevant laboratory results (e.g., potassium levels when the provider ordered furosemide) at the time of ordering. Both systems provided drug-allergy, drug-laboratory, and drug-drug interaction checks. The system in Indianapolis provided some consequent orders related to drug-laboratory checking, which are orders that should follow from other orders (e.g., anticonvulsant levels when selected anticonvulsants were ordered). The system in Boston provided default dosing and some therapeutic duplication alerting. We standardized the medication-related basic decision support to the extent possible across the 2 sites, although the presentation layers differed.
End Points We defined an ADE as harm associated with the administration of a drug and preventable ADEs were those due to a medication error (ME) that the provider could have entirely avoided. Our primary end points were the number of preventable ADEs 25 (Fig. 2). The reviewers based their decisions about whether an ADE was preventable on the physician's presumed knowledge at the time they prescribed the drug. If insufficient information was available, the reviewers assumed that the physician's decision was correct. Our secondary end points were the number of potential ADEs, which we defined a potential ADE as an error in the medication use process, including prescribing, transcribing, administering, and monitoring steps.27
ADE Detection We previously described methods to determine the incidence of ADEs and provide only a brief overview here.28 We drew on rules used in Regenstrief Institute’s and Brigham and Women’s Hospital’s previous inpatient and outpatient ADE studies to develop the 122 specific criteria used in our study. These criteria fell into 2 broad categories: those based on structured results such as coded laboratory tests and those based on symptoms mentioned in a provider’s notes. We expressed these criteria as database queries used to create the list of cases or candidate ADEs, which we call signals. After identifying these signals, we removed all duplicates and randomly sampled signals for review. Data analysts imported the randomly selected signals into a database. Pharmacists and nurses blinded to whether the event occurred with or without the use of computerized prescribing reviewed and adjudicated ADEs and potential ADEs, in consultation with physicians as necessary. The same pool of reviewers evaluated events from before and after intervention periods. The ability of our approach to capture medication errors not linked to patient harm was limited. © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Computerized Prescribing and Preventable ADEs
FIGURE 1. A comparison of the functions available at the time of the study in Boston and Indianapolis. Computerized prescribing systems using Bell's framework.25 Full circles indicate the functional capability was robustly present, half circles that the functional capability partially exists, and empty circles indicate that the functional capability was missing. An NA indicates that the functional capability was not applicable.
ADE Classification Reviewers used all available data to classify potential and preventable ADE severity as (fatal, life-threatening, serious, or significant) and the cause of preventable ADEs, which we classified as patient education issues, providers ignoring CDSS feedback, or CDSS knowledge base issues.29
separately. We measured events per 10,000 patient-months to account for variation over time in the number of patients seen. Because the implementation of computerized prescribing changed how physicians documented care and provided decision support to improve monitoring, both of which could result in detecting
Sample Size Estimates In previous work, we found 864 ADEs in 88,514 patient visits, of which, 329 (38%) were judged preventable.30 Our primary end point of preventable ADEs represented a rate of 329 preventable ADEs per 88,514 visits (0.37%). To demonstrate a 50% reduction in this rate, which we expected based on reductions seen in inpatient environments, with a 90% power and a 2-sided type I error of 5%, we needed 17,000 patient visits in both the intervention and control arms of the study. Although we did not have a measure of the correlation between ADEs among visits to the same physician, such a correlation would reduce the power of the study; therefore, we inflated the sample sizes by 20% to compensate for correlated responses. This brought the sample size to 20,400 visits per arm.
Analysis Because of differences in patient population and computerized prescribing systems at the 2 sites, we analyzed them © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 2. Diagrammatic representation of the relationship between adverse drug events (ADEs), medical errors, and preventable ADEs.26 www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
71
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
TABLE 1. Patient Characteristics Characteristics
Boston
Indianapolis
Sex (% male) Age in years (mean ± SD) Race % Black % White
28.1 48.1 ± 16.7
30.6 46.7 ± 15.5
22 59
40† 53
†
P < 0.05.
more ADEs, we also analyzed the change in rates with adjustments for the number of nonpreventable ADEs (the number of preventable ADEs subtracted from the total number of total ADEs), which we would not expect to be affected by implementation of computerized prescribing. Assuming that the number of events followed a Poisson distribution and tested for a change in events rates between the before and after periods by using a normal approximation of the Poisson. For our primary analyses, we assumed, as a null hypothesis, that there would be no change in the event rate over time. For analysis of secondary outcomes, we estimated the relative change in ADEs over the same period, assumed that this change reflected natural temporal trends, and used this estimated change as the null hypothesis.
105% increase in both preventable ADEs and (significantly) potential ADEs in Boston. The intervention was not associated with a significant change in symptom-based preventable ADEs (Table 4) in Indianapolis or Boston, or potential ADEs in Boston, but was associated with a significant 311% increase in potential ADEs in Indianapolis. There were no significant changes in serious or life-threatening events at either site. The most common cause of preventable ADEs was patient education issues (34% in Indianapolis and 44% in Boston), leading patients to take the wrong dose or to fail to undergo follow-up testing in some cases. A smaller proportion of preventable ADEs were due to physicians ignoring feedback from CDSS (33% in Indianapolis, and 28% in Boston). Finally, in a similar number of cases, the computerized knowledge base was incomplete, such as failing to include drug-age checking (33% in Indianapolis and 28% in Boston). Because the order entry system prompted providers to monitor certain laboratory tests to improve patient safety, increased surveillance could have increased the number of ADEs detected. We observed a 20% increase in the rate of laboratory monitoring tests performed. At the Boston site, the number of adverse events increased, but the preventable adverse events did not significantly change (Tables 2, 3, and 4). In addition, in Indianapolis, computerized prescribing implementation dramatically increased the quantity of electronic documentation available, which could have increased the number of symptom ADEs detected.
RESULTS
DISCUSSION
We studied 88 physicians in 6 primary care practices in Indiana and 140 physicians in 11 practices in Boston. There were 9128 patients in Indianapolis and 41,819 patients in Boston in the preintervention period. In the postintervention period, there were 9882 patients in Indianapolis and 42,180 patients in Boston. Patient age and sex were similar between study sites, except that there were a higher proportion of black patients in Indianapolis (Table 1). As shown in Table 2, total measured ADEs increased significantly by 112% in Indianapolis but did not change significantly in Boston (19%). Computerized prescribing was associated with a nonsignificant reduction in total preventable ADEs by 25% in Indianapolis and 2% in Boston. Compared with Boston practices, the rate of potential ADEs was more than 7-fold greater in Indianapolis (6.4/10,000 patient-months vs. 49.5/10,000 patientmonths, P < 0.001). Computerized prescribing was associated with a 56% decrease in the potential ADE rate in Indianapolis (49.5 to 21.9/10,000 patient-months, P < 0.001) but a 104% increase in Boston (6.4 to 13.0/10,000 patient-months, P < 0.001). Adverse drug events based on structured results are shown in Table 3. In this structured data subset, the intervention was associated with a significant reduction in preventable ADEs by 85% and potential ADEs by 71% in Indianapolis but was associated with a
Computerized prescribing with basic decision support to order medications was associated with a reduced rate of preventable and potential ADEs in ambulatory primary care practices associated with one medical center but not another. Absolute rates were much higher in Indianapolis than in Boston, even postintervention. The same pattern was evident for serious and life-threatening events. Some possible explanations for the differences in the ADE rate, including factors related to the providers, patient populations, and computerized prescribing systems. Provider differences seem unlikely to account for the findings because venues in both cities were associated with tertiary care university medical centers. We found differences in patient populations in terms of race, with Indianapolis sites having a higher percent of black patients. Significant racial differences in health literacy31 contribute to medication errors (such as taking incorrect doses),32,33 these may account for a portion of the preintervention difference in education-related preventable ADEs. Interestingly, the number of preventable errors attributed to patient education was lower in Indianapolis than in Boston potentially because the computerized prescribing system in Indianapolis provided richer patient education tools and coordination of care facilities. More likely significant is heterogeneity between the 2 computerized prescribing systems and the implementation at the sites. Although similar in many ways, there were
TABLE 2. ADEs, Preventable ADEs, and Potential ADEs, Mean and Standard Deviation, Adjusted Per 10,000 Patient-Months of Exposure Preintervention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
36.3 (5.38) 5.5 (1.16) 6.4 (0.60)
44.8 (5.85) 5.4 (1.19) 13.0 (1.03)†
51.5 (5.48) 12.2 (2.56) 49.5 (8.76)
109.3 (7.08)† 9.1 (2.19) 21.9 (3.20)†
†
P < 0.05.
72
www.journalpatientsafety.com
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
J Patient Saf • Volume 12, Number 2, June 2016
Computerized Prescribing and Preventable ADEs
TABLE 3. ADEs, Preventable ADEs, and Potential ADEs Identified from Structured Data Such as Laboratory Results, Allergies, and Medication Orders, Mean and Standard Deviation, Adjusted Per 10,000 Patient-Months of Exposure Preintrevention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
6.46 (1.28) 1.08 (0.29) 6.34 (0.60)
4.74 (0.63) 2.21 (1.03) 13.0 (1.03)†
18.1 (2.01) 6.57 (1.48) 47.5 (6.39)
28.2 (2.53) 1.01 (0.455)† 13.7 (2.19)†
†
P < 0.05.
important differences. First, although we tried to standardize the decision support offered , the system in Indianapolis had a different user interface and offered more decision support at the time of the study than did the Boston system. Important differences included safety alerts based on drug-choice errors (such as allergies), dosage calculation, patient education materials, and corollary orders (such as monitoring tests). Such differences might account for the failure of the intervention in Boston to be associated with a reduction in total ADEs and preventable ADEs, while facilitating such improvement in Indianapolis. Second, providers in Boston frequently chose not to use the system when writing prescriptions for drugs the patient would take for a limited time, such as an antibiotic. Our findings represent a lower bound of the proportion of ADEs that computerized prescribing can prevent. Some patients saw providers, such as specialists, who did not use computerized prescribing; as such, our study missed opportunities to avoid ADEs related to medications prescribed by these providers. Furthermore, our computerized prescribing systems included basic, but not advanced, decision support, such as pregnancy- and agebased alerting and renal function based dosing. Based on our analysis of causes of preventable ADEs during the postcomputerized prescribing period, we would expect advanced decision support (eliminating the knowledge base deficiencies) to prevent onethird of the preventable errors that occurred during the intervention period further reducing the number of in preventable ADEs. Finally, detection of ADEs in the postintervention period might have improved from increased drug therapy monitoring by the computerized prescribing systems.34,35 Although our study provides evidence that using computerized prescribing is associated with a reduction in preventable ADEs, it also highlights the importance of a multifaceted approach to improving patient safety. In our study, the most common causes of preventable ADEs were patient education and providers ignoring feedback from the CDSS. Although both of these factors may be partially addressed through improved system design and implementation—through more artfully crafted delivery of CDS feedback to providers for example—the level of patient engagement
and the culture of safety depend heavily on factors other than computerized prescribing implementation. More than one-half of preventable ADEs in the ambulatory setting result from errors in prescribing.5,6,36 Despite available evidence, less than 25% of physicians in a national survey rated “increasing the use of computers to order drugs and medical tests” as “very effective” for reducing potential ADEs.37 This suggests that physician’s lack “outcomes expectancy” for using computerized prescribing to order medications. Ideally, developers should match computerized prescribing systems functions with the types of errors made when manual prescribing processes are used.38 Even without advanced clinical decision support, computerized prescribing addresses potential sources of error in the prescribing process by creating legible, complete prescriptions. Adding clinical decision support can eliminate other sources of error. The study had a number of limitations, including factors related to the specific practices studied, the systems, and the study design. To improve generalizability, we studied practices with wellestablished computerized prescribing systems in 2 geographically distinct practices. These practices have affiliations with academic medical centers but are likely representative of a growing number of private primary care practices implementing computerized prescribing in that they were striving to provide care efficiently and with high, consistent quality. Nonetheless, the 2 computerized prescribing systems were “home-grown;” neither is commercially available, limiting generalizability. It would have been ideal to randomize half of the practices to computerized prescribing while maintaining the remainder on a paper system. This was not possible due to the complexity of computerized prescribing implementation in large organizations; therefore, we relied on a before-after design. Our results also show that the process of implementation is a real world issue and may present differences between systems in their ability to prevent potential or actual patient harm. Because the before-after comparisons showed improvement at one site, a temporal trend could have accounted for some of the differences. However, we found no evidence for an underlying temporal trend, and nonpreventable ADE rates remained the same.
TABLE 4. ADEs, Preventable ADEs, and Potential ADEs Identified from Symptom-Based Review of Clinical Notes, Mean and Standard Deviation, Adjusted per 10,000 Patient-Months of Exposure Preintervention and Postintervention Boston
Total adverse drug events Preventable adverse drug events Potential adverse drug events
Indianapolis
Pre-CP
Post-CP
Pre-CP
Post-CP
29.9 (5.22) 4.42 (1.16) 0.04 (0.048)
40.1 (5.81) 3.20 (0.97) 0 (0)
33.4 (5.11) 5.56 (0.20) 2.01 (1.19)
81.1 (6.58)† 8.10 (2.02) 8.26 (2.36)†
†
P < 0.05.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.journalpatientsafety.com
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
73
J Patient Saf • Volume 12, Number 2, June 2016
Overhage et al
Our study provides evidence that well-designed computerized prescribing with basic decision support and patient education materials is associated with a reduction in the number of potential and preventable ADEs in ambulatory settings. Because, at least in the near future, the United States’ HIT infrastructure will be heterogeneous (no single electronic health record), each vendor (or developer of home-grown system) must incorporate evidence-based user interaction, user interface, and decision support in the effort to reduce medication errors. We expect reductions in potential and preventable ADEs to translate into important improvements in morbidity, mortality, and cost. ACKNOWLEDGMENT The authors thank the data managers, research assistants, and others who performed important chart review and data management tasks that allowed this work to be completed. REFERENCES 1. IMS Institute for Healthcare Informatics. The use of Medicines in the United States: Review of 2011. Available at: http://www.imshealth.com/ ims/Global/Content/Insights/IMS%20Institute%20for%20Healthcare% 20Informatics/IHII_Medicines_in_U.S_Report_2011.pdf. Accessed October 20, 2012. 2. Schappert SM, Rechtsteiner EA. “Ambulatory medical care utilization estimates for 2007”. Vital Health Stat. 2011;13:1–38. 3. Hutchinson TA, Flegel KM, Kramer MS, et al. “Frequency, severity and risk factors for adverse drug reactions in adult out-patients: a prospective study”. J Chronic Dis. 1986;39:533–542. 4. Gurwitz JH, Field TS, Harrold LR, et al. “Incidence and preventability of adverse drug events among older persons in the ambulatory setting”. JAMA. 2003;289:1107–1116. 5. Gandhi TK, Weingart SN, Borus J, et al. “Adverse drug events in ambulatory care”. N Engl J Med. 2003;348:1556–1564. 6. Hanlon JT, Schmader KE, Koronkowski MJ, et al. “Adverse drug events in high risk older outpatients”. J Am Geriatr Soc. 1997;45:945–948. 7. Johnston D, Pan E, Walker J, et al. Patient Safety in the Physician's Office: Assessing the value of ambulatory CPOE. In: Center for Information Technology Leadership. Oakland, CA: California HealthCare Foundation. 8. Bates DW, Cullen DJ, Laird N, et al. “Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group”. JAMA. 1995;274:29–34. 9. Bates DW. “Using information technology to reduce rates of medication errors in hospitals”. BMJ. 2000;320:788–791. 10. Bates DW, Cohen M, Leape LL, et al. “Reducing the frequency of errors in medicine using information technology”. J Am Med Inform Assoc. 2001;8:299–308. 11. Wolfstadt JI, Gurwitz JH, Field TS, et al. “The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review”. J Gen Intern Med. 2008;23:451–458.
16. Hsiao C-J, Hing E. Use and Characteristics of Electronic Health Record Systems Among Office-Based Physician Practices: United States, 2001–2013. NCHS Data Brief, No. 143. Hyattsville, MD: National Center for Health Statistics; 2014. 17. Steele AW, Eisert S, Witter J, et al. “The effect of automated alerts on provider ordering behavior in an outpatient setting”. PLoS Med. 2005;2:e255. 18. Gandhi TK, Weingart SN, Seger AC, et al. “Impact of Basic Computerized Prescribing on Outpatient Medication Errors and Adverse Drug Events”. J Am Med Inform Assoc. 9(Suppl 6):S48–S49. 19. Kaushal R, Kern LM, Barron Y, et al. “Electronic prescribing improves medication safety in community-based office practices”. J Gen Intern Med. 2010;25:530–536. 20. Teich JM, Glaser JP, Beckley RF, et al. „The Brigham integrated computing system (BICS): advanced clinical systems in an academic hospital environment”. Int J Med Inform. 1999;54:197–208. 21. McDonald CJ, Tierney WM. “The Medical Gopher–a microcomputer system to help find, organize and decide about patient data”. West J Med. 1986;145:823–829. 22. McDonald CJ, Overhage JM, Tierny WM, et al. “The Regenstrief Medical Record System: a quarter century experience”. Int J Med Inform. 1999;54:225–253. 23. Abramson EL, Barron Y, Quaresimo J, et al. “Electronic prescribing within an electronic health record reduces ambulatory prescribing errors”. Jt Comm J Qual Patient Saf. 2011;37:470–478. 24. Spurr CD, Wang SJ, Kuperman GJ, et al. Confirming and Delivering the Benefits of an Ambulatory Electronic Medical Record for an Integrated Delivery System. Boston, MA: TEPR; 2001. 25. Bell DS, Cretin S, Marken RS, et al. “A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities”. J Am Med Inform Assoc. 2004;11:60–70. 26. Gandhi TK, Seger DL, Bates DW. “Identifying drug safety issues: from research to practice.” Int J Qual Health Care. 2000;12:69–76. 27. Overhage JM, Lukes A. “Practical, reliable, comprehensive method for characterizing pharmacists' clinical activities”. Am J Health Syst Pharm. 1999;56:2444–2450. 28. Hope C, Overhage JM, Seger A, et al. “A tiered approach is more cost effective than traditional pharmacist-based review for classifying computer-detected signals as adverse drug events”. J Biomed Inform. 2003;36:92–98. 29. Marimoto T, Gandhi TK, Seger AC, et al. “Adverse drug events and medication errors: detection and classification methods”. Qual Saf Health Care. 2004;13:306–314. 30. Honigman B, Lee J, Rothschild J, et al. “Using computerized data to identify adverse drug events in outpatients”. J Am Med Inform Assoc. 2001;8:254–266. 31. The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. U.S. Department of Education, 2006. Available at: http://www.eric.ed.gov/PDFS/ED493284.pdf. 32. Baker DW, Parker RM, Williams MV, et al. The health care experience of patients with low literacy. Arch Fam Med. 1996;5:329–334. 33. Davis TC, Wolf MS, Bass PF 3rd, et al. “Literacy and miunderstanding prescription drug labels”. Ann Intern Med. 2006;145:887–894.
12. Shamliyan TA, Duval S, Du J, et al. “Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors”. Health Serv Res. 2008;43(1 Pt 1):32–53.
34. Kahn MG, Ranade D. “The impact of electronic medical records data sources on an adverse drug event quality measure”. J Am Med Inform Assoc. 2010;17:185–191.
13. Overhage JM, Tierney WM, Zhou XH, et al. “A randomized trial of “corollary orders” to prevent errors of omission”. J Am Med Inform Assoc. 1997;4:364–375.
35. Nebeker JR, Hoffman JM, Weir CR, et al. “High rates of adverse drug events in a highly computerized hospital”. Arch Intern Med. 2005;165:1111–1116.
14. King WJ, Paice N, Rangrej J, et al. “The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients”. Pediatrics. 2003;112(3 Pt 1):506–509. 15. Devine EB, Hansen RN, Wilson-Norton JL, et al. “The impact of computerized provider order entry on medication errors in a multispecialty group practice”. J Am Med Inform Assoc. 2010;17:78–84.
74
www.journalpatientsafety.com
36. Burnum JF. “Letter: Preventability of adverse drug reactions”. Ann Intern Med. 1976;85:80–81. 37. Rosen AB, Blendon RJ, DesRoches CM, et al. “Physicians' views of interventions to reduce medical errors: does evidence of effectiveness matter?”. Acad Med. 2005;80:189–192. 38. Simon HA. The Science of Design: Creating the Artificial in the Science of the Artificial. 3rd ed. Cambridge, MA: MIT Press; 1996:111–138.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
