ORIGINAL REPORTS
National Incidence of Medication Error in Surgical Patients Before and After Accreditation Council for Graduate Medical Education Duty-Hour Reform Sumeet Vadera, MD,* Sandra D. Griffith, PhD,† Benjamin P. Rosenbaum, MD,‡ Alvin Y. Chan, BA,* Nicolas R. Thompson, MS,§ Varun R. Kshettry, MD,‡ Michael L. Kelly, MD,‡ Robert J. Weil, MD,‖ William Bingaman, MD,‡ and Lara Jehi, MD¶ Department of Neurosurgery, University of California Irvine Medical Center, Orange, California; †Flatiron Health, Inc., New York, New York; ‡Department of Neurosurgery, Cleveland Clinic Foundation, Cleveland, Ohio; §Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, Ohio ‖Department of Neurosurgery, Geisinger Health System, Danville, Pennsylvania; and ¶Department of Neurology, Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, Ohio *
OBJECTIVE: The Accreditation Council for Graduate
Medical Education (ACGME) established duty-hour regulations for accredited residency programs on July 1, 2003. It is unclear what changes occurred in the national incidence of medication errors in surgical patients before and after ACGME regulations. DESIGN: Patient and hospital characteristics for pre and postduty-hour reform were evaluated, comparing teaching and nonteaching hospitals. A difference-in-differences study design was used to assess the association between duty-hour reform and medication errors in teaching hospitals. SETTING: We used the Nationwide Inpatient Sample
database, which consists of approximately annual 20% stratified sample of all the United States nonfederal hospital inpatient admissions. PARTICIPANTS: A query of the database, including 4 years
before (2000-2003) and 8 years after (2003-2011) the ACGME duty-hour reform of July 2003, was performed to extract surgical inpatient hospitalizations (N ¼ 13,933,326). The years 2003 and 2004 were discarded in the analysis to allow for a wash-out period during duty-hour reform (though we still provide medication error rates). RESULTS: The Nationwide Inpatient Sample estimated the total national surgical inpatients (N ¼ 135,092,013) in nonfederal hospitals during these time periods with
Correspondence: Inquiries to Sumeet Vadera, MD, Department of Neurological Surgery, University of California, Irvine, 101 The City Drive S, Orange, CA 92868.; fax: (714) 456-8212; e-mail: [email protected]
68,736,863 patients in teaching hospitals and 66,355,150 in nonteaching hospitals. Shortly after duty-hour reform (2004 and 2006), teaching hospitals had a statistically significant increase in rate of medication error (p ¼ 0.019 and 0.006, respectively) when compared with nonteaching hospitals even after accounting for trends across all hospitals during this period. After 2007, no further statistically significant difference was noted. CONCLUSIONS: After ACGME duty-hour reform, med-
ication error rates increased in teaching hospitals, which diminished over time. This decrease in errors may be related to changes in training program structure to accommodate C 2015 Associaduty-hour reform. ( J Surg 72:1209-1216. J tion of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: Nationwide inpatient sample, Accreditation
Council for Graduate Medical Education, duty hours, medical errors COMPETENCIES: Patient Care, Practice-Based Learning
and Improvement
INTRODUCTION The establishment of the Accreditation Council for Graduate Medical Education (ACGME) duty-hour regulations on July 1, 2003, no longer permitted residents at ACGME accredited programs to stay in the hospital for more than a total of 80 hours per week and 24 hours at a time
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 1209 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2015.05.013
(an additional 6 hours was permitted for education and transfer of care). The mandate also required residents to have 10 hours off between shifts and a 24-hour-period off per week.1 In 2011, the ACGME mandated further limitations upon the amount of time residents are allowed to spend in the hospital.2 The 2003 and 2011 regulations were enacted in direct response to concerns about resident fatigue and patient carerelated errors occurring in teaching hospitals. Several studies conducted on residents during this time demonstrated that prolonged hospital shifts and lack of sleep were associated with increased rates of serious medical errors that led to diminished patient care and safety.3,4 Preventable medication errors, including medication overdose, can cause serious injury or death and have been shown to significantly increase length of stay; all of these were part of the safety concerns to be addressed by the dutyhour regulations.1,3,5-7 The incidence of all adverse drug events (ADEs) have been estimated to be at least as high as 6.5 per 100 encounters in adult patients although it is difficult to assess national rates of medication error owing to several constraints.4 Hospitals generally have a bias toward underreporting ADEs, the literature does not support a single definition of ADE, most studies examine single institutions and are underpowered, the overall rate of ADEs is often very low, and the rate of ADEs varies depending upon the patient population.3,5,6,8-11 As an example, older patients with multiple comorbidities and polypharmacy have a higher risk of ADE than young healthy patients admitted for the same procedure.6 Soon after duty-hour reform was instituted, unintended consequences began to be noted, including an increase in resident handoffs and development or fostering of a more piecemeal approach to work (what some have called a “shift-work mentality”). Unfortunately, studies were not able to show a decrease in resident fatigue or improvement in sleep, and the most concerning studies suggested that duty-hour reform did not result in a significant improvement in the quality or safety of inpatients in teaching hospitals.12-15 Although many effects of the duty-hour reform have been studied across a multitude of specialties and across a variety of procedures, no study has evaluated national incidence rates of medication errors occurring in patients admitted to the hospital for surgical procedures in the current period of duty-hour reform.1,16 Such an investigation would be critical as it would represent an objective, direct, and focused step aimed to measure the change occurring in a key patient-safety determinant with the duty-hour implementation. Our goal was to investigate whether the ACGME duty-hour restrictions affected the incidence of medication errors on a national level. Specifically, we used the Nationwide Inpatient Sample (NIS) database to compare the incidence of medication errors in surgical patients treated at teaching hospitals before and after the ACGME duty-hour restriction was instituted on July 1, 2003. 1210
We hypothesize that medication error rates would decrease, potentially owing in part to the restrictions implemented.
MATERIALS AND METHODS Patients We queried the NIS database for the years 2000 to 2003 (preduty-hour reform period ended in June 2003) and the years 2003 to 2011 (postduty-hour reform period began in July 2003). Although other studies removed 2003 and 2004 and considered this to be a wash-out period, the authors chose to include the data for completeness.17 We chose June 2011 as the endpoint of this study to avoid any uncertainty associated with the second duty-hour implementation, which occurred after this time. The NIS is maintained by the Agency for Healthcare Quality and Research under the Healthcare Cost and Utilization Project. Each year, the NIS consists of an approximate 20% stratified sample of all the United States nonfederal hospital inpatient admissions. The Healthcare Cost and Utilization Project groups all International Classifications of Diseases, Ninth Revision (ICD-9), Clinical Modification procedure codes into 4 classes: I (minor diagnostic), II (minor therapeutic), III (major diagnostic), and IV (major therapeutic). We defined surgical patients as those with at least 1 procedure code belonging to procedure class III or IV. (http://www.hcup-us.ahrq.gov/toolssoftware/procedure/pro cedure.jsp for more information.) A medication error was determined by the presence of 1 or more of the ICD-9 diagnosis codes that described medication errors. We aimed to omit patients with accidental or other drug overdoses occurring outside the hospital by excluding: (1) patients who had the procedure code for gastric lavage (963.3), as this procedure is typically done in overdose cases occurring outside the hospital and far less commonly as the result of an inpatient medication administration error and (2) patients who had a total of 3 or fewer diagnosis codes for an admission that were all medication errors, as these were suspected to have been presented to the hospital after incurring an ADE outside the hospital. Between 2000 and 2011, there were a total of 27,966,455 inpatient admissions involving an operating procedure (Agency for Healthcare Quality and Research procedure class III or IV). We excluded 5134 patients who had gastric lavage. We then excluded an additional 3 patients who had 3 or fewer ICD-9 codes for an admission that were all medication errors. We then excluded an additional 18,081 patients who had 1 of our predefined suicide codes. We also excluded an additional 1820 patients who had 1 of our predefined “unclear” codes. This leaves us with a total dataset of 27,941,417 patient admissions.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
Data Analysis For all analyses, we used the discharge-level weights provided in the NIS database to account for the hospitalstratified sampling scheme. All statistics computed are estimates of the entire population of all surgical cases in hospitals sampled in the NIS over the time periods of interest. We used the R survey package, version 3.28-2.18-20 Estimates of the unadjusted rates of medication errors were computed for teaching and nonteaching hospitals for each year. (The NIS database designated hospitals as having a teaching status if it had an American Medical Association– approved residency program, was a member of the Council of Teaching Hospitals, or had full-time equivalent interns and residents attending at least 1 out of every 4 beds. Any hospital that did not fulfill any of these criteria was designated as a nonteaching hospital.) Summary statistics for patients and hospital characteristics were computed for patient’s pre and postduty-hour regulations within teaching hospitals and nonteaching hospitals. We further stratified patients into whether or not they had a medication error within the time period and hospital teaching status. Student’s t-tests and chi-square tests were used to determine if there were significant differences between groups. A p o 0.05 was considered significant. To determine whether the change in duty-hour restrictions was associated with increased medication error rates, we used the multiple time series, also known as differencein-differences, research design. This study design has been used in other studies as a means to reduce potential bias from unmeasured variables.14 This design allows for evaluation of underlying trends in medication error incidence over time, but also examines whether trends deviate between teaching and nonteaching hospitals in the postreform time period. The design compares each group of patients— those in teaching hospitals and those in nonteaching hospitals—among themselves, before and after duty-hour reform, allowing for secular trends that were common to all hospitals (e.g., implementation of electronic medical records and nonphysician-related medication errors). The design assumes that prereform trends in medication errors, adjusted for patient and hospital characteristics, were similar between teaching and nonteaching hospitals. Under the difference-in-differences study design, we created a multivariable logistic regression model where presence/absence of at least one medication error was the response variable. The effect of change in duty-hour regulations was measured as the coefficient of the interaction terms created by taking the product of the hospital teaching status variable and indicator variables for each of the postreform years (July 2003-2011). To test the assumption that covariate-adjusted prereform trends in medication errors were similar between teaching and nonteaching hospitals, we performed a falsification test or test of controls.14 We included variables for teaching
status interacting with indicator variables for the predutyhour regulation year 2001 and June 2003. A statistically significant odds ratio (OR) for either of these interaction terms would indicate different medication error trends between teaching and nonteaching hospitals. We determined before the analysis that if the falsification test yielded significant results, we would compare each hospital by postyear interaction terms to only the last year before duty-hour reform (2002) instead of the full 3 years of prereform data. We used the Elixhauser comorbidity system to adjust for previous medical conditions by including a variable for the summary score developed by van Walraven et al.21,22 We treated this summary score as a continuous variable. A recent study showed that such derived comorbidity scores are mathematically valid and that the Elixhauser summary score derived by van Walraven performed well in a Surveillance, Epidemiology, and End Results -Medicare data example.22 We further adjusted for the following patient- and hospital-level covariates: patient age (years), length of hospital stay (days), patient race (White, African-American, and Other), patient gender (male or female), patient median income by zip code (low, low-mid, mid-high, and high), primary payer (Medicare, Medicaid, private, and self-pay/ other), admission source (routine, emergency, and other), weekend admission (yes or no), hospital size (small, medium, or large), and hospital region (Northeast, Midwest, South, or West).
RESULTS Patient/Hospital Characteristics There were 13,933,326 patient admissions in the NIS for years 2000 to June 2003 and July 2003 to 2011 involving at least 1 Class III or IV procedure code of the NIS procedure classes. Extrapolating to the entire national population of these cases, there were an estimated 135,092,013 such surgical cases in the United States nonfederal hospitals for these years (68,736,863 from teaching hospitals and 66,355,150 from nonteaching hospitals). We performed a univariate analysis to determine summary statistics related to presence of medication errors in teaching and nonteaching hospitals. Regardless of hospital teaching status and time period, patients with medication errors (1) had significantly higher mean age, (2) had significantly longer mean length of hospital stay (over twice as long for each group), (3) had significantly higher mean Elixhauser summary score, (4) were less likely to be classified as self-pay, (5) were more likely to have been admitted from the emergency room, (6) were more likely to have been admitted on a weekend, (7) were more likely to be admitted to a hospital in the West region, and (8) were less likely to have a routine disposition at the time of discharge.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1211
TABLE 1. p Values for the Multivariable Logistic Regression Model for Presence of Medication Error in Teaching Hospitals Prereform, Medication Error Sample size: N (n) Age: median (IQR) Length of stay: median (IQR) Elixhauser score: median (IQR) Admission source Routine Emergency department Other hospital/ health facility Admission type Emergency Elective Other admission type Weekend admission Disposition Routine discharge Transfer Home health care Against medical advice Died in hospital
3582 (728) 52 (37-70)
Prereform, No Medication Error p Value 18,909,607 (3,812,023) 50 (29-68)
Postreform, Medication Error 17,766 (3593)
o0.0001
55 (42-68)
Postreform, No Medication Error
p Value
49,805,908 (10,116,982) 51 (29-67)
o0.0001
7 (4-13)
3 (2-6)
o0.0001
7 (4-13)
3 (2-6)
o0.0001
0 (0-5)
0 (0-2)
o0.0001
3 (0-8)
0 (0-3)
o0.0001
43.1 46.2
72.7 21.2
o0.0001
43.6 46.5
72.2 22.0
o0.0001
10.6
6.0
9.8
5.8
49.5 32 18.6
23.7 49.2 27.1
0.1597
53.3 29.0 17.7
25.0 48.3 26.7
o0.0001
19.6
12.1
o0.0001
18.8
12.2
o0.0001
57.7
79.1
0.0010
49.5
74.6
o0.0001
20 12.1
10.0 8.1
30.0 15.7
10.0 11.6
0.4
0.1
1.0
0.2
5.1
1.6
5.3
1.4
“N” is the estimated total number of admissions in population represented by the NIS and “n” is sample size represented by actual admissions in the NIS. IQR, interquartile range.
Medication Error Rates The results of the multivariable logistic regression model are displayed in Tables 1 and 2 and the Figure. The test of controls (“the falsification test”) revealed no statistically significant divergent trends in teaching vs nonteaching hospitals for the prereform period (2001: OR ¼ 1.31, 95% CI ¼ 0.89-1.92, p ¼ 0.167; 2002: OR ¼ 1.41, 95% CI ¼ 0.92-2.15, p ¼ 0.12). Therefore, we used the entire prereform period as the reference group when calculating the ORs for each of the postreform years. Older patients, patients having longer length of hospital stay, and patients with higher Elixhauser summary score were more likely to have a medication error. Patients were more likely to have a medication error if their admission source was anything other than routine. Weekend admissions had slightly more occurrences of medication errors. The Figure displays a plot of adjusted medication error rates for each year in our study, stratified by hospital teaching status. The difference-in-differences study design allowed us to compare teaching hospitals after institution of duty hours with preduty-hour reform, and also generate 1212
an estimated medication error rate for teaching hospitals after duty-hour reform (diagramed in the Figure as the counterfactual line). Using counterfactual variables, we can measure correlation via surrogate markers.23 Generally, a counterfactual model postulates that all values for both surrogate “S” and the primary outcome “T” for a subject under different interventions are “Z.” Then we postulate that Z ¼ 0 or Z ¼ 1 for 2 values of S (denoted as S0 and S1), though only 1 could be observed for each subject. Within this counterfactual framework, the population is partitioned into subgroups where everyone within each subgroup has the same observed and counterfactual outcomes, and the proportion of population within the subgroups is a parameter of the model.23 After adjusting for the effects of measured patient and hospital characteristics as well as unmeasured secular trends common to all hospitals, teaching hospitals demonstrated a statistically significant increase in medication error rates compared with prereform years in 2005 (OR ¼ 1.40, 95% CI ¼ 1.031.90, p ¼ 0.029) and 2006 (OR ¼ 1.52, 95% CI ¼ 1.161.99, p ¼ 0.002). From 2007 to 2011, there was no statistically significant difference in rates of medication
Journal of Surgical Education Volume 72/Number 6 November/December 2015
TABLE 2. p Values for the Multivariable Logistic Regression Model for Presence of Medication Error in Nonteaching Hospitals Prereform, Medication Error Sample size: N (n) Age: median (IQR) Length of stay: median (IQR) Elixhauser score: median (IQR) Admission source Routine Emergency department Other hospital/ health facility Admission type Emergency Elective Other admission type Weekend admission Disposition Routine discharge Transfer Home health care Against medical advice Died in hospital
4046 (851) 61 (43-75)
Prereform, No Medication Error 19,060,834 (3,973,344) 50 (29-70)
p Value
Postreform, Medication Error
Postreform, No Medication Error
p Value
15,054 (3098)
47,275,216 (9,707,419) 52 (30-70)
o0.0001
o0.0001
60 (47-73)
6 (4-10)
3 (2-5)
o0.0001
6 (4-11)
3 (2-5)
o0.0001
1 (0-7)
0 (0-0)
o0.0001
3 (0-8)
0 (0-2)
o0.0001
46.3 50.0
73.5 23.2
40.5 55.8
72.6 24.5
o0.0001
3.7
3.2
3.7
2.9
51.2 30.9 17.9
22.6 46.3 31
o0.0001
55.3 28.0 16.7
24.5 47.4 28.1
o0.0001
17.9
12.8
o0.0001
19.0
12.4
o0.0001
55.7
78.7
o0.0001
45.0
74
o0.0001
30.0 9.6
10.0 7.0
30.0 18.0
10.0 10.8
0.3
0.1
0.6
0.1
3.9
1.4
3.5
1.1
0.0118
“N” is the estimated total number of admissions in population represented by the NIS and “n” is sample size represented by actual admissions in the NIS. IQR, interquartile range.
errors in teaching hospitals compared to preduty-hour reform (2007: p ¼ 0.07, 2008: p ¼ 0.22, 2009: p ¼ 0.90, 2010: p ¼ 0.23, 201: p ¼ 0.31). Table 3 lists exact medication error rates. Median values are assumed for age, length of stay, and Elixhauser summary score and reference groups for all categorical variables. The observed incidence of reported medication errors was low, ranging from approximately 1 to 3 occurrences for every 10,000 surgical hospitalizations. Medication error rates were lower in teaching hospitals than nonteaching hospitals before implementation of duty-hour regulations. In both teaching and nonteaching hospitals, medication error rates had increased in the postreform years, with teaching hospitals having a greater increase.
DISCUSSION Preventable medication errors, specifically medication overdose, cause significant morbidity and mortality and have been shown to increase hospital length of stay and increase cost of care.6,24,25 The institution of the ACGME
duty-hour reform for residents in July 2003 was designed, in part, to address concerns about preventable errors as well as to help improve patient care and foster resident learning.12 Causes for medication errors vary and include increased age, multiple comorbidities, increased length of stay, and complicated medication regimens.5-11,26-28 Contrary to our hypothesis, we found that medication rates increased initially in teaching hospitals after the hour restrictions were introduced. Although preventable medication errors are caused by many factors that are impossible to measure at the national level, the difference-in-differences study design reduces the effect of these variables when evaluating pre and postduty-hour reform in teaching and nonteaching hospitals. If all other factors remain unchanged in these hospitals over time, the incidence of medication errors for teaching hospitals should look similar to the counterfactual line in the Figure. Interestingly, the actual incidence of medication errors initially spiked in teaching hospitals after institution of the duty-hour reform, but from 2007 to 2011, this difference diminished and was not statistically significant. There are several potential explanations for why medication errors may have initially increased after the
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1213
FIGURE. Adjusted medication error rates by year, stratified by teaching status. The gray dashed line represents the assumed counterfactual of estimated medication error rates in teaching hospitals. Median values are assumed for age, length of stay, and Elixhauser summary score and reference groups for all categorical variables.
institution of duty-hour reform in teaching hospitals. The work-hour limitation may have caused an increase in patient “handoffs” between residents, resulting in increased risk of medication error. Older patients with complex medical comorbidities urgently admitted for nonelective procedures would be particularly vulnerable to a “shift” mentality with frequent handoffs. Furthermore, some studies have suggested that duty hours have had no positive effect on resident fatigue and amount of sleep: this may be another explanation for why medication errors showed an uptrend as residents were faced with less time to do the same amount of work while under the same effects of fatigue.12,14,15 On the contrary, the decrease in error rates from 2007 to 2011 may reflect long-term results of changes that were enacted within residency training programs to accommodate the duty-hour reform and improve residency training. Changes to rotation schedules, such as introduction of a night float rotation, increased oversight of junior residents by senior residents, and improvements to patient handoffs tools have all been incorporated into training programs to comply with duty-hour restrictions and may have played a role in this decrease.29,30
Owing to the nature of the NIS database there are several limitations to this study. Because the NIS database is retrospective and relies on hospital coders inputting physician documentation, there is a risk of sampling bias with underreporting of medication errors. In addition, medication errors with no sequelae are unlikely to be reported or documented and thus might not be included in this database. As previously mentioned, nonphysician causes for medication errors exist and it is impossible to separate these data points from physician medication errors in the NIS, but it is our belief that a large sample such as this allows for some mitigation of these influences on the data. Use of the difference-in-differences study design also reduces the effect of other measurable or nonmeasurable factors that could affect the results. Furthermore, there could have been other outside trends that contributed to this change in medication errors. For example, there has been a large increase in the number of hospitalists since the term was described by Wachter and Goldman.31,32 There was also a concern that fragmented care caused by this increased number of hospitalists could lead to higher numbers of medication errors33; thus, there is
TABLE 3. Medication Error Rates per 100,000 Cases Teaching Nonteaching Counterfactual
'00
'01
'02
'03a
'03b
'04
'05
'06
'07
'08
'09
'10
'11
10.9 12.3
10.0 11.3
9.6 10.8
10.8 12.1
14.6 13.9 12.3
13.8 11.5 10.2
15.6 13.4 11.8
17.6 14.0 12.4
16.8 15.0 13.3
21.6 19.9 17.6
18.2 21.0 18.6
20.8 17.7 15.7
19.5 18.7 16.5
Medication error rates per 100,000 cases for each year. 1214
Journal of Surgical Education Volume 72/Number 6 November/December 2015
a possibility that this could have contributed to our results. Although we are not aware of any other trends occurring during the time period we analyzed, it is entirely possible that they existed. We stress that our results were likely owing to a variety of influencing events rather than a specific trend. For example, it is possible that other team members (e.g., nurses) played a role in medication errors. Moreover, these trends may have influenced each other. Saint and Flanders note that with the ACGME duty-hour restrictions, hospitals may have turned to hospitalists as a solution to resident duty work problems.32 Therefore, different trends may have interacted or influenced each other that caused our results. The data could potentially be biased by increased reporting in teaching hospitals during this time frame because of an increase in emphasis on reporting ADEs. The use of the difference-in-differences methodology has its own drawbacks because if there are diverging trends that occur during this time frame, it is conceivable that we have found 2 separate issues that occur at similar times that may be unrelated but that are effecting medication errors. Although this is a valid argument and there are flaws inherent to the statistical design and database we used, this is the largest study to examine national levels of medication errors and correlate institution of duty hours. Also, the fact that there was a significant increase in July 2003 and 2004 in medication errors specifically in teaching hospitals also lends support to our hypothesis that medication errors do correlate with introduction of duty hours. It is difficult to explain such a focal jump in medication errors in teaching hospitals with another explanation. There was a large difference in medication rates between the year 2000 and 2011 (Figure). We are unsure of an explanation. It is possible that there was actually no difference, and that reporting of medication errors simply improved. Further research is necessary to provide a conclusive explanation. The other results obtained from this study agree with the literature and are intuitive.27 Older patients, especially those with higher Elixhauser summary scores, have higher comorbidities and require more medications, lending themselves to a higher risk of medication error. As mentioned earlier, medication errors are associated with an increased length of stay, although it is not possible to know from the database whether these patients had a medication error that required extended hospital stay or if the extended hospitalization put the patient at higher risk for medication error. Emergency and weekend admissions have also been shown to have a higher risk of incurring medication errors, which we showed as well, and they may be related to increased acuity of the patients when they are admitted.11,27 These features suggest that use of the NIS is a valid and powerful tool to examine the frequency and changes in time of medication errors in surgical patients in the United States.
CONCLUSION We identified a significant increase in medication errors in patients undergoing surgery at teaching hospitals after institution of duty-hour reform, which was mitigated over time. This increase may have potentially been related to the dutyhour restrictions. Although correlation does not imply causation, this study adds to the growing body of literature examining changes in patient care that have evolved during the era of duty-hour reform. The decrease in medication errors that occurred from 2007 to 2011 may be related to several factors, but changes in other aspects of residency training to accommodate for duty-hour restrictions may play a role as well. Development of sustainable and reliable tools for patient handoffs, changes to resident rotation schedules in teaching hospitals, and increased oversight may all have played a factor in the reduction of preventable errors as well as help improve patient care and foster resident learning.
REFERENCES 1. Bhavsar J, Montgomery D, Li J, et al. Impact of duty
hours restrictions on quality of care and clinical outcomes. Am J Med. 2007;120(11):968-974. 2. Pepper DJ, Schweinfurth M, Herrin VE. The effect of
new duty hours on resident academic performance and adult resuscitation outcomes. Am J Med. 2014;127 (4):337-342. 3. Lazarou J, Pomeranz BH, Corey PN. Incidence of
adverse drug reactions in hospitalized patients: a metaanalysis of prospective studies. J Am Med Assoc. 1998;279(15):1200. 4. Kaushal R. Medication errors and adverse drug events
in pediatric inpatients. J Am Med Assoc. 2001;285 (16):2114. 5. De Boer M, Boeker EB, Ramrattan MA, et al. Adverse
drug events in surgical patients: an observational multicentre study. Int J Clin Pharm. 2013;35(5):744-752. 6. Klopotowska JE, Wierenga PC, Stuijt CCM, et al.
Adverse drug events in older hospitalized patients: results and reliability of a comprehensive and structured identification strategy. PLoS ONE. 2013;8(8): e71045. 7. Hoonhout LH, de Bruijne MC, Wagner C, et al.
Nature, occurrence and consequences of medicationrelated adverse events during hospitalization: a retrospective chart review in the Netherlands. Drug Saf. 2010;33(10):853-864. 8. Keers RN, Williams SD, Cooke J, Ashcroft DM. Causes
of medication administration errors in hospitals: a
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1215
systematic review of quantitative and qualitative evidence. Drug Saf. 2013;36(11):1045-1067. 9. Weiss AJ, Elixhauser A. Characteristics of adverse drug
events originating during the hospital stay. Available at: 〈http://www.ncbi.nlm.nih.gov/books/NBK174680〉; 2011 Accessed 19.07.14. 10. Shah R, Blustein L, Kuffner E, Davis L. Communicating
21. Elixhauser A, Steiner C, Harris DR, Coffey RM.
Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. 22. Van Walraven C, Austin PC, Jennings A, Quan H,
Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6): 626-633.
doses of pediatric liquid medicines to parents/caregivers: a comparison of written dosing directions on prescriptions with labels applied by dispensed pharmacy. J Pediatr. 2014;164(3):596-601 e1.
23. Taylor JMG, Wang Y, Thiebaut R. Counterfactual
11. Wahr JA, Shore AD, Harris LH, et al. Comparison of
24. Rottenkolber D, Hasford J, Stausberg J. Costs of
intensive care unit medication errors reported to the United States’ MedMarx and the United Kingdom’s National Reporting and Learning System: a crosssectional study. Am J Med Qual. 2014;29(1):61-69. 12. Nasca TJ, Day SH, Amis ES. The new recommenda-
tions on duty hours from the ACGME task force. N Engl J Med. 2010;363(2):e3. 13. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects
of the Accreditation Council for Graduate Medical Education duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250-258. 14. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality
among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. J Am Med Assoc. 2007;298(9):975-983.
15. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality
link to the proportion of treatment effect explained by a surrogate marker. Biometrics. 2005;61:1102-1111. adverse drug events in german hospitals—a microcosting study. Value Health. 2012;15(6):868-875. 25. Al-Rowibah FA, Younis MZ, Parkash J. The impact of
computerized physician order entry on medication errors and adverse drug events. J Health Care Finance. 2013;40(1):93-102. 26. Kane-Gill SL, Wytiaz NP, Thompson LM, et al. A
real-world, multicenter assessment of drugs requiring weight-based calculations in overweight, adult critically ill patients. Sci World J. 2013;2013:1-9. 27. Budnitz DS, Lovegrove MC, Shehab N, Richards CL.
Emergency hospitalizations for adverse drug events in older Americans. N Engl J Med. 2011;365(21): 2002-2012. 28. Eginger KH, Yarborough LL, Inge LD, et al. Medi-
among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. J Am Med Assoc. 2007;298(9):984-992.
cation errors in HIV-infected hospitalized patients: a pharmacist’s impact. Ann Pharmacother. 2013;47(78):953-960.
16. Norby K, Siddiq F, Adil MM, Haines SJ. The effect of
29. George BP, Probasco JC, Dorsey ER, Venkatesan A.
duty hour regulations on outcomes of neurological surgery in training hospitals in the United States: duty hour regulations and patient outcomes. J Neurosurg. 2014;121:1-15.
30. Reed DA, Fletcher KE, Arora VM. Systematic review:
17. Dumont TM, Tranmer BI, Horgan MA, Rughani AI.
Trends in neurosurgical complication rates at teaching vs nonteaching hospitals following duty-hour restrictions. Neurosurg. 2012;71(5):1041-1046. 18. Lumley T. Analysis of complex survey samples. J Stat
Softw. 2004;9(1):1-19. 19. Lumley T. Survey: Analysis of Complex Survey
Impact of 2011 resident duty hour requirements on neurology residency programs and departments. Neurohospitalist. 2014;4(3):119-126. association of shift length, protected sleep time, and night float with patient care, residents’ health, and education. Ann Intern Med. 2010;153(12):829-842. 31. Wachter RM, Goldman L. The emerging role of
hospitalists in the American health care system. N Engl J Med. 1996;335:514-517. 32. Saint S, Flanders SA. Hospitalists in teaching hospitals:
Samples. Available at: 〈http://cran.r-project.org/web/ packages/survey/index.html〉.
opportunities but not without danger. J Gen Intern Med. 2004;19(4):392-393.
20. R Core Team. A Language and environment for stat-
33. Sox HC. The hospitalist model: perspectives of the
istical computing. Available at: 〈http://www.R-project. org/〉.
1216
patient, the internist, and internal medicine. Ann Intern Med. 1999;130:368-372.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
National Incidence of Medication Error in Surgical Patients Before and After Accreditation Council for Graduate Medical Education Duty-Hour Reform Sumeet Vadera, MD,* Sandra D. Griffith, PhD,† Benjamin P. Rosenbaum, MD,‡ Alvin Y. Chan, BA,* Nicolas R. Thompson, MS,§ Varun R. Kshettry, MD,‡ Michael L. Kelly, MD,‡ Robert J. Weil, MD,‖ William Bingaman, MD,‡ and Lara Jehi, MD¶ Department of Neurosurgery, University of California Irvine Medical Center, Orange, California; †Flatiron Health, Inc., New York, New York; ‡Department of Neurosurgery, Cleveland Clinic Foundation, Cleveland, Ohio; §Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, Ohio ‖Department of Neurosurgery, Geisinger Health System, Danville, Pennsylvania; and ¶Department of Neurology, Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, Ohio *
OBJECTIVE: The Accreditation Council for Graduate
Medical Education (ACGME) established duty-hour regulations for accredited residency programs on July 1, 2003. It is unclear what changes occurred in the national incidence of medication errors in surgical patients before and after ACGME regulations. DESIGN: Patient and hospital characteristics for pre and postduty-hour reform were evaluated, comparing teaching and nonteaching hospitals. A difference-in-differences study design was used to assess the association between duty-hour reform and medication errors in teaching hospitals. SETTING: We used the Nationwide Inpatient Sample
database, which consists of approximately annual 20% stratified sample of all the United States nonfederal hospital inpatient admissions. PARTICIPANTS: A query of the database, including 4 years
before (2000-2003) and 8 years after (2003-2011) the ACGME duty-hour reform of July 2003, was performed to extract surgical inpatient hospitalizations (N ¼ 13,933,326). The years 2003 and 2004 were discarded in the analysis to allow for a wash-out period during duty-hour reform (though we still provide medication error rates). RESULTS: The Nationwide Inpatient Sample estimated the total national surgical inpatients (N ¼ 135,092,013) in nonfederal hospitals during these time periods with
Correspondence: Inquiries to Sumeet Vadera, MD, Department of Neurological Surgery, University of California, Irvine, 101 The City Drive S, Orange, CA 92868.; fax: (714) 456-8212; e-mail: [email protected]
68,736,863 patients in teaching hospitals and 66,355,150 in nonteaching hospitals. Shortly after duty-hour reform (2004 and 2006), teaching hospitals had a statistically significant increase in rate of medication error (p ¼ 0.019 and 0.006, respectively) when compared with nonteaching hospitals even after accounting for trends across all hospitals during this period. After 2007, no further statistically significant difference was noted. CONCLUSIONS: After ACGME duty-hour reform, med-
ication error rates increased in teaching hospitals, which diminished over time. This decrease in errors may be related to changes in training program structure to accommodate C 2015 Associaduty-hour reform. ( J Surg 72:1209-1216. J tion of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: Nationwide inpatient sample, Accreditation
Council for Graduate Medical Education, duty hours, medical errors COMPETENCIES: Patient Care, Practice-Based Learning
and Improvement
INTRODUCTION The establishment of the Accreditation Council for Graduate Medical Education (ACGME) duty-hour regulations on July 1, 2003, no longer permitted residents at ACGME accredited programs to stay in the hospital for more than a total of 80 hours per week and 24 hours at a time
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 1209 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2015.05.013
(an additional 6 hours was permitted for education and transfer of care). The mandate also required residents to have 10 hours off between shifts and a 24-hour-period off per week.1 In 2011, the ACGME mandated further limitations upon the amount of time residents are allowed to spend in the hospital.2 The 2003 and 2011 regulations were enacted in direct response to concerns about resident fatigue and patient carerelated errors occurring in teaching hospitals. Several studies conducted on residents during this time demonstrated that prolonged hospital shifts and lack of sleep were associated with increased rates of serious medical errors that led to diminished patient care and safety.3,4 Preventable medication errors, including medication overdose, can cause serious injury or death and have been shown to significantly increase length of stay; all of these were part of the safety concerns to be addressed by the dutyhour regulations.1,3,5-7 The incidence of all adverse drug events (ADEs) have been estimated to be at least as high as 6.5 per 100 encounters in adult patients although it is difficult to assess national rates of medication error owing to several constraints.4 Hospitals generally have a bias toward underreporting ADEs, the literature does not support a single definition of ADE, most studies examine single institutions and are underpowered, the overall rate of ADEs is often very low, and the rate of ADEs varies depending upon the patient population.3,5,6,8-11 As an example, older patients with multiple comorbidities and polypharmacy have a higher risk of ADE than young healthy patients admitted for the same procedure.6 Soon after duty-hour reform was instituted, unintended consequences began to be noted, including an increase in resident handoffs and development or fostering of a more piecemeal approach to work (what some have called a “shift-work mentality”). Unfortunately, studies were not able to show a decrease in resident fatigue or improvement in sleep, and the most concerning studies suggested that duty-hour reform did not result in a significant improvement in the quality or safety of inpatients in teaching hospitals.12-15 Although many effects of the duty-hour reform have been studied across a multitude of specialties and across a variety of procedures, no study has evaluated national incidence rates of medication errors occurring in patients admitted to the hospital for surgical procedures in the current period of duty-hour reform.1,16 Such an investigation would be critical as it would represent an objective, direct, and focused step aimed to measure the change occurring in a key patient-safety determinant with the duty-hour implementation. Our goal was to investigate whether the ACGME duty-hour restrictions affected the incidence of medication errors on a national level. Specifically, we used the Nationwide Inpatient Sample (NIS) database to compare the incidence of medication errors in surgical patients treated at teaching hospitals before and after the ACGME duty-hour restriction was instituted on July 1, 2003. 1210
We hypothesize that medication error rates would decrease, potentially owing in part to the restrictions implemented.
MATERIALS AND METHODS Patients We queried the NIS database for the years 2000 to 2003 (preduty-hour reform period ended in June 2003) and the years 2003 to 2011 (postduty-hour reform period began in July 2003). Although other studies removed 2003 and 2004 and considered this to be a wash-out period, the authors chose to include the data for completeness.17 We chose June 2011 as the endpoint of this study to avoid any uncertainty associated with the second duty-hour implementation, which occurred after this time. The NIS is maintained by the Agency for Healthcare Quality and Research under the Healthcare Cost and Utilization Project. Each year, the NIS consists of an approximate 20% stratified sample of all the United States nonfederal hospital inpatient admissions. The Healthcare Cost and Utilization Project groups all International Classifications of Diseases, Ninth Revision (ICD-9), Clinical Modification procedure codes into 4 classes: I (minor diagnostic), II (minor therapeutic), III (major diagnostic), and IV (major therapeutic). We defined surgical patients as those with at least 1 procedure code belonging to procedure class III or IV. (http://www.hcup-us.ahrq.gov/toolssoftware/procedure/pro cedure.jsp for more information.) A medication error was determined by the presence of 1 or more of the ICD-9 diagnosis codes that described medication errors. We aimed to omit patients with accidental or other drug overdoses occurring outside the hospital by excluding: (1) patients who had the procedure code for gastric lavage (963.3), as this procedure is typically done in overdose cases occurring outside the hospital and far less commonly as the result of an inpatient medication administration error and (2) patients who had a total of 3 or fewer diagnosis codes for an admission that were all medication errors, as these were suspected to have been presented to the hospital after incurring an ADE outside the hospital. Between 2000 and 2011, there were a total of 27,966,455 inpatient admissions involving an operating procedure (Agency for Healthcare Quality and Research procedure class III or IV). We excluded 5134 patients who had gastric lavage. We then excluded an additional 3 patients who had 3 or fewer ICD-9 codes for an admission that were all medication errors. We then excluded an additional 18,081 patients who had 1 of our predefined suicide codes. We also excluded an additional 1820 patients who had 1 of our predefined “unclear” codes. This leaves us with a total dataset of 27,941,417 patient admissions.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
Data Analysis For all analyses, we used the discharge-level weights provided in the NIS database to account for the hospitalstratified sampling scheme. All statistics computed are estimates of the entire population of all surgical cases in hospitals sampled in the NIS over the time periods of interest. We used the R survey package, version 3.28-2.18-20 Estimates of the unadjusted rates of medication errors were computed for teaching and nonteaching hospitals for each year. (The NIS database designated hospitals as having a teaching status if it had an American Medical Association– approved residency program, was a member of the Council of Teaching Hospitals, or had full-time equivalent interns and residents attending at least 1 out of every 4 beds. Any hospital that did not fulfill any of these criteria was designated as a nonteaching hospital.) Summary statistics for patients and hospital characteristics were computed for patient’s pre and postduty-hour regulations within teaching hospitals and nonteaching hospitals. We further stratified patients into whether or not they had a medication error within the time period and hospital teaching status. Student’s t-tests and chi-square tests were used to determine if there were significant differences between groups. A p o 0.05 was considered significant. To determine whether the change in duty-hour restrictions was associated with increased medication error rates, we used the multiple time series, also known as differencein-differences, research design. This study design has been used in other studies as a means to reduce potential bias from unmeasured variables.14 This design allows for evaluation of underlying trends in medication error incidence over time, but also examines whether trends deviate between teaching and nonteaching hospitals in the postreform time period. The design compares each group of patients— those in teaching hospitals and those in nonteaching hospitals—among themselves, before and after duty-hour reform, allowing for secular trends that were common to all hospitals (e.g., implementation of electronic medical records and nonphysician-related medication errors). The design assumes that prereform trends in medication errors, adjusted for patient and hospital characteristics, were similar between teaching and nonteaching hospitals. Under the difference-in-differences study design, we created a multivariable logistic regression model where presence/absence of at least one medication error was the response variable. The effect of change in duty-hour regulations was measured as the coefficient of the interaction terms created by taking the product of the hospital teaching status variable and indicator variables for each of the postreform years (July 2003-2011). To test the assumption that covariate-adjusted prereform trends in medication errors were similar between teaching and nonteaching hospitals, we performed a falsification test or test of controls.14 We included variables for teaching
status interacting with indicator variables for the predutyhour regulation year 2001 and June 2003. A statistically significant odds ratio (OR) for either of these interaction terms would indicate different medication error trends between teaching and nonteaching hospitals. We determined before the analysis that if the falsification test yielded significant results, we would compare each hospital by postyear interaction terms to only the last year before duty-hour reform (2002) instead of the full 3 years of prereform data. We used the Elixhauser comorbidity system to adjust for previous medical conditions by including a variable for the summary score developed by van Walraven et al.21,22 We treated this summary score as a continuous variable. A recent study showed that such derived comorbidity scores are mathematically valid and that the Elixhauser summary score derived by van Walraven performed well in a Surveillance, Epidemiology, and End Results -Medicare data example.22 We further adjusted for the following patient- and hospital-level covariates: patient age (years), length of hospital stay (days), patient race (White, African-American, and Other), patient gender (male or female), patient median income by zip code (low, low-mid, mid-high, and high), primary payer (Medicare, Medicaid, private, and self-pay/ other), admission source (routine, emergency, and other), weekend admission (yes or no), hospital size (small, medium, or large), and hospital region (Northeast, Midwest, South, or West).
RESULTS Patient/Hospital Characteristics There were 13,933,326 patient admissions in the NIS for years 2000 to June 2003 and July 2003 to 2011 involving at least 1 Class III or IV procedure code of the NIS procedure classes. Extrapolating to the entire national population of these cases, there were an estimated 135,092,013 such surgical cases in the United States nonfederal hospitals for these years (68,736,863 from teaching hospitals and 66,355,150 from nonteaching hospitals). We performed a univariate analysis to determine summary statistics related to presence of medication errors in teaching and nonteaching hospitals. Regardless of hospital teaching status and time period, patients with medication errors (1) had significantly higher mean age, (2) had significantly longer mean length of hospital stay (over twice as long for each group), (3) had significantly higher mean Elixhauser summary score, (4) were less likely to be classified as self-pay, (5) were more likely to have been admitted from the emergency room, (6) were more likely to have been admitted on a weekend, (7) were more likely to be admitted to a hospital in the West region, and (8) were less likely to have a routine disposition at the time of discharge.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1211
TABLE 1. p Values for the Multivariable Logistic Regression Model for Presence of Medication Error in Teaching Hospitals Prereform, Medication Error Sample size: N (n) Age: median (IQR) Length of stay: median (IQR) Elixhauser score: median (IQR) Admission source Routine Emergency department Other hospital/ health facility Admission type Emergency Elective Other admission type Weekend admission Disposition Routine discharge Transfer Home health care Against medical advice Died in hospital
3582 (728) 52 (37-70)
Prereform, No Medication Error p Value 18,909,607 (3,812,023) 50 (29-68)
Postreform, Medication Error 17,766 (3593)
o0.0001
55 (42-68)
Postreform, No Medication Error
p Value
49,805,908 (10,116,982) 51 (29-67)
o0.0001
7 (4-13)
3 (2-6)
o0.0001
7 (4-13)
3 (2-6)
o0.0001
0 (0-5)
0 (0-2)
o0.0001
3 (0-8)
0 (0-3)
o0.0001
43.1 46.2
72.7 21.2
o0.0001
43.6 46.5
72.2 22.0
o0.0001
10.6
6.0
9.8
5.8
49.5 32 18.6
23.7 49.2 27.1
0.1597
53.3 29.0 17.7
25.0 48.3 26.7
o0.0001
19.6
12.1
o0.0001
18.8
12.2
o0.0001
57.7
79.1
0.0010
49.5
74.6
o0.0001
20 12.1
10.0 8.1
30.0 15.7
10.0 11.6
0.4
0.1
1.0
0.2
5.1
1.6
5.3
1.4
“N” is the estimated total number of admissions in population represented by the NIS and “n” is sample size represented by actual admissions in the NIS. IQR, interquartile range.
Medication Error Rates The results of the multivariable logistic regression model are displayed in Tables 1 and 2 and the Figure. The test of controls (“the falsification test”) revealed no statistically significant divergent trends in teaching vs nonteaching hospitals for the prereform period (2001: OR ¼ 1.31, 95% CI ¼ 0.89-1.92, p ¼ 0.167; 2002: OR ¼ 1.41, 95% CI ¼ 0.92-2.15, p ¼ 0.12). Therefore, we used the entire prereform period as the reference group when calculating the ORs for each of the postreform years. Older patients, patients having longer length of hospital stay, and patients with higher Elixhauser summary score were more likely to have a medication error. Patients were more likely to have a medication error if their admission source was anything other than routine. Weekend admissions had slightly more occurrences of medication errors. The Figure displays a plot of adjusted medication error rates for each year in our study, stratified by hospital teaching status. The difference-in-differences study design allowed us to compare teaching hospitals after institution of duty hours with preduty-hour reform, and also generate 1212
an estimated medication error rate for teaching hospitals after duty-hour reform (diagramed in the Figure as the counterfactual line). Using counterfactual variables, we can measure correlation via surrogate markers.23 Generally, a counterfactual model postulates that all values for both surrogate “S” and the primary outcome “T” for a subject under different interventions are “Z.” Then we postulate that Z ¼ 0 or Z ¼ 1 for 2 values of S (denoted as S0 and S1), though only 1 could be observed for each subject. Within this counterfactual framework, the population is partitioned into subgroups where everyone within each subgroup has the same observed and counterfactual outcomes, and the proportion of population within the subgroups is a parameter of the model.23 After adjusting for the effects of measured patient and hospital characteristics as well as unmeasured secular trends common to all hospitals, teaching hospitals demonstrated a statistically significant increase in medication error rates compared with prereform years in 2005 (OR ¼ 1.40, 95% CI ¼ 1.031.90, p ¼ 0.029) and 2006 (OR ¼ 1.52, 95% CI ¼ 1.161.99, p ¼ 0.002). From 2007 to 2011, there was no statistically significant difference in rates of medication
Journal of Surgical Education Volume 72/Number 6 November/December 2015
TABLE 2. p Values for the Multivariable Logistic Regression Model for Presence of Medication Error in Nonteaching Hospitals Prereform, Medication Error Sample size: N (n) Age: median (IQR) Length of stay: median (IQR) Elixhauser score: median (IQR) Admission source Routine Emergency department Other hospital/ health facility Admission type Emergency Elective Other admission type Weekend admission Disposition Routine discharge Transfer Home health care Against medical advice Died in hospital
4046 (851) 61 (43-75)
Prereform, No Medication Error 19,060,834 (3,973,344) 50 (29-70)
p Value
Postreform, Medication Error
Postreform, No Medication Error
p Value
15,054 (3098)
47,275,216 (9,707,419) 52 (30-70)
o0.0001
o0.0001
60 (47-73)
6 (4-10)
3 (2-5)
o0.0001
6 (4-11)
3 (2-5)
o0.0001
1 (0-7)
0 (0-0)
o0.0001
3 (0-8)
0 (0-2)
o0.0001
46.3 50.0
73.5 23.2
40.5 55.8
72.6 24.5
o0.0001
3.7
3.2
3.7
2.9
51.2 30.9 17.9
22.6 46.3 31
o0.0001
55.3 28.0 16.7
24.5 47.4 28.1
o0.0001
17.9
12.8
o0.0001
19.0
12.4
o0.0001
55.7
78.7
o0.0001
45.0
74
o0.0001
30.0 9.6
10.0 7.0
30.0 18.0
10.0 10.8
0.3
0.1
0.6
0.1
3.9
1.4
3.5
1.1
0.0118
“N” is the estimated total number of admissions in population represented by the NIS and “n” is sample size represented by actual admissions in the NIS. IQR, interquartile range.
errors in teaching hospitals compared to preduty-hour reform (2007: p ¼ 0.07, 2008: p ¼ 0.22, 2009: p ¼ 0.90, 2010: p ¼ 0.23, 201: p ¼ 0.31). Table 3 lists exact medication error rates. Median values are assumed for age, length of stay, and Elixhauser summary score and reference groups for all categorical variables. The observed incidence of reported medication errors was low, ranging from approximately 1 to 3 occurrences for every 10,000 surgical hospitalizations. Medication error rates were lower in teaching hospitals than nonteaching hospitals before implementation of duty-hour regulations. In both teaching and nonteaching hospitals, medication error rates had increased in the postreform years, with teaching hospitals having a greater increase.
DISCUSSION Preventable medication errors, specifically medication overdose, cause significant morbidity and mortality and have been shown to increase hospital length of stay and increase cost of care.6,24,25 The institution of the ACGME
duty-hour reform for residents in July 2003 was designed, in part, to address concerns about preventable errors as well as to help improve patient care and foster resident learning.12 Causes for medication errors vary and include increased age, multiple comorbidities, increased length of stay, and complicated medication regimens.5-11,26-28 Contrary to our hypothesis, we found that medication rates increased initially in teaching hospitals after the hour restrictions were introduced. Although preventable medication errors are caused by many factors that are impossible to measure at the national level, the difference-in-differences study design reduces the effect of these variables when evaluating pre and postduty-hour reform in teaching and nonteaching hospitals. If all other factors remain unchanged in these hospitals over time, the incidence of medication errors for teaching hospitals should look similar to the counterfactual line in the Figure. Interestingly, the actual incidence of medication errors initially spiked in teaching hospitals after institution of the duty-hour reform, but from 2007 to 2011, this difference diminished and was not statistically significant. There are several potential explanations for why medication errors may have initially increased after the
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1213
FIGURE. Adjusted medication error rates by year, stratified by teaching status. The gray dashed line represents the assumed counterfactual of estimated medication error rates in teaching hospitals. Median values are assumed for age, length of stay, and Elixhauser summary score and reference groups for all categorical variables.
institution of duty-hour reform in teaching hospitals. The work-hour limitation may have caused an increase in patient “handoffs” between residents, resulting in increased risk of medication error. Older patients with complex medical comorbidities urgently admitted for nonelective procedures would be particularly vulnerable to a “shift” mentality with frequent handoffs. Furthermore, some studies have suggested that duty hours have had no positive effect on resident fatigue and amount of sleep: this may be another explanation for why medication errors showed an uptrend as residents were faced with less time to do the same amount of work while under the same effects of fatigue.12,14,15 On the contrary, the decrease in error rates from 2007 to 2011 may reflect long-term results of changes that were enacted within residency training programs to accommodate the duty-hour reform and improve residency training. Changes to rotation schedules, such as introduction of a night float rotation, increased oversight of junior residents by senior residents, and improvements to patient handoffs tools have all been incorporated into training programs to comply with duty-hour restrictions and may have played a role in this decrease.29,30
Owing to the nature of the NIS database there are several limitations to this study. Because the NIS database is retrospective and relies on hospital coders inputting physician documentation, there is a risk of sampling bias with underreporting of medication errors. In addition, medication errors with no sequelae are unlikely to be reported or documented and thus might not be included in this database. As previously mentioned, nonphysician causes for medication errors exist and it is impossible to separate these data points from physician medication errors in the NIS, but it is our belief that a large sample such as this allows for some mitigation of these influences on the data. Use of the difference-in-differences study design also reduces the effect of other measurable or nonmeasurable factors that could affect the results. Furthermore, there could have been other outside trends that contributed to this change in medication errors. For example, there has been a large increase in the number of hospitalists since the term was described by Wachter and Goldman.31,32 There was also a concern that fragmented care caused by this increased number of hospitalists could lead to higher numbers of medication errors33; thus, there is
TABLE 3. Medication Error Rates per 100,000 Cases Teaching Nonteaching Counterfactual
'00
'01
'02
'03a
'03b
'04
'05
'06
'07
'08
'09
'10
'11
10.9 12.3
10.0 11.3
9.6 10.8
10.8 12.1
14.6 13.9 12.3
13.8 11.5 10.2
15.6 13.4 11.8
17.6 14.0 12.4
16.8 15.0 13.3
21.6 19.9 17.6
18.2 21.0 18.6
20.8 17.7 15.7
19.5 18.7 16.5
Medication error rates per 100,000 cases for each year. 1214
Journal of Surgical Education Volume 72/Number 6 November/December 2015
a possibility that this could have contributed to our results. Although we are not aware of any other trends occurring during the time period we analyzed, it is entirely possible that they existed. We stress that our results were likely owing to a variety of influencing events rather than a specific trend. For example, it is possible that other team members (e.g., nurses) played a role in medication errors. Moreover, these trends may have influenced each other. Saint and Flanders note that with the ACGME duty-hour restrictions, hospitals may have turned to hospitalists as a solution to resident duty work problems.32 Therefore, different trends may have interacted or influenced each other that caused our results. The data could potentially be biased by increased reporting in teaching hospitals during this time frame because of an increase in emphasis on reporting ADEs. The use of the difference-in-differences methodology has its own drawbacks because if there are diverging trends that occur during this time frame, it is conceivable that we have found 2 separate issues that occur at similar times that may be unrelated but that are effecting medication errors. Although this is a valid argument and there are flaws inherent to the statistical design and database we used, this is the largest study to examine national levels of medication errors and correlate institution of duty hours. Also, the fact that there was a significant increase in July 2003 and 2004 in medication errors specifically in teaching hospitals also lends support to our hypothesis that medication errors do correlate with introduction of duty hours. It is difficult to explain such a focal jump in medication errors in teaching hospitals with another explanation. There was a large difference in medication rates between the year 2000 and 2011 (Figure). We are unsure of an explanation. It is possible that there was actually no difference, and that reporting of medication errors simply improved. Further research is necessary to provide a conclusive explanation. The other results obtained from this study agree with the literature and are intuitive.27 Older patients, especially those with higher Elixhauser summary scores, have higher comorbidities and require more medications, lending themselves to a higher risk of medication error. As mentioned earlier, medication errors are associated with an increased length of stay, although it is not possible to know from the database whether these patients had a medication error that required extended hospital stay or if the extended hospitalization put the patient at higher risk for medication error. Emergency and weekend admissions have also been shown to have a higher risk of incurring medication errors, which we showed as well, and they may be related to increased acuity of the patients when they are admitted.11,27 These features suggest that use of the NIS is a valid and powerful tool to examine the frequency and changes in time of medication errors in surgical patients in the United States.
CONCLUSION We identified a significant increase in medication errors in patients undergoing surgery at teaching hospitals after institution of duty-hour reform, which was mitigated over time. This increase may have potentially been related to the dutyhour restrictions. Although correlation does not imply causation, this study adds to the growing body of literature examining changes in patient care that have evolved during the era of duty-hour reform. The decrease in medication errors that occurred from 2007 to 2011 may be related to several factors, but changes in other aspects of residency training to accommodate for duty-hour restrictions may play a role as well. Development of sustainable and reliable tools for patient handoffs, changes to resident rotation schedules in teaching hospitals, and increased oversight may all have played a factor in the reduction of preventable errors as well as help improve patient care and foster resident learning.
REFERENCES 1. Bhavsar J, Montgomery D, Li J, et al. Impact of duty
hours restrictions on quality of care and clinical outcomes. Am J Med. 2007;120(11):968-974. 2. Pepper DJ, Schweinfurth M, Herrin VE. The effect of
new duty hours on resident academic performance and adult resuscitation outcomes. Am J Med. 2014;127 (4):337-342. 3. Lazarou J, Pomeranz BH, Corey PN. Incidence of
adverse drug reactions in hospitalized patients: a metaanalysis of prospective studies. J Am Med Assoc. 1998;279(15):1200. 4. Kaushal R. Medication errors and adverse drug events
in pediatric inpatients. J Am Med Assoc. 2001;285 (16):2114. 5. De Boer M, Boeker EB, Ramrattan MA, et al. Adverse
drug events in surgical patients: an observational multicentre study. Int J Clin Pharm. 2013;35(5):744-752. 6. Klopotowska JE, Wierenga PC, Stuijt CCM, et al.
Adverse drug events in older hospitalized patients: results and reliability of a comprehensive and structured identification strategy. PLoS ONE. 2013;8(8): e71045. 7. Hoonhout LH, de Bruijne MC, Wagner C, et al.
Nature, occurrence and consequences of medicationrelated adverse events during hospitalization: a retrospective chart review in the Netherlands. Drug Saf. 2010;33(10):853-864. 8. Keers RN, Williams SD, Cooke J, Ashcroft DM. Causes
of medication administration errors in hospitals: a
Journal of Surgical Education Volume 72/Number 6 November/December 2015
1215
systematic review of quantitative and qualitative evidence. Drug Saf. 2013;36(11):1045-1067. 9. Weiss AJ, Elixhauser A. Characteristics of adverse drug
events originating during the hospital stay. Available at: 〈http://www.ncbi.nlm.nih.gov/books/NBK174680〉; 2011 Accessed 19.07.14. 10. Shah R, Blustein L, Kuffner E, Davis L. Communicating
21. Elixhauser A, Steiner C, Harris DR, Coffey RM.
Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. 22. Van Walraven C, Austin PC, Jennings A, Quan H,
Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6): 626-633.
doses of pediatric liquid medicines to parents/caregivers: a comparison of written dosing directions on prescriptions with labels applied by dispensed pharmacy. J Pediatr. 2014;164(3):596-601 e1.
23. Taylor JMG, Wang Y, Thiebaut R. Counterfactual
11. Wahr JA, Shore AD, Harris LH, et al. Comparison of
24. Rottenkolber D, Hasford J, Stausberg J. Costs of
intensive care unit medication errors reported to the United States’ MedMarx and the United Kingdom’s National Reporting and Learning System: a crosssectional study. Am J Med Qual. 2014;29(1):61-69. 12. Nasca TJ, Day SH, Amis ES. The new recommenda-
tions on duty hours from the ACGME task force. N Engl J Med. 2010;363(2):e3. 13. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects
of the Accreditation Council for Graduate Medical Education duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250-258. 14. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality
among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. J Am Med Assoc. 2007;298(9):975-983.
15. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality
link to the proportion of treatment effect explained by a surrogate marker. Biometrics. 2005;61:1102-1111. adverse drug events in german hospitals—a microcosting study. Value Health. 2012;15(6):868-875. 25. Al-Rowibah FA, Younis MZ, Parkash J. The impact of
computerized physician order entry on medication errors and adverse drug events. J Health Care Finance. 2013;40(1):93-102. 26. Kane-Gill SL, Wytiaz NP, Thompson LM, et al. A
real-world, multicenter assessment of drugs requiring weight-based calculations in overweight, adult critically ill patients. Sci World J. 2013;2013:1-9. 27. Budnitz DS, Lovegrove MC, Shehab N, Richards CL.
Emergency hospitalizations for adverse drug events in older Americans. N Engl J Med. 2011;365(21): 2002-2012. 28. Eginger KH, Yarborough LL, Inge LD, et al. Medi-
among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. J Am Med Assoc. 2007;298(9):984-992.
cation errors in HIV-infected hospitalized patients: a pharmacist’s impact. Ann Pharmacother. 2013;47(78):953-960.
16. Norby K, Siddiq F, Adil MM, Haines SJ. The effect of
29. George BP, Probasco JC, Dorsey ER, Venkatesan A.
duty hour regulations on outcomes of neurological surgery in training hospitals in the United States: duty hour regulations and patient outcomes. J Neurosurg. 2014;121:1-15.
30. Reed DA, Fletcher KE, Arora VM. Systematic review:
17. Dumont TM, Tranmer BI, Horgan MA, Rughani AI.
Trends in neurosurgical complication rates at teaching vs nonteaching hospitals following duty-hour restrictions. Neurosurg. 2012;71(5):1041-1046. 18. Lumley T. Analysis of complex survey samples. J Stat
Softw. 2004;9(1):1-19. 19. Lumley T. Survey: Analysis of Complex Survey
Impact of 2011 resident duty hour requirements on neurology residency programs and departments. Neurohospitalist. 2014;4(3):119-126. association of shift length, protected sleep time, and night float with patient care, residents’ health, and education. Ann Intern Med. 2010;153(12):829-842. 31. Wachter RM, Goldman L. The emerging role of
hospitalists in the American health care system. N Engl J Med. 1996;335:514-517. 32. Saint S, Flanders SA. Hospitalists in teaching hospitals:
Samples. Available at: 〈http://cran.r-project.org/web/ packages/survey/index.html〉.
opportunities but not without danger. J Gen Intern Med. 2004;19(4):392-393.
20. R Core Team. A Language and environment for stat-
33. Sox HC. The hospitalist model: perspectives of the
istical computing. Available at: 〈http://www.R-project. org/〉.
1216
patient, the internist, and internal medicine. Ann Intern Med. 1999;130:368-372.
Journal of Surgical Education Volume 72/Number 6 November/December 2015
