58
Journal for Healthcare Quality
Improving Medication Administration Safety: Using Na¨ıve Observation to Assess Practice and Guide Improvements in Process and Outcomes Nancy Donaldson, Carolyn Aydin, Moshe Fridman, and Mary Foley Abstract: Purpose: To present findings from the Collaborative Alliance for Nursing Outcomes’ (CALNOC) hospital medication administration (MA) accuracy assessment in a sample of acute care hospitals. Aims were as follows: (1) to describe the CALNOC MA accuracy assessment, (2) to examine nurse adherence to six safe practices during MA, (3) to examine the prevalence of MA errors in adult acute care, and (4) to explore associations between safe practices and MA accuracy. Methods: Using a cross-sectional design, point in time, and convenience sample, direct observation data were collected by 43 hospitals participating in CALNOC’s benchmarking registry. Data included 33,425 doses from 333 observation studies on 157 adult acute care units. Results reveal that the most common MA safe practice deviations were distraction/interruption (22.89%), not explaining medication to patients (13.90%), and not checking two forms of ID (12.47%). The most common MA errors were drug not available (0.76%) and wrong dose (0.45%). The overall percentage of safe practice deviations per encounter was 11.40%, whereas the overall percentage of MA errors was 0.32%. Conclusions and Implications: Findings predict that for 10,000 MA encounters, 27,630 safe practice deviations and 770 MA errors will occur. A 36% reduction in practice deviation per encounter prevents 4.4% MA errors. Ultimately, reliably performing safe practices improves MA accuracy.
Keywords acute care medication safety medication administration accuracy nurse medication safe practices
C
Journal for Healthcare Quality Vol. 36, No. 6, pp. 58–68 2014 National Association for Healthcare Quality
Purpose With the publication of To Err is Human (Kohn, Corrigan, & Donaldson, 2000), the nation confronted the impacts of errors in healthcare. In 2010, the U.S. Office of the Inspector General reported 13.5% of hospitalized Medicare patients experienced a healthcare error resulting in serious harm, which is termed as an adverse event (AE; Adverse events in hospitals: national incidence among medicare beneficiaries, 2010). Reviewers of AE data determined that 44% were preventable. The most common (42%) preventable AEs were related to medications. Medication administration (MA) is a complex activity. Time and work studies and field observations have found that MA consumes 27–40% of nurses’ time (Armitage & Knapman,
2003; Elganzouri, Standish, & Androwich, 2009; Keohane et al., 2008). Rates of MA errors vary widely, from 0.01% to 20%. Bates, Boyle, Vander Vliet, Schneider, and Leape (1995) found that, among preventable adverse drug events (ADEs), 26% errors occurred during administration, and none were intercepted by others. Although a source of ADEs resulting in patient harm, nurses also serve as a protective role. Of the ADEs intercepted before reaching the patient, most are intercepted by the nurse (Cullen, Bates, & Leape, 2000; Leape et al., 1995; Rogers, Dean, Hwang, & Scott, 2008). Most of the MA errors go unreported because the clinician is unaware that an error has occurred (Barker, Flynn, Pepper, Bates, & Mikeal, 2002). As the last line of defense against errors, nurses must reliably perform fundamental safe practices ensuring MA accuracy.
Aims of This Study This study presents findings from the Collaborative Alliance for Nursing Outcomes’ (CALNOC) ongoing study of acute care MA accuracy. Aims of the study were as follows:
r To describe the CALNOC na¨ıve observation MA accuracy assessment method.
r To examine nurse adherence to six fundamental safe practices during MA.
r To examine the prevalence of MA errors in adult acute care.
r To explore associations between nurse deviation from fundamental safe practices and MA errors in adult acute care.
Review of the Literature The turbulent hospital environment in which nurses administer medications was explicated by Jennings, Sandelowski, and Mark (2011) in an ethnographic study of MA on two adult acute care units in a large community hospital. Following field observations and interviews, the
Vol. 36 No. 6 November/December 2014
authors concluded “the nurse’s day was defined by medication administration” (p. 1442). In a systematic review of 23 studies from 1980 to 2008, Biron, Loiselle, and Lavoie-Tremblay (2009) examined the effect of nurses’ work interruptions on MA accuracy. The authors found on average 67 work interruptions per hour, which were based on 14 studies. One nonexperimental study showed a significant association between work interruptions and MA errors (Scott-Cawiezell et al., 2007). The Medication Administration Accuracy Assessment was developed in 2004 by the CALNOC, a self-sustaining, not-for-profit, nursing sensitive benchmarking registry serving nearly 300 hospitals in six states with unit, service line, and facility level benchmarks for structure, process, and outcome measures. CALNOC metrics, methods, and benchmarks have been reported elsewhere (Aydin et al., 2004; Brown, Aydin, Donaldson, Fridman, & Sandhu, 2010; Brown, Donaldson, Burnes Bolton, & Aydin, 2010). CALNOC’s MA accuracy assessment was based on the findings from published direct observation studies (Barker et al., 2002; Pepper, 1995). Barker and colleagues (2002) found that the vast majority of MA errors are unreported and direct observation, plus medical record review, was the most reliable method to determine MA accuracy. The validity and precision of direct observation was reaffirmed by Meyer-Massetti and colleagues (2011) in a systematic review of medication safety assessment methods, concluding no single method was superior to the other and the combination of multiple assessment methods revealed different types of MA errors. The authors noted the value of direct observation as a compliment to ADE incident reporting systems. The CALNOC MA accuracy assessment method has been successfully used for research and quality improvement (Ching, Long, Williams, & Blackmore, 2013; Cochran, 2009; Helmons, Wargel, & Daniels, 2009; Kliger, Blegen, Gootee, & O’Neil, 2009).
Study Design and Methods This cross-sectional study used MA direct observations and a prevalence or point in time sampling approach with a minimum sample of 100 observed doses per patient care unit. A fundamental assumption of the prevalence sampling approach is that it is representative
of the actual error incidence (Barker, Flynn, & Pepper, 2002). To ensure the reliability and validity of the data, observers from participating hospitals were trained by CALNOC in a standardized workshop.
Institutional Review Board (IRB) CALNOC has sustained IRB approval at Cedars-Sinai Medical Center and University of California San Francisco since 1998. It should be noted that individual nurses were not identified during observations. Observers, blinded to patient histories and medication orders, were unable to proactively identify errors; however, they were instructed to interrupt administration of a medication if they observed an obvious error that would potentially harm the patient. Errors discovered during chart review that affected patient safety were reported to the patient’s nurse.
Sample Accrual and Characteristics MA accuracy assessment data collected by CALNOC hospitals for this study included patients in adult acute care units from July 2006 through October 2009. Each hospital determined which unit(s), staff, and shifts were observed as well as the timing of data collection. To minimize staff discomfort with the observation process, observers were trained to limit intrusion. Despite knowing the focus of the observations, staff made process errors and in many instances breached relevant hospital policies and procedures, suggesting that staff habituated to the observer’s presence as they performed routine tasks (Pepper & Blegen, 2004; Schnelle, Ouslander, & Simmons, 2006). The CALNOC MA assessment used the following definitions:
r MA accuracy: Administration of a dose of medication exactly as ordered by the physician, which includes the fundamental five rights—right drug, right patient, right dose, right route, and right time (Carlton & Blegen, 2006; Eisenhauer, Hurley, & Dolan, 2007; Elliott & Liu, 2010). r MA error: A dose administered differently than ordered by the physician. r Opportunity for error (OE): The basic unit of data in observational accuracy studies; OE included any dose ordered plus any
59
60
Journal for Healthcare Quality
unordered doses given and any ordered doses omitted. In conducting the MA assessment, nurse observers, purposefully “na¨ıve” to, the patient history and treatment, collected dose-level data in three steps: (1) systematic observation of six selected fundamental practices performed by the nurse during preparation, administration, and documentation of medications (a patient administered multiple doses was observed for all doses), (2) review of the medical record to extract relevant medication orders, and (3) comparison of medications administered and medications ordered to determine MA accuracy. Excluded medications included infusions and selfadministered medications. In performing tasks during the preparation, administration, and documentation of medications, nurses have typically been inculcated with the “five rights”—right patient, drug, dose, route, and time. Despite strong face validity, the “rights” were not evidence-based (Carlton & Blegen, 2006; Eisenhauer et al., 2007; Elliott & Liu, 2010). CALNOC operationalized the following safe practices from the “rights” literature and observers noted whether the nurse: 1. compared medication with MA record (MAR), 2. minimized distraction or interruption during medication preparation or administration, 3. ensured medication is labeled throughout the process from preparation to administration, 4. checked two forms of patient identification prior to administration of medication, 5. explained medication to patient or family as appropriate, 6. charted/documented MA immediately after completion. To determine MA accuracy, the observer compared medication orders in the patient’s record with observed medications administered, revealing 10 possible outcomes that were then coded according to the following definitions: 1. No error observed. 2. Unauthorized drug error: Administration of a dose never ordered for that patient.
3. Wrong dose error: Any dose of a drug (excluding an injectable drug) that contained the wrong number of dosage units (such as tablets) or is, in the judgment of the observer, more than 17% greater or less than the correct dosage. 4. Wrong form error: The administration of a drug dose in a different form than ordered by the prescriber when prescriber wrote for a specific dosage form. 5. Wrong route error: Medication administered to a patient using a different route than ordered. 6. Wrong technique error: Using an inappropriate procedure or improper technique in administration of a drug. Focus is on technique violations that can alter drug effect. 7. Extra dose error: Any dose given in excess of the total number of times authorized by physician order. 8. Omission error: (a) Failure to give an ordered dose that appears on the MAR by the time the next dose is due. Patient refusals or drugs appropriately withheld are not considered omissions. (b) Order found in the medical record that does not appear on the MAR. 9. Wrong time error: Administration of a dose more than 60 min before or after scheduled administration time. If food is involved in the order, the dose should be given within 30 min of scheduled time. 10. Drug not available error: Administration of a dose more than 60 min after scheduled administration time due to nonavailability of the medication.
Statistical Methods Statistical analyses were designed to address three fundamental questions: 1. What percentage of the time do direct care nurses perform the selected six safe practices during routine administration of medications? 2. What percentage of medication doses are administered differently than ordered? 3. To what extent does the performance of selected safe practices during MA predict MA accuracy?
Vol. 36 No. 6 November/December 2014
Statistical Procedures CALNOC MA accuracy data consisted of one record for each medication dose observed. Each unit’s observation “study” included a minimum sample of 100 doses. Each single dose record captured the presence or absence of both safe practice behaviors performed by the nurse and outcome, either no error or the type of error(s). Data were first analyzed and values were calculated at the dose level. Doses were then aggregated to the encounter level, with each encounter including all doses administered during a single observation of a single patient. For each patient who received more than one dose during a single observation, safe practices and MA errors for the encounter were summarized to avoid correlations among same encounter doses. As a patient was not likely to be sampled twice within a single unit level 100 dose observational period, encounters and patients typically constituted a one-to-one relationship. To aid in interpreting results, safe practices, with the exception of “distracted/interrupted,” were recoded so that a value of 1 indicated a safe practice was not observed (i.e., a deviation from safe practices) and 0 indicated the safe practice was performed. MA errors were coded with a value of 1. The omission error rate was reported and then excluded from further analyses because the majority of omitted doses were ordered but not given, thus not observable. Except for descriptive purposes, “Wrong Time” and “Wrong Technique” errors were excluded from analyses, as these errors were confounded by wide variability in observer reporting/interpretation and hospital policies. The remaining six types of errors were retained for further analysis. The prevalence of both safe practice deviations and MA errors was calculated individually and in aggregate for each safe practice and MA error type. Noting the possibility of multiple safe practice deviations and MA errors per dose and multiple doses per encounter, the following summary variables were calculated for each encounter: 1. Overall percentage of errors per encounter calculated as two measures: (a) Overall percentage of safe practice deviations per encounter: Total number of times any safe practice deviation was coded for
all doses, divided by total doses in the encounter × six possible practice deviations. (b) Overall percentage of MA errors per encounter: Total number of times any error was coded across all doses divided by total number of doses in the encounter × six possible errors. 2. Percentage of each individual safe practice deviation/MA type of error per encounter: Total number of times a specific practice deviation/MA type of error was coded across all doses in the encounter divided by total number of doses in the encounter for each safe practice deviation and MA error. The total number of practice deviations/MA errors for each type of deviation/error, and overall, were defined as the numerator counts in definitions 1 and 2 above to be used as the dependent variables in Poisson regression models. To answer questions (1) and (2), analyses included descriptive statistics for both safe practice deviations and MA errors, followed by an examination of the effect of selected safe practice deviations on errors (question 3) at the encounter level. However, analysis of the data at the encounter level required accommodation of the multilevel structure in the data. For question 3, generalized mixed (multilevel) linear models of the overall error count were fitted on the number of each of the safe practices separately (one per model). Random error terms for study, unit, and facility were used to account for the data hierarchy of encounters within study, within units, within facilities. MA error counts were assumed to follow a Poisson distribution. The number of doses administered during each encounter was used as an offset variable in all simple Poisson regression models and all models were adjusted for linear yearly trend during the 4 years of study. Incidence rate ratios for each individual safe practice were calculated using the estimated coefficients from the fitted Poisson regression models. The combined effect of the selected six safe practices on the overall error counts was measured by fitting a Poisson model using overall number of practice deviations per encounter as a covariate. Estimated coefficients and t-test p-values along with variance components for each level and the incidence rate ratios are reported for each model.
61
62
Journal for Healthcare Quality
Procedures Hospitals’ na¨ıve observers watched staff nurses prepare, administer, and document medications dose by dose. Observations were coded noting safe practices and the incidence of distractions and interruptions, plus the specifics of the medication dose, route, timing, and technique. Following completion of the MA pass per patient, or per cluster of patients, the observer reviewed the medical record and compared what was observed with medication orders, coding each dose either “No error” or the specific error(s). Data were submitted to CALNOC using customized Excel files with automated data checking and validation.
Results The sample included 333 MA accuracy assessment studies (minimum 100 doses each) drawn from 43 CALNOC hospitals and 157 units. The majority of doses were observed on medical– surgical units (74%) with the remaining doses observed equally on step-down (13%) and critical care units (13%) with 95% of doses administered by registered nurses. Descriptive statistics for key variables are displayed in Table 1. Most units completed one or two observation studies, with one unit completing seven studies; 27.30% of patients were administered one medication dose, 17.20% two doses, 11.60% three doses, 10.60% four doses, and 33.30% of patients had five doses or more up to a maximum of 31 doses. Omission errors comprised 1% of all doses (318/33,643) and are excluded from further analyses. The remaining data included 33,325 doses administered to 8,594 patients.
Frequency of MA Safe Practice Deviations and MA Errors Due to coding variability in wrong time (3.30% of dose administrations and 2.96% of encounters) and wrong technique errors (2.33% of single dose administrations and 2.88% of all MA encounters), these errors were excluded from further analyses. A summary of safe medication practice deviations and MA errors categorized by dose and encounter is presented in Table 2. The most common safe practice deviations observed per encounter were as follows: nurse distraction/interruption (22.89%), medication not explained to patient (13.90%), two IDs not checked (12.47%), and MA not documented
immediately (10.68%). The most common MA errors were drug not available (0.76%) and wrong dose (0.45%). The overall percentage of safe practice deviations per encounter was 11.40%, whereas the overall percentage of MA errors was 0.32%.
Impact of Safe Practices on MA Accuracy Findings reveal a significant adjusted effect of each individual practice deviation on overall MA error counts (Table 3). The individual practice deviations of “Not compared to MAR,” “Not labeled throughout process,” “Not documented immediately,” “Two forms of ID not checked,” and “Medication not explained to patient” had large effect sizes. Examining the association of overall number of practice deviations with overall number of MA errors per encounter revealed that a reduction by one safe practice deviation in the overall number of practice deviations per encounter was associated with an estimated 4.37% decrease in the number of MA errors (p = .0001). Examination of sources of MA error variability partitioned by the hospital, unit, and study levels reveals that the largest proportion of error variability was associated with between studies variation, followed closely by between facility variation. A much smaller fraction of outcome error variability was associated with variation between units (Table 3). Findings from this study predict that for 10,000 MA encounters, there will be 27,630 safe practice deviations and 770 MA errors. We estimate that a reduction by one safe practice deviation per encounter will prevent 34 (4.40%) MA errors. These findings support the impact of safe practices on MA safety and have the potential to guide crucial improvements in patient care quality, safety, and costs.
Limitations The convenience sample included all studies conducted by CALNOC registry hospitals during the specified time period. It is possible that unmeasured systematic differences differentiate this sample from the population of hospitals at large. Variation in observer coding likely introduced coding error despite efforts to foster interrator reliability. We found that two errors (wrong technique and wrong time) were confounded by variability in hospital policies and
Vol. 36 No. 6 November/December 2014
Table 1. Summary of Hospital, Unit, and Encounter (Patient) Characteristics Sample Characteristics Number of doses Number of encounters (patients) Number of studies Number of units Number of hospitals
July 2006 Through October 2009 33,325 8,594 333 157 43
Distribution of units by unit type
Number
Percentage
107 26 24 157
68.2 16.5 15.3 100
Medical/surgical Step down Critical care Total units Distribution of encounters by unit type Medical/surgical Critical care Step down Total encounters Patient age (mean)
Encounter (patient)characteristics Number Number
6,378 1,130 1,086 8,594 59.3 Primary reason for hospitalization
Percentage 74.2 13.2 12.6 100
Medical diagnosis (%) Surgical diagnosis (%)
65.9 34.1 Patient gender
Male (%) Female (%)
51.7 48.3
Table 2. Percentage of Safe Practice Deviations and Medication Administration (MA) Errors
Measure
Mean Percentage Per encounter (N = 157 units)
Per Dose Percentage (N = 33,325)
2.83 22.89 5.64 12.47 13.90 10.68 11.40
2.49 25.02 5.48 12.31 15.48 10.66 11.88
0.76 0.45 0.28 0.24 0.12 0.08 0.32
0.86 0.44 0.32 0.20 0.09 0.08 0.33
Safe practice deviation typea Not compared with MAR Nurse distracted/interrupted Not labeled throughout process Two forms of ID not checked Not explained to patient Not documented immediately after administration Overall percentage of safe practice deviationsa Medication administration outcome error typeb Drug not available Wrong dose Wrong route Unauthorized drug Extra dose Wrong form Overall percentage of errorsb
a Total number of times the specific error was coded for all doses in the encounter divided by the total number of doses in the encounter. Result was multiplied by 100 to obtain a percentage. b Total number of times any deviation was coded for all doses in encounter divided by the total number of doses in the encounter × six possible errors. Result was multiplied by 100 to obtain a percentage.
63
b
a
(0.014) 0.106 (0.013) 0.095 (0.012) 0.030 (0.013) 0.045 (0.004)
0.214 (0.021) 0.120 (0.019) 0.129
Slope (SE)
4.37
0.50
1.57
1.75
2.13
3.27
3.50
Incidence Rate Ratio (%)
0.0001
0.0189
0.0001
0.0001
0.0001
0.0001
0.0001
p - Value
0.41
0.44
0.51
0.52
0.48
0.45
0.49
Between facilities
0.07
0.16
0.10
0.15
0.08
0.11
0.11
Between units
Variance Components
0.68
0.59
0.66
0.58
0.66
0.66
0.65
Between studies
The dependent variable is overall outcome error count. Each table row reports on a separate model. Incidence rate ratio calculated for a decrease by one-sixth for each of the six individual practice deviation types and for a decrease by 1 for the overall practice deviation total to make the results comparable.
Overall number of safe practice deviations
Distracted/interrupted
Not explained to patient
Two forms of ID not checked
Not charted/documented immediately after administration
Not labeled throughout process
Not compared with MAR
Model Predictor
b
Table 3. Poisson Regression Models for Each Safe Practice Deviation on Overall Medication Administration (MA) Error Counts Using Encounter Level Data and Adjusted for Linear Yearly Trend (N = 8,594)a
64 Journal for Healthcare Quality
Vol. 36 No. 6 November/December 2014
procedures, limiting confidence in these error types. It is noteworthy that the extent to which hospitals used the same staff observers across multiple observation assessments is unknown, introducing the potential for coding “drift” due to observer staffing variability.
Directions for Future Research Analysis to explore the extent to which multiple observations per hospital may have affected these findings did not reveal significant differences in rates over time when taking into account same hospital repeated assessments and the time elapsed between assessments. Future research will address this question in more detail.
Discussion This study provides the most robust sample of MA doses directly observed yet reported. The overall MA error rate was substantially lower than previous reports using similar methods with overall error rates ranging from 4% to 19% (Barker et al., 2002; Helmons et al., 2009; Kliger et al., 2009; Westbrook, Woods, & Rob, Dunsmuir, & Day, 2010). Safe practice deviations were generally comparable to previous pre– post design reports of baseline performance, although design variation confounds comparisons (Helmons et al., 2009; Kliger et al., 2009; Westbrook et al., 2010). These results support previous findings associating interruptions and distractions with MA errors (Westbrook et al., 2010), although the volume of interruptions observed in the CALNOC dataset were half reported by Westbrook and colleagues (2010). The finding of greater variation between unit “studies,” rather than units alone or hospitals, suggests the importance of microsystem factors unique to shifts on units, for example, culture of safety or availability of support services. Further inquiry exploring the impact of microsystem characteristics is vital to greater understanding of this finding. Variation in staffing hours or skill mix may be viewed as less influential, given the impact of mandated nurse–patient ratios in a majority of CALNOC hospitals and the finding that 95% of medication doses were administered by RNs. Unlike other research using direct observation to measure MA accuracy, these data were derived from a performance improvement
registry dataset that reflected the quality imperatives of participating hospitals rather than the design precision of research studies (Barker et al., 2002; Helmons et al., 2009; Kliger et al., 2009; Westbrook et al., 2010). Although CALNOC’s data were obtained by nurse observers, other investigators have noted the benefits of using pharmacists (Helmons et al., 2009), suggesting the potential for collaboration between nurses and pharmacists in the quest to better understand MA safety.
Implications for Practice Findings confirm the usefulness of the CALNOC MA accuracy assessment for examining medication safety within and between hospitals and the link between safe practices and MA accuracy. Despite the resources and training required to conduct unit-level MA assessment, 43 hospitals voluntarily submitted data to CALNOC, suggesting they considered the direct observation process important to their medication safety improvement efforts. Hospital feedback indicated that the CALNOC assessment added value by providing a point-in-time systematic assessment of the reliability of incident-reporting systems and the opportunity to target improvement at the unit-level based on assessment findings. Ultimately, these findings affirm that nurse adherence to safe practices reduces MA errors. Although MA errors account for a minority of overall ADEs, they are typically not intercepted by other members of the healthcare team. Integrating these findings into evidence-based staffing considerations, nursing education and clinical practice competency validation has the potential to improve the quality, cost, and outcomes of MA.
Acknowledgments This study was funded, in part, by grants from the Gordon and Betty Moore Foundation, Betty Irene Moore Nursing Initiative, and the Robert Wood Johnson Foundation, Interdisciplinary Nursing Quality Research Initiative.
References Adverse events in hospitals: National incidence among Medicare beneficiaries. (2010). Report No. OEI-06-0900090. Washington, DC: U.S. Department of Health and Human Services. Armitage, G., & Knapman. H. (2003). Adverse events in drug administration: A literature review. Journal of Nursing Management, 11, 130–140.
65
66
Journal for Healthcare Quality
Aydin, C. E., Bolton, L. B., Donaldson, N., Brown, D. S., Buffum, M., et al. (2004). Creating and analyzing a statewide nursing quality measurement database. Journal of Nursing Scholarship, 36, 371–378. Barker, K. N., Flynn, E. A., & Pepper, G. A. (2002). Observation method in detecting medication errors. American Journal of Health-System Pharmacy, 59, 2314–2316. Barker, K. N., Flynn, E. A., Pepper, G. A., Bates, D. W., & Mikeal, R. L. (2002). Medication errors observed in 36 health care facilities. Archives of Internal Medicine, 162, 1897–1903. Bates, D. W., Boyle, D. L., Vander Vliet, M. B., Schneider, J., & Leape, L. (1995). Relationship between medication errors and adverse drug events. Journal of General Internal Medicine, 10, 199–205. Biron, A. D., Loiselle, C. G., & Lavoie-Tremblay, M. (2009). Work interruptions and their contribution to medication administration errors: An evidence review. Worldviews on Evidence-based Nursing, 6, 70–86. Brown, D. S., Aydin, C. E., Donaldson, N., Fridman, M., & Sandhu, M. (2010). Benchmarking for small hospitals: Size didn’t matter! Journal for Healthcare Quality, 32, 50– 60. Brown, D. S., Donaldson, N., Burnes Bolton, L., & Aydin, C. E. (2010). Nursing-sensitive benchmarks for hospitals to gauge high-reliability performance. Journal for Healthcare Quality, 32, 9–17. Carlton, G., & Blegen, M. A. (2006). Medication-related errors: A literature review of incidence and antecedents. Annual Review of Nursing Research, 24, 19–38. Ching, J. M., Long, C., Williams, B. L., & Blackmore, C. C. (2013). Using lean to improve medication administration safety: In search of the “perfect dose”. Joint Commission Journal on Quality and Patient Safety, 39, 195–204. Cochran, G. (2009). Comparing the effectiveness of medication use systems in small rural hospitals. Rockville, MD: Agency for Healthcare Research and Quality. Cullen, D. J., Bates, D. W., & Leape, L. L. (2000). Prevention of adverse drug events: A decade of progress in patient safety. Journal of Clinical Anesthesiology, 12, 600–614. Eisenhauer, L. A., Hurley, A. C., & Dolan, N. (2007). Nurses’ reported thinking during medication administration. Journal of Nursing Scholarship, 39, 82–87. Elganzouri, E. S., Standish, C. A., & Androwich, I. (2009). Medication Administration Time Study (MATS): Nursing staff performance of medication administration. Journal of Nursing Administration, 39, 204–210. Elliott, M., & Liu, Y. (2010). The nine rights of medication administration: An overview. British Journal of Nursing, 19, 300–305. Helmons, P. J., Wargel, L. N., & Daniels, C. E. (2009). Effect of bar-code-assisted medication administration on medication administration errors and accuracy in multiple patient care areas. American Journal of Health-System Pharmacy, 66, 1202–1210. Jennings, B. M., Sandelowski, M., & Mark, B. (2011). The nurse’s medication day. Qualitative Health Research, 21, 1441–1451. Keohane, C. A., Bane, A. D., Featherstone, E., Hayes, J., Woolf, S., Hurley, A., et al. (2008). Quantifying nursing workflow in medication administration. Journal of Nursing Administration, 38, 19–26. Kliger, J., Blegen, M. A., Gootee, D., & O’Neil, E. (2009). Empowering frontline nurses: A structured intervention enables nurses to improve medication administration accuracy. Joint Commission Journal on Quality and Patient Safety, 35, 604–612.
Kohn, L. T., Corrigan, J. M., Donaldson, M. S., & Institute of Medicine. (2000). To err is human. Building a safer health system. Washington, DC: National Academy Press. Leape, L. L., Bates, D. W., Cullen, D. J., Cooper, J., Demonaco, H. J., Gallivan, T., et al. (1995). Systems analysis of adverse drug events. ADE Prevention Study Group. Journal of the American Medical Association, 274, 35–43. Meyer-Massetti, C., Cheng, C. M., Schwappach, D. L., Paulsen, L., Ide, B., Meier, C., et al. (2011). Systematic review of medication safety assessment methods. American Journal of Health-System Pharmacy, 68, 227–240. Pepper, G. A. (1995). Errors in drug administration by nurses. American Journal of Health-System Pharmacy, 52, 390–395. Pepper, G. A., & Blegen, M. A. (2004). How intrusive is direct observation of nursing care? Communicating Nursing Research, 37, 258. Rogers, A. E., Dean, G. E., Hwang, W. T., & Scott, L. D. (2008). Role of registered nurses in error prevention, discovery and correction. Quality & Safety in Health Care, 17, 117–121. Schnelle, J. F., Ouslander, J. G., & Simmons, S. F. (2006). Direct observations of nursing home quality: Does care change when observed? Journal American Medical Director Association, 7, 541–544. Retrieved August 11, 2013 from www.ncbi.nih.gov/pubmed/17095417. Scott-Cawiezell, J., Pepper, G. A., Madsen, R. W., Petroski, G., Vogelsmeier, A., & Zellmer, D. (2007). Nursing home error and level of staff credentials. Clinical Nursing Research, 16, 72–78. Westbrook, J. I., Woods, A., Rob, M. I., Dunsmuir, W. T. M, & Day, R. (2010). Associations of interruptions with an increased risk and severity of medication administration errors. Archives Internal Medicine, 170, 683–690.
Authors’ Biographies Nancy Donaldson, PhD, RN, FAAN, UCSF School of Nursing, San Francisco, CA. Donaldson is a clinical professor. Since 1996, Donaldson has served as the CoPrincipal Investigator for the Collaborative Alliance for Nursing Outcomes Project (CALNOC) and now serves as CALNOC Senior Scientist Emeritus. Carolyn Aydin, PhD, is a senior research scientist IV at Cedars-Sinai Medical Center and Burns & Allen Research Institute, Los Angeles, CA. Aydin is also a CALNOC data manager and coinvestigator. Moshe Fridman, PhD, is the president and owner of AMF Consulting, Inc., Los Angeles, CA, since 1998 till present. Fridman provides clients with data management and statistical analysis consulting services and head analysis teams and provides technical support for long-term projects (large clinical trials, grants). Mary Foley, PhD, RN, is the director of the Center for Nursing Research and Innovation at the University of California, San Francisco (UCSF) School of Nursing. Formerly she was the president of the American Nurses Association. For more information on this article, contact Nancy Donaldson at [email protected].
Vol. 36 No. 6 November/December 2014
Journal for Healthcare Quality is pleased to offer the opportunity to earn continuing education (CE) credit to those who read this article and take the online posttest at http://www. nahq.org/education/content/jhq-ce.html. This continuing education offering, JHQ 251, will provide 1 contact hour to those who complete it appropriately.
Core CPHQ Examination Content Area IV. Patient Safety
Objectives r Describe the 3 basic steps in the na¨ıve observation medication administration accuracy assessment method. r Identify 6 fundamental safe practices during medication administration that are key to ensuring medication delivery accuracy. r Confirm the most prevalent type of medication administration error found in the CALNOC study.
Multiple Choice Questions 1. Reviewers of adverse event data for the 2010 report by the US Office of the Inspector General concluded the most common preventable adverse events were related to: a. Culture of safety b. Medications c. Administrative oversight d. Poor communication 2. Most medication administration errors go unreported because: a. The clinician is unaware an error has occurred. b. They did not harm the patient. c. Adverse event reporting systems are not designed to capture these data. d. The clinician is embarrassed and avoids reporting. 3. Use of direct observation for medication administration accuracy assessment is: a. Not as reliable as adverse event reporting systems. b. More reliable than adverse event reporting systems, when combined with medical record review. c. Strictly a research method. d. None of the above
4. The CALNOC medication administration accuracy assessment uses na¨ıve observers to collect data to ensure: a. Observers are less experienced than the staff they are observing. b. Observers are friendly but not intrusive during observations. c. Observers are unaware of the patient’s actual medication orders. d. All the above 5. The CALNOC medication administration accuracy assessment includes: a. Systematic observation of six selected fundamental practices performed by the nurse during preparation, administration and documentation of medications. b. Review of the medical record to extract relevant medication orders. c. Comparison of medications administered and medications ordered to determine MA accuracy. d. All the above 6. Which of the following is not one of the 6 safe practices observed in the CALNOC medication administration accuracy assessment: a. The nurse compared the medication with MA record. b. The nurse ensured the medication was labeled throughout the process from preparation to administration. c. The nurse asked the patient to verify the purpose of the medication. d. The nurse charted/documented medication administration immediately after completion. 7. The most common safe practice deviation by the nurse observed in this study per encounter was: a. Nurse distraction/interruption b. Medication not explained to patient c. Two IDs not checked d. Medication administration not documented immediately 8. The most common medication administration error found in the CALNOC study was: a. Drug not available b. Wrong dose c. Wrong technique d. Wrong time
67
68
Journal for Healthcare Quality
9. Study findings revealed that: a. Safe practices are not significantly associated with medication administration accuracy. b. Reducing safe practice deviations per encounter would decrease medication administration errors. c. Not all safe practices are associated with medication accuracy. d. None of the above 10. Findings from the CALNOC study suggest:
a.
Future study is needed to explore the impact of microsystem characteristics on medication accuracy. b. CALNOC medication administration accuracy assessment is useful for examining medication safety within and between hospitals. c. The CALNOC assessment may be helpful in examining the reliability of adverse event reporting systems. d. All the above
Journal for Healthcare Quality
Improving Medication Administration Safety: Using Na¨ıve Observation to Assess Practice and Guide Improvements in Process and Outcomes Nancy Donaldson, Carolyn Aydin, Moshe Fridman, and Mary Foley Abstract: Purpose: To present findings from the Collaborative Alliance for Nursing Outcomes’ (CALNOC) hospital medication administration (MA) accuracy assessment in a sample of acute care hospitals. Aims were as follows: (1) to describe the CALNOC MA accuracy assessment, (2) to examine nurse adherence to six safe practices during MA, (3) to examine the prevalence of MA errors in adult acute care, and (4) to explore associations between safe practices and MA accuracy. Methods: Using a cross-sectional design, point in time, and convenience sample, direct observation data were collected by 43 hospitals participating in CALNOC’s benchmarking registry. Data included 33,425 doses from 333 observation studies on 157 adult acute care units. Results reveal that the most common MA safe practice deviations were distraction/interruption (22.89%), not explaining medication to patients (13.90%), and not checking two forms of ID (12.47%). The most common MA errors were drug not available (0.76%) and wrong dose (0.45%). The overall percentage of safe practice deviations per encounter was 11.40%, whereas the overall percentage of MA errors was 0.32%. Conclusions and Implications: Findings predict that for 10,000 MA encounters, 27,630 safe practice deviations and 770 MA errors will occur. A 36% reduction in practice deviation per encounter prevents 4.4% MA errors. Ultimately, reliably performing safe practices improves MA accuracy.
Keywords acute care medication safety medication administration accuracy nurse medication safe practices
C
Journal for Healthcare Quality Vol. 36, No. 6, pp. 58–68 2014 National Association for Healthcare Quality
Purpose With the publication of To Err is Human (Kohn, Corrigan, & Donaldson, 2000), the nation confronted the impacts of errors in healthcare. In 2010, the U.S. Office of the Inspector General reported 13.5% of hospitalized Medicare patients experienced a healthcare error resulting in serious harm, which is termed as an adverse event (AE; Adverse events in hospitals: national incidence among medicare beneficiaries, 2010). Reviewers of AE data determined that 44% were preventable. The most common (42%) preventable AEs were related to medications. Medication administration (MA) is a complex activity. Time and work studies and field observations have found that MA consumes 27–40% of nurses’ time (Armitage & Knapman,
2003; Elganzouri, Standish, & Androwich, 2009; Keohane et al., 2008). Rates of MA errors vary widely, from 0.01% to 20%. Bates, Boyle, Vander Vliet, Schneider, and Leape (1995) found that, among preventable adverse drug events (ADEs), 26% errors occurred during administration, and none were intercepted by others. Although a source of ADEs resulting in patient harm, nurses also serve as a protective role. Of the ADEs intercepted before reaching the patient, most are intercepted by the nurse (Cullen, Bates, & Leape, 2000; Leape et al., 1995; Rogers, Dean, Hwang, & Scott, 2008). Most of the MA errors go unreported because the clinician is unaware that an error has occurred (Barker, Flynn, Pepper, Bates, & Mikeal, 2002). As the last line of defense against errors, nurses must reliably perform fundamental safe practices ensuring MA accuracy.
Aims of This Study This study presents findings from the Collaborative Alliance for Nursing Outcomes’ (CALNOC) ongoing study of acute care MA accuracy. Aims of the study were as follows:
r To describe the CALNOC na¨ıve observation MA accuracy assessment method.
r To examine nurse adherence to six fundamental safe practices during MA.
r To examine the prevalence of MA errors in adult acute care.
r To explore associations between nurse deviation from fundamental safe practices and MA errors in adult acute care.
Review of the Literature The turbulent hospital environment in which nurses administer medications was explicated by Jennings, Sandelowski, and Mark (2011) in an ethnographic study of MA on two adult acute care units in a large community hospital. Following field observations and interviews, the
Vol. 36 No. 6 November/December 2014
authors concluded “the nurse’s day was defined by medication administration” (p. 1442). In a systematic review of 23 studies from 1980 to 2008, Biron, Loiselle, and Lavoie-Tremblay (2009) examined the effect of nurses’ work interruptions on MA accuracy. The authors found on average 67 work interruptions per hour, which were based on 14 studies. One nonexperimental study showed a significant association between work interruptions and MA errors (Scott-Cawiezell et al., 2007). The Medication Administration Accuracy Assessment was developed in 2004 by the CALNOC, a self-sustaining, not-for-profit, nursing sensitive benchmarking registry serving nearly 300 hospitals in six states with unit, service line, and facility level benchmarks for structure, process, and outcome measures. CALNOC metrics, methods, and benchmarks have been reported elsewhere (Aydin et al., 2004; Brown, Aydin, Donaldson, Fridman, & Sandhu, 2010; Brown, Donaldson, Burnes Bolton, & Aydin, 2010). CALNOC’s MA accuracy assessment was based on the findings from published direct observation studies (Barker et al., 2002; Pepper, 1995). Barker and colleagues (2002) found that the vast majority of MA errors are unreported and direct observation, plus medical record review, was the most reliable method to determine MA accuracy. The validity and precision of direct observation was reaffirmed by Meyer-Massetti and colleagues (2011) in a systematic review of medication safety assessment methods, concluding no single method was superior to the other and the combination of multiple assessment methods revealed different types of MA errors. The authors noted the value of direct observation as a compliment to ADE incident reporting systems. The CALNOC MA accuracy assessment method has been successfully used for research and quality improvement (Ching, Long, Williams, & Blackmore, 2013; Cochran, 2009; Helmons, Wargel, & Daniels, 2009; Kliger, Blegen, Gootee, & O’Neil, 2009).
Study Design and Methods This cross-sectional study used MA direct observations and a prevalence or point in time sampling approach with a minimum sample of 100 observed doses per patient care unit. A fundamental assumption of the prevalence sampling approach is that it is representative
of the actual error incidence (Barker, Flynn, & Pepper, 2002). To ensure the reliability and validity of the data, observers from participating hospitals were trained by CALNOC in a standardized workshop.
Institutional Review Board (IRB) CALNOC has sustained IRB approval at Cedars-Sinai Medical Center and University of California San Francisco since 1998. It should be noted that individual nurses were not identified during observations. Observers, blinded to patient histories and medication orders, were unable to proactively identify errors; however, they were instructed to interrupt administration of a medication if they observed an obvious error that would potentially harm the patient. Errors discovered during chart review that affected patient safety were reported to the patient’s nurse.
Sample Accrual and Characteristics MA accuracy assessment data collected by CALNOC hospitals for this study included patients in adult acute care units from July 2006 through October 2009. Each hospital determined which unit(s), staff, and shifts were observed as well as the timing of data collection. To minimize staff discomfort with the observation process, observers were trained to limit intrusion. Despite knowing the focus of the observations, staff made process errors and in many instances breached relevant hospital policies and procedures, suggesting that staff habituated to the observer’s presence as they performed routine tasks (Pepper & Blegen, 2004; Schnelle, Ouslander, & Simmons, 2006). The CALNOC MA assessment used the following definitions:
r MA accuracy: Administration of a dose of medication exactly as ordered by the physician, which includes the fundamental five rights—right drug, right patient, right dose, right route, and right time (Carlton & Blegen, 2006; Eisenhauer, Hurley, & Dolan, 2007; Elliott & Liu, 2010). r MA error: A dose administered differently than ordered by the physician. r Opportunity for error (OE): The basic unit of data in observational accuracy studies; OE included any dose ordered plus any
59
60
Journal for Healthcare Quality
unordered doses given and any ordered doses omitted. In conducting the MA assessment, nurse observers, purposefully “na¨ıve” to, the patient history and treatment, collected dose-level data in three steps: (1) systematic observation of six selected fundamental practices performed by the nurse during preparation, administration, and documentation of medications (a patient administered multiple doses was observed for all doses), (2) review of the medical record to extract relevant medication orders, and (3) comparison of medications administered and medications ordered to determine MA accuracy. Excluded medications included infusions and selfadministered medications. In performing tasks during the preparation, administration, and documentation of medications, nurses have typically been inculcated with the “five rights”—right patient, drug, dose, route, and time. Despite strong face validity, the “rights” were not evidence-based (Carlton & Blegen, 2006; Eisenhauer et al., 2007; Elliott & Liu, 2010). CALNOC operationalized the following safe practices from the “rights” literature and observers noted whether the nurse: 1. compared medication with MA record (MAR), 2. minimized distraction or interruption during medication preparation or administration, 3. ensured medication is labeled throughout the process from preparation to administration, 4. checked two forms of patient identification prior to administration of medication, 5. explained medication to patient or family as appropriate, 6. charted/documented MA immediately after completion. To determine MA accuracy, the observer compared medication orders in the patient’s record with observed medications administered, revealing 10 possible outcomes that were then coded according to the following definitions: 1. No error observed. 2. Unauthorized drug error: Administration of a dose never ordered for that patient.
3. Wrong dose error: Any dose of a drug (excluding an injectable drug) that contained the wrong number of dosage units (such as tablets) or is, in the judgment of the observer, more than 17% greater or less than the correct dosage. 4. Wrong form error: The administration of a drug dose in a different form than ordered by the prescriber when prescriber wrote for a specific dosage form. 5. Wrong route error: Medication administered to a patient using a different route than ordered. 6. Wrong technique error: Using an inappropriate procedure or improper technique in administration of a drug. Focus is on technique violations that can alter drug effect. 7. Extra dose error: Any dose given in excess of the total number of times authorized by physician order. 8. Omission error: (a) Failure to give an ordered dose that appears on the MAR by the time the next dose is due. Patient refusals or drugs appropriately withheld are not considered omissions. (b) Order found in the medical record that does not appear on the MAR. 9. Wrong time error: Administration of a dose more than 60 min before or after scheduled administration time. If food is involved in the order, the dose should be given within 30 min of scheduled time. 10. Drug not available error: Administration of a dose more than 60 min after scheduled administration time due to nonavailability of the medication.
Statistical Methods Statistical analyses were designed to address three fundamental questions: 1. What percentage of the time do direct care nurses perform the selected six safe practices during routine administration of medications? 2. What percentage of medication doses are administered differently than ordered? 3. To what extent does the performance of selected safe practices during MA predict MA accuracy?
Vol. 36 No. 6 November/December 2014
Statistical Procedures CALNOC MA accuracy data consisted of one record for each medication dose observed. Each unit’s observation “study” included a minimum sample of 100 doses. Each single dose record captured the presence or absence of both safe practice behaviors performed by the nurse and outcome, either no error or the type of error(s). Data were first analyzed and values were calculated at the dose level. Doses were then aggregated to the encounter level, with each encounter including all doses administered during a single observation of a single patient. For each patient who received more than one dose during a single observation, safe practices and MA errors for the encounter were summarized to avoid correlations among same encounter doses. As a patient was not likely to be sampled twice within a single unit level 100 dose observational period, encounters and patients typically constituted a one-to-one relationship. To aid in interpreting results, safe practices, with the exception of “distracted/interrupted,” were recoded so that a value of 1 indicated a safe practice was not observed (i.e., a deviation from safe practices) and 0 indicated the safe practice was performed. MA errors were coded with a value of 1. The omission error rate was reported and then excluded from further analyses because the majority of omitted doses were ordered but not given, thus not observable. Except for descriptive purposes, “Wrong Time” and “Wrong Technique” errors were excluded from analyses, as these errors were confounded by wide variability in observer reporting/interpretation and hospital policies. The remaining six types of errors were retained for further analysis. The prevalence of both safe practice deviations and MA errors was calculated individually and in aggregate for each safe practice and MA error type. Noting the possibility of multiple safe practice deviations and MA errors per dose and multiple doses per encounter, the following summary variables were calculated for each encounter: 1. Overall percentage of errors per encounter calculated as two measures: (a) Overall percentage of safe practice deviations per encounter: Total number of times any safe practice deviation was coded for
all doses, divided by total doses in the encounter × six possible practice deviations. (b) Overall percentage of MA errors per encounter: Total number of times any error was coded across all doses divided by total number of doses in the encounter × six possible errors. 2. Percentage of each individual safe practice deviation/MA type of error per encounter: Total number of times a specific practice deviation/MA type of error was coded across all doses in the encounter divided by total number of doses in the encounter for each safe practice deviation and MA error. The total number of practice deviations/MA errors for each type of deviation/error, and overall, were defined as the numerator counts in definitions 1 and 2 above to be used as the dependent variables in Poisson regression models. To answer questions (1) and (2), analyses included descriptive statistics for both safe practice deviations and MA errors, followed by an examination of the effect of selected safe practice deviations on errors (question 3) at the encounter level. However, analysis of the data at the encounter level required accommodation of the multilevel structure in the data. For question 3, generalized mixed (multilevel) linear models of the overall error count were fitted on the number of each of the safe practices separately (one per model). Random error terms for study, unit, and facility were used to account for the data hierarchy of encounters within study, within units, within facilities. MA error counts were assumed to follow a Poisson distribution. The number of doses administered during each encounter was used as an offset variable in all simple Poisson regression models and all models were adjusted for linear yearly trend during the 4 years of study. Incidence rate ratios for each individual safe practice were calculated using the estimated coefficients from the fitted Poisson regression models. The combined effect of the selected six safe practices on the overall error counts was measured by fitting a Poisson model using overall number of practice deviations per encounter as a covariate. Estimated coefficients and t-test p-values along with variance components for each level and the incidence rate ratios are reported for each model.
61
62
Journal for Healthcare Quality
Procedures Hospitals’ na¨ıve observers watched staff nurses prepare, administer, and document medications dose by dose. Observations were coded noting safe practices and the incidence of distractions and interruptions, plus the specifics of the medication dose, route, timing, and technique. Following completion of the MA pass per patient, or per cluster of patients, the observer reviewed the medical record and compared what was observed with medication orders, coding each dose either “No error” or the specific error(s). Data were submitted to CALNOC using customized Excel files with automated data checking and validation.
Results The sample included 333 MA accuracy assessment studies (minimum 100 doses each) drawn from 43 CALNOC hospitals and 157 units. The majority of doses were observed on medical– surgical units (74%) with the remaining doses observed equally on step-down (13%) and critical care units (13%) with 95% of doses administered by registered nurses. Descriptive statistics for key variables are displayed in Table 1. Most units completed one or two observation studies, with one unit completing seven studies; 27.30% of patients were administered one medication dose, 17.20% two doses, 11.60% three doses, 10.60% four doses, and 33.30% of patients had five doses or more up to a maximum of 31 doses. Omission errors comprised 1% of all doses (318/33,643) and are excluded from further analyses. The remaining data included 33,325 doses administered to 8,594 patients.
Frequency of MA Safe Practice Deviations and MA Errors Due to coding variability in wrong time (3.30% of dose administrations and 2.96% of encounters) and wrong technique errors (2.33% of single dose administrations and 2.88% of all MA encounters), these errors were excluded from further analyses. A summary of safe medication practice deviations and MA errors categorized by dose and encounter is presented in Table 2. The most common safe practice deviations observed per encounter were as follows: nurse distraction/interruption (22.89%), medication not explained to patient (13.90%), two IDs not checked (12.47%), and MA not documented
immediately (10.68%). The most common MA errors were drug not available (0.76%) and wrong dose (0.45%). The overall percentage of safe practice deviations per encounter was 11.40%, whereas the overall percentage of MA errors was 0.32%.
Impact of Safe Practices on MA Accuracy Findings reveal a significant adjusted effect of each individual practice deviation on overall MA error counts (Table 3). The individual practice deviations of “Not compared to MAR,” “Not labeled throughout process,” “Not documented immediately,” “Two forms of ID not checked,” and “Medication not explained to patient” had large effect sizes. Examining the association of overall number of practice deviations with overall number of MA errors per encounter revealed that a reduction by one safe practice deviation in the overall number of practice deviations per encounter was associated with an estimated 4.37% decrease in the number of MA errors (p = .0001). Examination of sources of MA error variability partitioned by the hospital, unit, and study levels reveals that the largest proportion of error variability was associated with between studies variation, followed closely by between facility variation. A much smaller fraction of outcome error variability was associated with variation between units (Table 3). Findings from this study predict that for 10,000 MA encounters, there will be 27,630 safe practice deviations and 770 MA errors. We estimate that a reduction by one safe practice deviation per encounter will prevent 34 (4.40%) MA errors. These findings support the impact of safe practices on MA safety and have the potential to guide crucial improvements in patient care quality, safety, and costs.
Limitations The convenience sample included all studies conducted by CALNOC registry hospitals during the specified time period. It is possible that unmeasured systematic differences differentiate this sample from the population of hospitals at large. Variation in observer coding likely introduced coding error despite efforts to foster interrator reliability. We found that two errors (wrong technique and wrong time) were confounded by variability in hospital policies and
Vol. 36 No. 6 November/December 2014
Table 1. Summary of Hospital, Unit, and Encounter (Patient) Characteristics Sample Characteristics Number of doses Number of encounters (patients) Number of studies Number of units Number of hospitals
July 2006 Through October 2009 33,325 8,594 333 157 43
Distribution of units by unit type
Number
Percentage
107 26 24 157
68.2 16.5 15.3 100
Medical/surgical Step down Critical care Total units Distribution of encounters by unit type Medical/surgical Critical care Step down Total encounters Patient age (mean)
Encounter (patient)characteristics Number Number
6,378 1,130 1,086 8,594 59.3 Primary reason for hospitalization
Percentage 74.2 13.2 12.6 100
Medical diagnosis (%) Surgical diagnosis (%)
65.9 34.1 Patient gender
Male (%) Female (%)
51.7 48.3
Table 2. Percentage of Safe Practice Deviations and Medication Administration (MA) Errors
Measure
Mean Percentage Per encounter (N = 157 units)
Per Dose Percentage (N = 33,325)
2.83 22.89 5.64 12.47 13.90 10.68 11.40
2.49 25.02 5.48 12.31 15.48 10.66 11.88
0.76 0.45 0.28 0.24 0.12 0.08 0.32
0.86 0.44 0.32 0.20 0.09 0.08 0.33
Safe practice deviation typea Not compared with MAR Nurse distracted/interrupted Not labeled throughout process Two forms of ID not checked Not explained to patient Not documented immediately after administration Overall percentage of safe practice deviationsa Medication administration outcome error typeb Drug not available Wrong dose Wrong route Unauthorized drug Extra dose Wrong form Overall percentage of errorsb
a Total number of times the specific error was coded for all doses in the encounter divided by the total number of doses in the encounter. Result was multiplied by 100 to obtain a percentage. b Total number of times any deviation was coded for all doses in encounter divided by the total number of doses in the encounter × six possible errors. Result was multiplied by 100 to obtain a percentage.
63
b
a
(0.014) 0.106 (0.013) 0.095 (0.012) 0.030 (0.013) 0.045 (0.004)
0.214 (0.021) 0.120 (0.019) 0.129
Slope (SE)
4.37
0.50
1.57
1.75
2.13
3.27
3.50
Incidence Rate Ratio (%)
0.0001
0.0189
0.0001
0.0001
0.0001
0.0001
0.0001
p - Value
0.41
0.44
0.51
0.52
0.48
0.45
0.49
Between facilities
0.07
0.16
0.10
0.15
0.08
0.11
0.11
Between units
Variance Components
0.68
0.59
0.66
0.58
0.66
0.66
0.65
Between studies
The dependent variable is overall outcome error count. Each table row reports on a separate model. Incidence rate ratio calculated for a decrease by one-sixth for each of the six individual practice deviation types and for a decrease by 1 for the overall practice deviation total to make the results comparable.
Overall number of safe practice deviations
Distracted/interrupted
Not explained to patient
Two forms of ID not checked
Not charted/documented immediately after administration
Not labeled throughout process
Not compared with MAR
Model Predictor
b
Table 3. Poisson Regression Models for Each Safe Practice Deviation on Overall Medication Administration (MA) Error Counts Using Encounter Level Data and Adjusted for Linear Yearly Trend (N = 8,594)a
64 Journal for Healthcare Quality
Vol. 36 No. 6 November/December 2014
procedures, limiting confidence in these error types. It is noteworthy that the extent to which hospitals used the same staff observers across multiple observation assessments is unknown, introducing the potential for coding “drift” due to observer staffing variability.
Directions for Future Research Analysis to explore the extent to which multiple observations per hospital may have affected these findings did not reveal significant differences in rates over time when taking into account same hospital repeated assessments and the time elapsed between assessments. Future research will address this question in more detail.
Discussion This study provides the most robust sample of MA doses directly observed yet reported. The overall MA error rate was substantially lower than previous reports using similar methods with overall error rates ranging from 4% to 19% (Barker et al., 2002; Helmons et al., 2009; Kliger et al., 2009; Westbrook, Woods, & Rob, Dunsmuir, & Day, 2010). Safe practice deviations were generally comparable to previous pre– post design reports of baseline performance, although design variation confounds comparisons (Helmons et al., 2009; Kliger et al., 2009; Westbrook et al., 2010). These results support previous findings associating interruptions and distractions with MA errors (Westbrook et al., 2010), although the volume of interruptions observed in the CALNOC dataset were half reported by Westbrook and colleagues (2010). The finding of greater variation between unit “studies,” rather than units alone or hospitals, suggests the importance of microsystem factors unique to shifts on units, for example, culture of safety or availability of support services. Further inquiry exploring the impact of microsystem characteristics is vital to greater understanding of this finding. Variation in staffing hours or skill mix may be viewed as less influential, given the impact of mandated nurse–patient ratios in a majority of CALNOC hospitals and the finding that 95% of medication doses were administered by RNs. Unlike other research using direct observation to measure MA accuracy, these data were derived from a performance improvement
registry dataset that reflected the quality imperatives of participating hospitals rather than the design precision of research studies (Barker et al., 2002; Helmons et al., 2009; Kliger et al., 2009; Westbrook et al., 2010). Although CALNOC’s data were obtained by nurse observers, other investigators have noted the benefits of using pharmacists (Helmons et al., 2009), suggesting the potential for collaboration between nurses and pharmacists in the quest to better understand MA safety.
Implications for Practice Findings confirm the usefulness of the CALNOC MA accuracy assessment for examining medication safety within and between hospitals and the link between safe practices and MA accuracy. Despite the resources and training required to conduct unit-level MA assessment, 43 hospitals voluntarily submitted data to CALNOC, suggesting they considered the direct observation process important to their medication safety improvement efforts. Hospital feedback indicated that the CALNOC assessment added value by providing a point-in-time systematic assessment of the reliability of incident-reporting systems and the opportunity to target improvement at the unit-level based on assessment findings. Ultimately, these findings affirm that nurse adherence to safe practices reduces MA errors. Although MA errors account for a minority of overall ADEs, they are typically not intercepted by other members of the healthcare team. Integrating these findings into evidence-based staffing considerations, nursing education and clinical practice competency validation has the potential to improve the quality, cost, and outcomes of MA.
Acknowledgments This study was funded, in part, by grants from the Gordon and Betty Moore Foundation, Betty Irene Moore Nursing Initiative, and the Robert Wood Johnson Foundation, Interdisciplinary Nursing Quality Research Initiative.
References Adverse events in hospitals: National incidence among Medicare beneficiaries. (2010). Report No. OEI-06-0900090. Washington, DC: U.S. Department of Health and Human Services. Armitage, G., & Knapman. H. (2003). Adverse events in drug administration: A literature review. Journal of Nursing Management, 11, 130–140.
65
66
Journal for Healthcare Quality
Aydin, C. E., Bolton, L. B., Donaldson, N., Brown, D. S., Buffum, M., et al. (2004). Creating and analyzing a statewide nursing quality measurement database. Journal of Nursing Scholarship, 36, 371–378. Barker, K. N., Flynn, E. A., & Pepper, G. A. (2002). Observation method in detecting medication errors. American Journal of Health-System Pharmacy, 59, 2314–2316. Barker, K. N., Flynn, E. A., Pepper, G. A., Bates, D. W., & Mikeal, R. L. (2002). Medication errors observed in 36 health care facilities. Archives of Internal Medicine, 162, 1897–1903. Bates, D. W., Boyle, D. L., Vander Vliet, M. B., Schneider, J., & Leape, L. (1995). Relationship between medication errors and adverse drug events. Journal of General Internal Medicine, 10, 199–205. Biron, A. D., Loiselle, C. G., & Lavoie-Tremblay, M. (2009). Work interruptions and their contribution to medication administration errors: An evidence review. Worldviews on Evidence-based Nursing, 6, 70–86. Brown, D. S., Aydin, C. E., Donaldson, N., Fridman, M., & Sandhu, M. (2010). Benchmarking for small hospitals: Size didn’t matter! Journal for Healthcare Quality, 32, 50– 60. Brown, D. S., Donaldson, N., Burnes Bolton, L., & Aydin, C. E. (2010). Nursing-sensitive benchmarks for hospitals to gauge high-reliability performance. Journal for Healthcare Quality, 32, 9–17. Carlton, G., & Blegen, M. A. (2006). Medication-related errors: A literature review of incidence and antecedents. Annual Review of Nursing Research, 24, 19–38. Ching, J. M., Long, C., Williams, B. L., & Blackmore, C. C. (2013). Using lean to improve medication administration safety: In search of the “perfect dose”. Joint Commission Journal on Quality and Patient Safety, 39, 195–204. Cochran, G. (2009). Comparing the effectiveness of medication use systems in small rural hospitals. Rockville, MD: Agency for Healthcare Research and Quality. Cullen, D. J., Bates, D. W., & Leape, L. L. (2000). Prevention of adverse drug events: A decade of progress in patient safety. Journal of Clinical Anesthesiology, 12, 600–614. Eisenhauer, L. A., Hurley, A. C., & Dolan, N. (2007). Nurses’ reported thinking during medication administration. Journal of Nursing Scholarship, 39, 82–87. Elganzouri, E. S., Standish, C. A., & Androwich, I. (2009). Medication Administration Time Study (MATS): Nursing staff performance of medication administration. Journal of Nursing Administration, 39, 204–210. Elliott, M., & Liu, Y. (2010). The nine rights of medication administration: An overview. British Journal of Nursing, 19, 300–305. Helmons, P. J., Wargel, L. N., & Daniels, C. E. (2009). Effect of bar-code-assisted medication administration on medication administration errors and accuracy in multiple patient care areas. American Journal of Health-System Pharmacy, 66, 1202–1210. Jennings, B. M., Sandelowski, M., & Mark, B. (2011). The nurse’s medication day. Qualitative Health Research, 21, 1441–1451. Keohane, C. A., Bane, A. D., Featherstone, E., Hayes, J., Woolf, S., Hurley, A., et al. (2008). Quantifying nursing workflow in medication administration. Journal of Nursing Administration, 38, 19–26. Kliger, J., Blegen, M. A., Gootee, D., & O’Neil, E. (2009). Empowering frontline nurses: A structured intervention enables nurses to improve medication administration accuracy. Joint Commission Journal on Quality and Patient Safety, 35, 604–612.
Kohn, L. T., Corrigan, J. M., Donaldson, M. S., & Institute of Medicine. (2000). To err is human. Building a safer health system. Washington, DC: National Academy Press. Leape, L. L., Bates, D. W., Cullen, D. J., Cooper, J., Demonaco, H. J., Gallivan, T., et al. (1995). Systems analysis of adverse drug events. ADE Prevention Study Group. Journal of the American Medical Association, 274, 35–43. Meyer-Massetti, C., Cheng, C. M., Schwappach, D. L., Paulsen, L., Ide, B., Meier, C., et al. (2011). Systematic review of medication safety assessment methods. American Journal of Health-System Pharmacy, 68, 227–240. Pepper, G. A. (1995). Errors in drug administration by nurses. American Journal of Health-System Pharmacy, 52, 390–395. Pepper, G. A., & Blegen, M. A. (2004). How intrusive is direct observation of nursing care? Communicating Nursing Research, 37, 258. Rogers, A. E., Dean, G. E., Hwang, W. T., & Scott, L. D. (2008). Role of registered nurses in error prevention, discovery and correction. Quality & Safety in Health Care, 17, 117–121. Schnelle, J. F., Ouslander, J. G., & Simmons, S. F. (2006). Direct observations of nursing home quality: Does care change when observed? Journal American Medical Director Association, 7, 541–544. Retrieved August 11, 2013 from www.ncbi.nih.gov/pubmed/17095417. Scott-Cawiezell, J., Pepper, G. A., Madsen, R. W., Petroski, G., Vogelsmeier, A., & Zellmer, D. (2007). Nursing home error and level of staff credentials. Clinical Nursing Research, 16, 72–78. Westbrook, J. I., Woods, A., Rob, M. I., Dunsmuir, W. T. M, & Day, R. (2010). Associations of interruptions with an increased risk and severity of medication administration errors. Archives Internal Medicine, 170, 683–690.
Authors’ Biographies Nancy Donaldson, PhD, RN, FAAN, UCSF School of Nursing, San Francisco, CA. Donaldson is a clinical professor. Since 1996, Donaldson has served as the CoPrincipal Investigator for the Collaborative Alliance for Nursing Outcomes Project (CALNOC) and now serves as CALNOC Senior Scientist Emeritus. Carolyn Aydin, PhD, is a senior research scientist IV at Cedars-Sinai Medical Center and Burns & Allen Research Institute, Los Angeles, CA. Aydin is also a CALNOC data manager and coinvestigator. Moshe Fridman, PhD, is the president and owner of AMF Consulting, Inc., Los Angeles, CA, since 1998 till present. Fridman provides clients with data management and statistical analysis consulting services and head analysis teams and provides technical support for long-term projects (large clinical trials, grants). Mary Foley, PhD, RN, is the director of the Center for Nursing Research and Innovation at the University of California, San Francisco (UCSF) School of Nursing. Formerly she was the president of the American Nurses Association. For more information on this article, contact Nancy Donaldson at [email protected].
Vol. 36 No. 6 November/December 2014
Journal for Healthcare Quality is pleased to offer the opportunity to earn continuing education (CE) credit to those who read this article and take the online posttest at http://www. nahq.org/education/content/jhq-ce.html. This continuing education offering, JHQ 251, will provide 1 contact hour to those who complete it appropriately.
Core CPHQ Examination Content Area IV. Patient Safety
Objectives r Describe the 3 basic steps in the na¨ıve observation medication administration accuracy assessment method. r Identify 6 fundamental safe practices during medication administration that are key to ensuring medication delivery accuracy. r Confirm the most prevalent type of medication administration error found in the CALNOC study.
Multiple Choice Questions 1. Reviewers of adverse event data for the 2010 report by the US Office of the Inspector General concluded the most common preventable adverse events were related to: a. Culture of safety b. Medications c. Administrative oversight d. Poor communication 2. Most medication administration errors go unreported because: a. The clinician is unaware an error has occurred. b. They did not harm the patient. c. Adverse event reporting systems are not designed to capture these data. d. The clinician is embarrassed and avoids reporting. 3. Use of direct observation for medication administration accuracy assessment is: a. Not as reliable as adverse event reporting systems. b. More reliable than adverse event reporting systems, when combined with medical record review. c. Strictly a research method. d. None of the above
4. The CALNOC medication administration accuracy assessment uses na¨ıve observers to collect data to ensure: a. Observers are less experienced than the staff they are observing. b. Observers are friendly but not intrusive during observations. c. Observers are unaware of the patient’s actual medication orders. d. All the above 5. The CALNOC medication administration accuracy assessment includes: a. Systematic observation of six selected fundamental practices performed by the nurse during preparation, administration and documentation of medications. b. Review of the medical record to extract relevant medication orders. c. Comparison of medications administered and medications ordered to determine MA accuracy. d. All the above 6. Which of the following is not one of the 6 safe practices observed in the CALNOC medication administration accuracy assessment: a. The nurse compared the medication with MA record. b. The nurse ensured the medication was labeled throughout the process from preparation to administration. c. The nurse asked the patient to verify the purpose of the medication. d. The nurse charted/documented medication administration immediately after completion. 7. The most common safe practice deviation by the nurse observed in this study per encounter was: a. Nurse distraction/interruption b. Medication not explained to patient c. Two IDs not checked d. Medication administration not documented immediately 8. The most common medication administration error found in the CALNOC study was: a. Drug not available b. Wrong dose c. Wrong technique d. Wrong time
67
68
Journal for Healthcare Quality
9. Study findings revealed that: a. Safe practices are not significantly associated with medication administration accuracy. b. Reducing safe practice deviations per encounter would decrease medication administration errors. c. Not all safe practices are associated with medication accuracy. d. None of the above 10. Findings from the CALNOC study suggest:
a.
Future study is needed to explore the impact of microsystem characteristics on medication accuracy. b. CALNOC medication administration accuracy assessment is useful for examining medication safety within and between hospitals. c. The CALNOC assessment may be helpful in examining the reliability of adverse event reporting systems. d. All the above
