Nurse Educator
Nurse Educator Vol. 40, No. 2, pp. 83-86 Copyright * 2015 Wolters Kluwer Health, Inc. All rights reserved.
Using an Eye Tracker During Medication Administration to Identify Gaps in Nursing Students’ Contextual Knowledge An Observational Study Brian Amster, MS & Jenna Marquard, PhD & Elizabeth Henneman, PhD, RN & Donald Fisher, PhD In this clinical simulation study using an eye-tracking device, 40% of senior nursing students administered a contraindicated medication to a patient. Our findings suggest that the participants who did not identify the error did not know that amoxicillin is a type of penicillin. Eye-tracking devices may be valuable for determining whether nursing students are making rule- or knowledge-based errors, a distinction not easily captured via observations and interviews. Keywords: adverse drug event; eye-tracking device; medication administration; nursing students; simulation
T
his study builds on recently published work by Henneman et al,1 who found that 40% of nursing students failed to identify an allergy error when administering a medication to a simulated patient. The simulated patient was prescribed amoxicillin but had a documented allergy to penicillin. Henneman et al1 used an eye-tracking device to record what the nursing students looked at, did, said, and heard. The goal of the study described here was to understand whether nursing students failed to identify the allergy error because they did not locate the information required to identify the error (eg, patient medication orders, information on the patient’s allergy band, etc) or because they were unable to apply their knowledge of pharmacology during the medication administration process (eg, they did not know amoxicillin should not be given to a patient with an allergy to penicillin).
Background Harm to a patient from medical treatment or advice has long been a problem in health care, with adverse drug events Author Affiliations: PhD Student (Mr Amster), Associate Professor (Dr Marquard), Professor (Dr Fisher), College of Engineering, Associate Professor (Dr Henneman), College of Nursing, University of Massachusetts Amherst. This work was supported in part by a National Science Foundation award (no. 1032574), ‘‘BRIGE: Quantitative Model-Based Visualizations of Complex Health Care Processes.’’ The authors declare no conflicts of interest. Correspondence: Mr Amster, University of Massachusetts Amherst, 220 Engineering Laboratory, 160 Governors Dr, Amherst, MA 01002 ([email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.nurseeducatoronline.com). Accepted for publication: August 23, 2014 Published online before print: October 6, 2014 DOI: 10.1097/NNE.0000000000000097
Nurse Educator
(ADEs) being an especially significant problem.2,3 The consequences of ADEs can be severe, yet are often preventable.4,5 Several studies in the last 15 years have found that 10% to 20% of all hospitalized patients experience an ADE.6-8 Studies measuring the occurrence of ADEs likely provide low estimates because many errors go undocumented, unreported, and unnoticed.4,9 Of all causes of ADEs, allergy errors are especially prevalent. Several studies have found that up to 20% of all ADEs were associated with prescribing a medication to a patient with a known allergy to that medication class.6,10-13 Nurses play an important role in preventing, detecting, and intercepting medication errors, thereby helping to minimize ADEs and other adverse events.14-17 To prepare for this important role, nursing students need to acquire theoretical knowledge and learn to apply this knowledge in the context of clinical processes. Clinical simulation may help prepare nursing students for clinical settings, and some studies have embedded errors in the simulations to teach nursing students about the importance of patient safety.1,14,18,19 In their studies, Henneman et al have included errors such as a medication prescribed for a patient with the same name but different date of birth as the simulated patient, an order for a medication to which the patient was allergic, missing information in the medication orders (eg, route, dose), and an intravenous pump set to an incorrect rate. Henneman et al1,14 used a modified Eindhoven model to categorize errors made by the nursing students in these simulated settings. The modified Eindhoven model categorizes human errors into 3 major categories: rule-based, knowledgebased, and skill-based errors.1,20,21 Unfortunately, if nursing students give an incorrect medication to a patient in a simulated setting, it is difficult to discern whether the students did not have the necessary theoretical knowledge to notice Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
83
the error (knowledge-based error) or did not follow a systematic procedure that would allow them to gather enough information about the patient or their medications (rule-based error). Skill-based errors occur during routine tasks that require little or no attention, so are likely not applicable to nursing students in clinical simulations. If we can make this distinction, we can better target education efforts toward reducing nursing students’ knowledge- and rule-based errors.
Methods Design and Participants Nursing students (N = 12) completed the medication administration process in a simulated patient care setting. Participants were all senior-level nursing students, had received a passing grade in a pharmacology class, and had been exposed to about 20 hours of simulation time before the study was conducted. The participants were told that we were studying how they use visual cues to perform patient care procedures and were evaluating the usefulness of eye-tracking technology to understand and improve patient care decision making. The participants did not know we were studying allergy errors. The university’s institutional review board approved the study. Participants wore an eye-tracking device during this study (Figure, Applied Science Laboratories Mobile Eye, Bedford, Massachusetts). Marquard et al22 detail the eye-tracking device’s accuracy, specifications, and process for calibrating the device to each participant.
Procedures Each participant was asked to give a medication (amoxicillin) to a simulated patient, a mannequin in a bed. A human actor sat behind a 1-way transparent window. The human actor could see clearly the behavior of the participant, but the participant could not see the actor. The participant was allowed to interact with the simulated patient and could ask the patient any questions, such as ‘‘Do you have any allergies?’’ If asked, the human actor would state that she (the patient) had an allergy to penicillin. The simulated patient had a band on each wrist. On the patient’s left wrist was an ID band, and on the right wrist was an allergy band. The allergy band indicated that the patient had an allergy to penicillin.19
Figure. Mobile Eye Tracker developed by Applied Science Laboratories (ASL), www.asleyetracking.com. Copyright ASL. Reprinted by permission, 2014.
84
Each participant was given 2 additional documents: a patient chart and a medication chart designed to represent what a nurse would receive in a clinical setting. The patient chart included information about the patient’s medication names and doses. The patient chart ID area contained information about the patient’s demographics and stated the allergy to penicillin; the patient chart medication orders area included the patient’s order for amoxicillin. The medication chart allergy area also stated the patient’s allergy to penicillin. We termed these especially relevant areas, in addition to the allergy band and the patients’ statements about their allergies, areas of interest (AOIs). We considered the verbal step asking the patient about allergies as an AOI because it was 1 way that the participant could get information about the patient’s allergies. The remaining areas where the participant could have looked included the ID band, patient chart other orders, medication chart ID, and medication chart orders (ie, the location where the nurse documented medications administered), none of which contained relevant information for identifying the allergy error.
Data Coding We reviewed the eye-tracking videos for quality. Of the 12 videos, we discarded 2 videos (17%) because of insufficient video quality, leaving 10 videos (83%) for analysis. Two researchers who were not involved in this study and had no contact with the participants independently reviewed the videos and noted all medication administration–related events (both visual and verbal) performed by each participant. In the few cases when there was disagreement between these 2 researchers, a third researcher reviewed the video and made a final decision.
Results The patient’s prescription for amoxicillin was located in the patient chart medication orders area. This information had to be combined with information in at least 1 of the following other AOIs to realize that the patient was allergic to the prescribed medication: (1) patient chart ID, (2) medication chart allergy, (3) allergy band, or (4) asking the patient about allergies. The Table summarizes the number of fixations each participant directed toward the AOIs and if there were statistically significant differences in these values between those participants who identified the allergy error (n = 6) and those who did not identify the error (n = 4). The Table also includes the percentages of the participants’ total fixations that were directed on these AOIs. Participants who did not identify the error had more fixations overall and had a higher percentage of those fixations on AOIs, although the differences were not statistically significant (P = 0.22). Figure, Supplemental Digital Content 1, depicts the percentages of fixations on each AOI, http://links.lww.com/NE/ A174. Overall, the distribution of fixations across the AOIs appears to be similar between the 2 groups. Figure, Supplemental Digital Content 2, depicts the percentage of participants in each group that fixated on both the patient chart medication orders AOI and each 1 of the other key AOIs, http://links.lww.com/NE/A175. Roughly the same percentage of participants in each group performed each of the 4 key combinations. Of note, all of the participants in both groups fixated on the patient chart medication orders and allergy
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Nurse Educator
band, and asked the patient if she had an allergy at least once. Despite completing these 2 combinations that would allow the participant to identify the error, 40% of the participants still gave amoxicillin to the patient. Figure, Supplemental Digital Content 3, depicts the number of fixations between the patient chart medication orders area and each 1 of the other key AOIs, http://links.lww.com/NE/ A176. This measure is important because one would presume that the fewer fixations between 2 AOIs, the more likely the participant would be to identify the error using that combination. The number of fixations was similar between the 2 groups, except for the comparison between the patient chart medication orders and asking about the patient’s allergies; for that comparison, the group that did not identify the error had fewer fixations between the AOIs. Figure, Supplemental Digital Content 4, depicts the average number of fixations after viewing each of the 4 AOI combinations, for the participants in the group that identified the medication error, http://links.lww.com/NE/A177. Key combinations completed just before the end of the process might be the ones that were used to identify the error. The combination of patient chart medication orders and patient chart ID was completed closest to the end of the process; for both groups, this combination also had the fewest fixations between parts of the combination. Figure, Supplemental Digital Content 5, depicts a set of timeline belt visualizations, with the colors representing the areas where the participants fixated throughout the process, http://links.lww.com/NE/A178. For example, most participants began by fixating on the ID band, represented by the light blue color in the timeline belt visualization. Each box in the timeline belt visualizations represents a single fixation in a particular area. For instance, the first participant who did not identify the error completed 3 consecutive fixations on the ID band, followed by 1 fixation on the patient chart other orders area. The timeline belt visualizations are separated into participants who identified the error, in the top part of the visualization, and participants who did not identify the error, in the bottom part of the visualization. The timeline belt visualization shows that the participants’ orders and numbers of fixations appear to be similar in both groups (in that there are no consistent patterns except beginning by looking at the ID band). The most discernible difference was that 1 participant in the group who identified the error had far fewer fixations than did the other participants, and 1 par-
ticipant in the group who did not identify the error had a great deal more fixations than did the other participants.
Discussion In this study, there were no significant differences between the group of nursing students who identified the medication error and the group that did not with respect to the overall numbers of fixations. The distributions of fixations across the AOIs were similar between the 2 groups, although those participants who identified the error had a lower percentage of their fixations on AOIs. Only 1 key combination between the patient chart medication orders and another AOI was necessary to identify the allergy error. Every nursing student in this study performed at least 2 of the key combinations because every student looked at the patient chart medication orders and allergy band and asked the patient if she had any allergies, to which the patient answered, ‘‘Yes, I am allergic to penicillin’’ (see Figure, Supplemental Digital Content 2, http://links.lww.com/NE/A175). In addition, there was no difference between the 2 groups with respect to the number of fixations between each of the key combinations. All participants who looked at both the patient chart medication orders and the patient chart ID (65% for those who identified the error and 72% for those who did not) either gave the amoxicillin to the simulated patient or stated that there was an error immediately after viewing this combination (see Figure, Supplemental Digital Content 3, http://links.lww.com/NE/A176). This suggests that neither group was distracted by intervening fixations on non-AOIs. The timeline belt visualization (see Figure, Supplemental Digital Content 4, http://links.lww.com/NE/A177) shows that the orders of the fixations also appear to similar in both groups. In the studies of Henneman et al, the categories used to classify students’ errors were process oriented (rule based).1,14 Their category ‘‘error related to patient allergy information’’ assumed a required level of pharmacological knowledge and focused on the processes of checking correct patient identifiers and patient allergies prior to giving medication. Our study suggests that the nursing students made knowledge-based errors because there were no differences in the procedures used by the group that identified the medication error and the group that did not. If the deficiencies in the knowledge base of the nursing students in this simulation study are a reflection of what could happen in an actual clinical setting, perhaps these deficiencies should be addressed along with the procedural
Table. Summary of Fixations on AOIs Identified the Error Mean (SD) Total fixations Fixations on patient chart medication orders Fixations on patient chart ID Fixations on medication chart allergy Fixations on allergy band % of Fixations on AOIs
22.3 6.3 2.3 0.3 2.0
(11.8) (4.8), 28.3% (3.0), 10.3% (0.8), 1.3% (1.6), 9.0% 48.9%
Did Not Identify the Error Mean (SD)
P
39.0 (20.9) 16.3 (7.2), 41.8% 5.0 (5.4), 12.8% 0.3 (0.5), 0.8% 1.0 (0.0), 2.6% 57.9%
0.22 0.06 0.41 0.85 0.17
Includes percentages of participants’ total fixations that were directed on AOIs.
Nurse Educator
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
85
failures as defined by Henneman and Gawlinski.20 In addition, eye-tracking devices may be valuable for determining whether nursing students are making rule- or knowledge-based errors, a distinction not easily captured via observations and interviews.
Limitations This study has several limitations. There were only 10 participants in this analysis, and it was conducted in a single nursing program. Yet, all of the nursing students exhibited similar visual scanning patterns, suggesting that our findings may be representative of a larger population. Further studies should examine other programs with different students to see if these findings can be replicated. In addition, this study used a post hoc analysis because the nursing students who identified the error, and the students who did not identify it were identified after the data were collected. This was necessary because there was no way to randomly assign participants into groups who would either identify the error or not. We also assumed Just and Carpenter’s23 eye-mind hypothesis is true, meaning that when participants fixate on an object they are also thinking about what they are looking at. This assumption may not always hold true, but provides a reasonable starting point for this type of research. In addition, some nursing students may not have wanted to question the authority of the person who wrote the order for the simulated patient. This might be an area to be studied in future research.
Conclusions On several metrics, participants who identified the error and those who did not completed the medication administration process in a similar manner. All participants collected enough information to be able to identify the medication error. Our findings suggest that the participants who did not identify the error may have lacked the knowledge that amoxicillin is a penicillin. Nursing students may be able to apply their procedural rule-based knowledge in the context of the clinical process, but if they do not have a better knowledge of medication classes and the most common medications in those classes, even the best procedures may not save the patient from an ADE. Furthermore, our findings suggest that eyetracking devices may be a valuable tool to distinguish between students’ knowledge- and rule-based errors.
References 1. Henneman EA, Cunningham H, Fisher DL, et al. Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimens Crit Care Nurs. 2014;33(3):129-135. 2. Moser RH. Diseases of Medical Progress: A Study of Iatrogenic Disease; A Contemporary Analysis of Illness Produced by Drugs and Other Therapeutic Procedures. Springfield, IL: Charles C. Thomas; 1969. 3. Tache´ SV, So¨nnichsen A, Ashcroft DM. Prevalence of adverse drug events in ambulatory care: a systematic review. Ann Pharmacother. 2011;45(7-8):977-989. 4. Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events:
86
a problem for quality improvement. Jt Comm J Qual Improv. 1995;21(10):541-548. 5. Gurwitz JH, Field TS, Judge J, et al. The incidence of adverse drug events in two large academic long-term care facilities. Am J Med. 2005;118(3):251-258. 6. Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. JAMA. 1997;277(4):312-317. 7. Villaman ˜ ´an E, Larrubia Y, Ruano M, et al. Medication errors associated to notification of drug allergies: effect of computerized order entry on their prevention. Int Res Pharm Pharmacol. 2011; 1(8):203-210. Available at http://interesjournal.org/IRJPP/Pdf/ 2011/November/Villaman%C3%A1n%20et%20al.pdf. Accessed July 2, 2014. 8. Gomes ER, Demoly P. Epidemiology of hypersensitivity drug reactions. Curr Opin Allergy Clin Immunol. 2005;5(4):309-316. 9. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000. 10. Lesar TS, Briceland LL, Delcoure K, Parmalee JC, Masta-Gornic V, Pohl H. Medication prescribing errors in a teaching hospital. JAMA. 1990;263(17):2329-2334. 11. Lesar TS, Lomaestro BM, Pohl H. Medication-prescribing errors in a teaching hospital: a 9-year experience. Arch Intern Med. 1997;157(14):1569-1576. 12. Pohl S, Reis MD, Forjuoh SN. Medication allergy documentation in ambulatory care: a case report of errors and missed opportunities quantified during the unique transition from paper records to electronic medical records. Internet J Fam Pract. 2010;8(1). Available at http://ispub.com/IJFP/8/1/6306. Accessed July 2, 2014. 13. Benkhaial A, Kaltschmidt J, Weisshaar E, Diepgen TL, Haefeli WE. Prescribing errors in patients with documented drug allergies: comparison of ICD-10 coding and written patient notes. Pharm World Sci. 2009;31(4):464-472. 14. Henneman EA, Roche JP, Fisher DL, et al Error identification and recovery by student nurses using human patient simulation: opportunity to improve patient safety. Appl Nurs Res. 2010;23(1): 11-21. 15. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002;346(22):1715-1722. 16. Rogers AE, Dean GE, Hwang WT, Scott LD. Role of registered nurses in error prevention, discovery and correction. Qual Saf Health Care. 2008;17(2):117-121. 17. Henneman EA, Gawlinski A, Giuliano KK. Surveillance: a strategy for improving patient safety in acute and critical care units. Crit Care Nurse. 2012;32(2):e9-e18. 18. Katz GB, Peifer KL, Armstrong G. Assessment of patient simulation use in selected baccalaureate nursing programs in the United States. Simul Healthc. 2010;5(1):46-51. 19. Henneman EA, Cunningham H. Using clinical simulation to teach patient safety in an acute/critical care nursing course. Nurse Educ. 2005;30(4):172-177. 20. Henneman EA, Gawlinski A. A ‘‘near-miss’’ model for describing the nurse’s role in the recovery of medical errors. J Prof Nurs. 2004; 20(3):196-201. 21. van der Schaaf TW. Near miss reporting in the chemical process industry. Unpublished doctoral dissertation, Eindhoven University of Technology, Eindhoven, The Netherlands; 1992 22. Marquard JL, Henneman PL, He Z, Jo J, Fisher DL, Henneman EA. Nurses’ behaviors and visual scanning patterns may reduce patient identification errors. J Exp Psychol Appl. 2011;17(3):247-256. 23. Just MA, Carpenter PA. A theory of reading: from eye fixations to comprehension. Psychol Rev. 1980;87(4):329-354.
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Nurse Educator
Nurse Educator Vol. 40, No. 2, pp. 83-86 Copyright * 2015 Wolters Kluwer Health, Inc. All rights reserved.
Using an Eye Tracker During Medication Administration to Identify Gaps in Nursing Students’ Contextual Knowledge An Observational Study Brian Amster, MS & Jenna Marquard, PhD & Elizabeth Henneman, PhD, RN & Donald Fisher, PhD In this clinical simulation study using an eye-tracking device, 40% of senior nursing students administered a contraindicated medication to a patient. Our findings suggest that the participants who did not identify the error did not know that amoxicillin is a type of penicillin. Eye-tracking devices may be valuable for determining whether nursing students are making rule- or knowledge-based errors, a distinction not easily captured via observations and interviews. Keywords: adverse drug event; eye-tracking device; medication administration; nursing students; simulation
T
his study builds on recently published work by Henneman et al,1 who found that 40% of nursing students failed to identify an allergy error when administering a medication to a simulated patient. The simulated patient was prescribed amoxicillin but had a documented allergy to penicillin. Henneman et al1 used an eye-tracking device to record what the nursing students looked at, did, said, and heard. The goal of the study described here was to understand whether nursing students failed to identify the allergy error because they did not locate the information required to identify the error (eg, patient medication orders, information on the patient’s allergy band, etc) or because they were unable to apply their knowledge of pharmacology during the medication administration process (eg, they did not know amoxicillin should not be given to a patient with an allergy to penicillin).
Background Harm to a patient from medical treatment or advice has long been a problem in health care, with adverse drug events Author Affiliations: PhD Student (Mr Amster), Associate Professor (Dr Marquard), Professor (Dr Fisher), College of Engineering, Associate Professor (Dr Henneman), College of Nursing, University of Massachusetts Amherst. This work was supported in part by a National Science Foundation award (no. 1032574), ‘‘BRIGE: Quantitative Model-Based Visualizations of Complex Health Care Processes.’’ The authors declare no conflicts of interest. Correspondence: Mr Amster, University of Massachusetts Amherst, 220 Engineering Laboratory, 160 Governors Dr, Amherst, MA 01002 ([email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.nurseeducatoronline.com). Accepted for publication: August 23, 2014 Published online before print: October 6, 2014 DOI: 10.1097/NNE.0000000000000097
Nurse Educator
(ADEs) being an especially significant problem.2,3 The consequences of ADEs can be severe, yet are often preventable.4,5 Several studies in the last 15 years have found that 10% to 20% of all hospitalized patients experience an ADE.6-8 Studies measuring the occurrence of ADEs likely provide low estimates because many errors go undocumented, unreported, and unnoticed.4,9 Of all causes of ADEs, allergy errors are especially prevalent. Several studies have found that up to 20% of all ADEs were associated with prescribing a medication to a patient with a known allergy to that medication class.6,10-13 Nurses play an important role in preventing, detecting, and intercepting medication errors, thereby helping to minimize ADEs and other adverse events.14-17 To prepare for this important role, nursing students need to acquire theoretical knowledge and learn to apply this knowledge in the context of clinical processes. Clinical simulation may help prepare nursing students for clinical settings, and some studies have embedded errors in the simulations to teach nursing students about the importance of patient safety.1,14,18,19 In their studies, Henneman et al have included errors such as a medication prescribed for a patient with the same name but different date of birth as the simulated patient, an order for a medication to which the patient was allergic, missing information in the medication orders (eg, route, dose), and an intravenous pump set to an incorrect rate. Henneman et al1,14 used a modified Eindhoven model to categorize errors made by the nursing students in these simulated settings. The modified Eindhoven model categorizes human errors into 3 major categories: rule-based, knowledgebased, and skill-based errors.1,20,21 Unfortunately, if nursing students give an incorrect medication to a patient in a simulated setting, it is difficult to discern whether the students did not have the necessary theoretical knowledge to notice Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
83
the error (knowledge-based error) or did not follow a systematic procedure that would allow them to gather enough information about the patient or their medications (rule-based error). Skill-based errors occur during routine tasks that require little or no attention, so are likely not applicable to nursing students in clinical simulations. If we can make this distinction, we can better target education efforts toward reducing nursing students’ knowledge- and rule-based errors.
Methods Design and Participants Nursing students (N = 12) completed the medication administration process in a simulated patient care setting. Participants were all senior-level nursing students, had received a passing grade in a pharmacology class, and had been exposed to about 20 hours of simulation time before the study was conducted. The participants were told that we were studying how they use visual cues to perform patient care procedures and were evaluating the usefulness of eye-tracking technology to understand and improve patient care decision making. The participants did not know we were studying allergy errors. The university’s institutional review board approved the study. Participants wore an eye-tracking device during this study (Figure, Applied Science Laboratories Mobile Eye, Bedford, Massachusetts). Marquard et al22 detail the eye-tracking device’s accuracy, specifications, and process for calibrating the device to each participant.
Procedures Each participant was asked to give a medication (amoxicillin) to a simulated patient, a mannequin in a bed. A human actor sat behind a 1-way transparent window. The human actor could see clearly the behavior of the participant, but the participant could not see the actor. The participant was allowed to interact with the simulated patient and could ask the patient any questions, such as ‘‘Do you have any allergies?’’ If asked, the human actor would state that she (the patient) had an allergy to penicillin. The simulated patient had a band on each wrist. On the patient’s left wrist was an ID band, and on the right wrist was an allergy band. The allergy band indicated that the patient had an allergy to penicillin.19
Figure. Mobile Eye Tracker developed by Applied Science Laboratories (ASL), www.asleyetracking.com. Copyright ASL. Reprinted by permission, 2014.
84
Each participant was given 2 additional documents: a patient chart and a medication chart designed to represent what a nurse would receive in a clinical setting. The patient chart included information about the patient’s medication names and doses. The patient chart ID area contained information about the patient’s demographics and stated the allergy to penicillin; the patient chart medication orders area included the patient’s order for amoxicillin. The medication chart allergy area also stated the patient’s allergy to penicillin. We termed these especially relevant areas, in addition to the allergy band and the patients’ statements about their allergies, areas of interest (AOIs). We considered the verbal step asking the patient about allergies as an AOI because it was 1 way that the participant could get information about the patient’s allergies. The remaining areas where the participant could have looked included the ID band, patient chart other orders, medication chart ID, and medication chart orders (ie, the location where the nurse documented medications administered), none of which contained relevant information for identifying the allergy error.
Data Coding We reviewed the eye-tracking videos for quality. Of the 12 videos, we discarded 2 videos (17%) because of insufficient video quality, leaving 10 videos (83%) for analysis. Two researchers who were not involved in this study and had no contact with the participants independently reviewed the videos and noted all medication administration–related events (both visual and verbal) performed by each participant. In the few cases when there was disagreement between these 2 researchers, a third researcher reviewed the video and made a final decision.
Results The patient’s prescription for amoxicillin was located in the patient chart medication orders area. This information had to be combined with information in at least 1 of the following other AOIs to realize that the patient was allergic to the prescribed medication: (1) patient chart ID, (2) medication chart allergy, (3) allergy band, or (4) asking the patient about allergies. The Table summarizes the number of fixations each participant directed toward the AOIs and if there were statistically significant differences in these values between those participants who identified the allergy error (n = 6) and those who did not identify the error (n = 4). The Table also includes the percentages of the participants’ total fixations that were directed on these AOIs. Participants who did not identify the error had more fixations overall and had a higher percentage of those fixations on AOIs, although the differences were not statistically significant (P = 0.22). Figure, Supplemental Digital Content 1, depicts the percentages of fixations on each AOI, http://links.lww.com/NE/ A174. Overall, the distribution of fixations across the AOIs appears to be similar between the 2 groups. Figure, Supplemental Digital Content 2, depicts the percentage of participants in each group that fixated on both the patient chart medication orders AOI and each 1 of the other key AOIs, http://links.lww.com/NE/A175. Roughly the same percentage of participants in each group performed each of the 4 key combinations. Of note, all of the participants in both groups fixated on the patient chart medication orders and allergy
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Nurse Educator
band, and asked the patient if she had an allergy at least once. Despite completing these 2 combinations that would allow the participant to identify the error, 40% of the participants still gave amoxicillin to the patient. Figure, Supplemental Digital Content 3, depicts the number of fixations between the patient chart medication orders area and each 1 of the other key AOIs, http://links.lww.com/NE/ A176. This measure is important because one would presume that the fewer fixations between 2 AOIs, the more likely the participant would be to identify the error using that combination. The number of fixations was similar between the 2 groups, except for the comparison between the patient chart medication orders and asking about the patient’s allergies; for that comparison, the group that did not identify the error had fewer fixations between the AOIs. Figure, Supplemental Digital Content 4, depicts the average number of fixations after viewing each of the 4 AOI combinations, for the participants in the group that identified the medication error, http://links.lww.com/NE/A177. Key combinations completed just before the end of the process might be the ones that were used to identify the error. The combination of patient chart medication orders and patient chart ID was completed closest to the end of the process; for both groups, this combination also had the fewest fixations between parts of the combination. Figure, Supplemental Digital Content 5, depicts a set of timeline belt visualizations, with the colors representing the areas where the participants fixated throughout the process, http://links.lww.com/NE/A178. For example, most participants began by fixating on the ID band, represented by the light blue color in the timeline belt visualization. Each box in the timeline belt visualizations represents a single fixation in a particular area. For instance, the first participant who did not identify the error completed 3 consecutive fixations on the ID band, followed by 1 fixation on the patient chart other orders area. The timeline belt visualizations are separated into participants who identified the error, in the top part of the visualization, and participants who did not identify the error, in the bottom part of the visualization. The timeline belt visualization shows that the participants’ orders and numbers of fixations appear to be similar in both groups (in that there are no consistent patterns except beginning by looking at the ID band). The most discernible difference was that 1 participant in the group who identified the error had far fewer fixations than did the other participants, and 1 par-
ticipant in the group who did not identify the error had a great deal more fixations than did the other participants.
Discussion In this study, there were no significant differences between the group of nursing students who identified the medication error and the group that did not with respect to the overall numbers of fixations. The distributions of fixations across the AOIs were similar between the 2 groups, although those participants who identified the error had a lower percentage of their fixations on AOIs. Only 1 key combination between the patient chart medication orders and another AOI was necessary to identify the allergy error. Every nursing student in this study performed at least 2 of the key combinations because every student looked at the patient chart medication orders and allergy band and asked the patient if she had any allergies, to which the patient answered, ‘‘Yes, I am allergic to penicillin’’ (see Figure, Supplemental Digital Content 2, http://links.lww.com/NE/A175). In addition, there was no difference between the 2 groups with respect to the number of fixations between each of the key combinations. All participants who looked at both the patient chart medication orders and the patient chart ID (65% for those who identified the error and 72% for those who did not) either gave the amoxicillin to the simulated patient or stated that there was an error immediately after viewing this combination (see Figure, Supplemental Digital Content 3, http://links.lww.com/NE/A176). This suggests that neither group was distracted by intervening fixations on non-AOIs. The timeline belt visualization (see Figure, Supplemental Digital Content 4, http://links.lww.com/NE/A177) shows that the orders of the fixations also appear to similar in both groups. In the studies of Henneman et al, the categories used to classify students’ errors were process oriented (rule based).1,14 Their category ‘‘error related to patient allergy information’’ assumed a required level of pharmacological knowledge and focused on the processes of checking correct patient identifiers and patient allergies prior to giving medication. Our study suggests that the nursing students made knowledge-based errors because there were no differences in the procedures used by the group that identified the medication error and the group that did not. If the deficiencies in the knowledge base of the nursing students in this simulation study are a reflection of what could happen in an actual clinical setting, perhaps these deficiencies should be addressed along with the procedural
Table. Summary of Fixations on AOIs Identified the Error Mean (SD) Total fixations Fixations on patient chart medication orders Fixations on patient chart ID Fixations on medication chart allergy Fixations on allergy band % of Fixations on AOIs
22.3 6.3 2.3 0.3 2.0
(11.8) (4.8), 28.3% (3.0), 10.3% (0.8), 1.3% (1.6), 9.0% 48.9%
Did Not Identify the Error Mean (SD)
P
39.0 (20.9) 16.3 (7.2), 41.8% 5.0 (5.4), 12.8% 0.3 (0.5), 0.8% 1.0 (0.0), 2.6% 57.9%
0.22 0.06 0.41 0.85 0.17
Includes percentages of participants’ total fixations that were directed on AOIs.
Nurse Educator
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
85
failures as defined by Henneman and Gawlinski.20 In addition, eye-tracking devices may be valuable for determining whether nursing students are making rule- or knowledge-based errors, a distinction not easily captured via observations and interviews.
Limitations This study has several limitations. There were only 10 participants in this analysis, and it was conducted in a single nursing program. Yet, all of the nursing students exhibited similar visual scanning patterns, suggesting that our findings may be representative of a larger population. Further studies should examine other programs with different students to see if these findings can be replicated. In addition, this study used a post hoc analysis because the nursing students who identified the error, and the students who did not identify it were identified after the data were collected. This was necessary because there was no way to randomly assign participants into groups who would either identify the error or not. We also assumed Just and Carpenter’s23 eye-mind hypothesis is true, meaning that when participants fixate on an object they are also thinking about what they are looking at. This assumption may not always hold true, but provides a reasonable starting point for this type of research. In addition, some nursing students may not have wanted to question the authority of the person who wrote the order for the simulated patient. This might be an area to be studied in future research.
Conclusions On several metrics, participants who identified the error and those who did not completed the medication administration process in a similar manner. All participants collected enough information to be able to identify the medication error. Our findings suggest that the participants who did not identify the error may have lacked the knowledge that amoxicillin is a penicillin. Nursing students may be able to apply their procedural rule-based knowledge in the context of the clinical process, but if they do not have a better knowledge of medication classes and the most common medications in those classes, even the best procedures may not save the patient from an ADE. Furthermore, our findings suggest that eyetracking devices may be a valuable tool to distinguish between students’ knowledge- and rule-based errors.
References 1. Henneman EA, Cunningham H, Fisher DL, et al. Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimens Crit Care Nurs. 2014;33(3):129-135. 2. Moser RH. Diseases of Medical Progress: A Study of Iatrogenic Disease; A Contemporary Analysis of Illness Produced by Drugs and Other Therapeutic Procedures. Springfield, IL: Charles C. Thomas; 1969. 3. Tache´ SV, So¨nnichsen A, Ashcroft DM. Prevalence of adverse drug events in ambulatory care: a systematic review. Ann Pharmacother. 2011;45(7-8):977-989. 4. Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events:
86
a problem for quality improvement. Jt Comm J Qual Improv. 1995;21(10):541-548. 5. Gurwitz JH, Field TS, Judge J, et al. The incidence of adverse drug events in two large academic long-term care facilities. Am J Med. 2005;118(3):251-258. 6. Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. JAMA. 1997;277(4):312-317. 7. Villaman ˜ ´an E, Larrubia Y, Ruano M, et al. Medication errors associated to notification of drug allergies: effect of computerized order entry on their prevention. Int Res Pharm Pharmacol. 2011; 1(8):203-210. Available at http://interesjournal.org/IRJPP/Pdf/ 2011/November/Villaman%C3%A1n%20et%20al.pdf. Accessed July 2, 2014. 8. Gomes ER, Demoly P. Epidemiology of hypersensitivity drug reactions. Curr Opin Allergy Clin Immunol. 2005;5(4):309-316. 9. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000. 10. Lesar TS, Briceland LL, Delcoure K, Parmalee JC, Masta-Gornic V, Pohl H. Medication prescribing errors in a teaching hospital. JAMA. 1990;263(17):2329-2334. 11. Lesar TS, Lomaestro BM, Pohl H. Medication-prescribing errors in a teaching hospital: a 9-year experience. Arch Intern Med. 1997;157(14):1569-1576. 12. Pohl S, Reis MD, Forjuoh SN. Medication allergy documentation in ambulatory care: a case report of errors and missed opportunities quantified during the unique transition from paper records to electronic medical records. Internet J Fam Pract. 2010;8(1). Available at http://ispub.com/IJFP/8/1/6306. Accessed July 2, 2014. 13. Benkhaial A, Kaltschmidt J, Weisshaar E, Diepgen TL, Haefeli WE. Prescribing errors in patients with documented drug allergies: comparison of ICD-10 coding and written patient notes. Pharm World Sci. 2009;31(4):464-472. 14. Henneman EA, Roche JP, Fisher DL, et al Error identification and recovery by student nurses using human patient simulation: opportunity to improve patient safety. Appl Nurs Res. 2010;23(1): 11-21. 15. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002;346(22):1715-1722. 16. Rogers AE, Dean GE, Hwang WT, Scott LD. Role of registered nurses in error prevention, discovery and correction. Qual Saf Health Care. 2008;17(2):117-121. 17. Henneman EA, Gawlinski A, Giuliano KK. Surveillance: a strategy for improving patient safety in acute and critical care units. Crit Care Nurse. 2012;32(2):e9-e18. 18. Katz GB, Peifer KL, Armstrong G. Assessment of patient simulation use in selected baccalaureate nursing programs in the United States. Simul Healthc. 2010;5(1):46-51. 19. Henneman EA, Cunningham H. Using clinical simulation to teach patient safety in an acute/critical care nursing course. Nurse Educ. 2005;30(4):172-177. 20. Henneman EA, Gawlinski A. A ‘‘near-miss’’ model for describing the nurse’s role in the recovery of medical errors. J Prof Nurs. 2004; 20(3):196-201. 21. van der Schaaf TW. Near miss reporting in the chemical process industry. Unpublished doctoral dissertation, Eindhoven University of Technology, Eindhoven, The Netherlands; 1992 22. Marquard JL, Henneman PL, He Z, Jo J, Fisher DL, Henneman EA. Nurses’ behaviors and visual scanning patterns may reduce patient identification errors. J Exp Psychol Appl. 2011;17(3):247-256. 23. Just MA, Carpenter PA. A theory of reading: from eye fixations to comprehension. Psychol Rev. 1980;87(4):329-354.
Volume 40 & Number 2 & March/April 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Nurse Educator
