International Emergency Nursing 25 (2016) 43–47
Contents lists available at ScienceDirect
International Emergency Nursing j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / a a e n
The use and limits of eye-tracking in high-fidelity clinical scenarios: A pilot study Mark Browning BN (hon), RN, BHlthSc (Lecturer, Course Coordinator) a,*, Simon Cooper PhD, MEd, DipSocRes, BA, RGN (Professor, Emergency Care and Research Development) a, Robyn Cant PhD, MHlthSc (Research Fellow) a, Louise Sparkes MN, GCertEmerg Nsg, RN (Lecturer) a, Fiona Bogossian PhD MPH BAppSc DipAppSc(Nurs Ed) RN Midwife (Associate Professor, Director of Research) b, Brett Williams BAdultVocEd, GradCertIntensiveCareParamed, GradDipEmergHlth, MHlthSc, PhD, FPA (Associate Professor, Head of Paramedicine Department) c, Peter O’Meara BHA, MPP, PhD, FPA (Professor, Head of Paramedicine, Public & Community Health) d, Linda Ross BTeach, GradDipPhysEd, DipHlth(Amb), BParaStud, MHlthProfessEd (Lecturer) c, Graham Munro BHS, MHSM (Lecturer) d, Barbara Black RN,Grad Cert Peri-op Nursing, MHlthProfessEd (Lecturer) b a
School of Nursing and Midwifery, Monash University, Berwick, Vic. 3806, Australia School of Nursing & Midwifery, The University of Queensland, Herston Campus, QLD, Australia c Department of Community Emergency Health & Paramedic Practice, Monash University, Vic., Australia d La Trobe Rural Health School, La Trobe University, Bendigo, Vic., Australia b
A R T I C L E
I N F O
Article history: Received 22 January 2015 Received in revised form 30 July 2015 Accepted 4 August 2015 Keywords: Debriefing Eye-tracking Education Clinical Simulation Point of view (PoV) Gaze
A B S T R A C T
Aim: To explore the potential of mobile eye-tracking to identify healthcare students’ area of visual interest and its relationship to performance ratings. Background: Eye-tracking identifies an individual’s visual attention focus, and has been used as a training technique in medicine and in nursing. In this study participants wore a point of view (PoV) camera within a spectacle frame during simulation education experiences. Methods: Thirty-nine final year nursing and paramedicine students individually participated in three 8 minute clinical simulations with debriefing using videoed eye-tracking recordings. Coloured dots on the video depicted the participant’s pupil fixation on five targeted areas. Data extracted from the video camera were collated to report time spent on each target (their ‘gaze’). Results: The mean total gaze of expert designated targets in the environment for three 8 minute scenarios was 40–77%. Of 35 participants’ focus on three main areas of interest, their priority was the patient’s head (34%), the patient’s trunk (24%) and their clinical assistant (5%), with significant differences between nursing and paramedic disciplines (P < 0.05). Objectively rated clinical performance improved significantly by the third scenario (P ≤ 0.001). Participants were positive regarding use of eye tracking during debriefing. Conclusions: Eye tracking has the potential to enhance debriefing and educational outcomes, although there are limitations to gaze capture in high fidelity environments and resource cost is high. Further study is warranted to enable better understanding of how expert clinicians achieve high levels of performance. © 2016 Elsevier Ltd. All rights reserved.
1. Introduction
* Corresponding author. School of Nursing and Midwifery, Monash University, PO Box 1071, Narre Warren, Vic. 3805, Australia. Tel.: +61 3 9904 7128; fax: +61 3 9904 4655. E-mail address: [email protected] (M. Browning). http://dx.doi.org/10.1016/j.ienj.2015.08.002 1755-599X/© 2016 Elsevier Ltd. All rights reserved.
Eye movement tracking or eye-tracking is a technology that has emerged from the field of marketing, whilst more recently being trialled as a technique for the training of healthcare professionals. Eye-tracking can identify the focus of an individual’s attention and
44
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
Box 1. Definitions used Eye-tracking: measurement of eye movements based on where the eye’s pupil is focused, using video technology AoI (area of interest): visual attention tracked by camera Gaze: the total time an individual’s attention is given to visual target/s in the environment (i.e. fixation time) Gaze capture: timed attention to a visual target of more than 150 milliseconds – anything less is not noted by the human brain. PoV glasses: point of view glasses worn by participant contain a camera that is calibrated to each individual’s right eye retina.
measure eye movements based on where their pupil is focused (Poole and Ball, 2004). These measurements have been used in conjunction with other training assessments to examine how well clinicians perform procedures (Hermens et al., 2013; Poole and Ball, 2004). Some examples include investigation of nurses’ scanning patterns during medication administration (Ze, 2011); the use of eyetracking to explore patient identification errors in nursing (Henneman et al., 2010); and examination of electrocardiograph interpretation by novice expert and nurses (Horsley et al., 2014). We hypothesised that recording performance whilst wearing a point of view (PoV) camera within a spectacle frame during simulation education experiences would enhance debriefing for health professional students. Eye-tracking recordings may further enable detailed analysis of the student’s area of attention, inform their reflective review and, ultimately, assist decision-making and enhance performance (Box 1). Fig. 3. The eye tracking video output. Note the pupil tracking (red) overlaid with the PoV capture.
1.1. Eye-tracking technology
Fig. 1. The Tobii eye tracking headset worn by participants.
As humans, we make between three and five eye movements per second to examine and make sense of visual information (Holmqvist et al., 2011). Visual attention is the first step in making sense of complex information (Horsley et al., 2014). In clinical nursing, individuals need to collect and prioritise patient cues and make decisions about patient management using visual cues (Levett-Jones et al., 2010). Video technology incorporated into a pair of hightechnology glasses and computerised software developed by Tobii Technology (Fig. 1) enables timed data replay. The Glasses capture PoV video and the pupil movement is mapped over this (Figs 2 and 3). With the assumption that gaze fixation is normally the focus of thought (Duchowski, 2007), it follows that eye-tracking could substantiate the timing and sequence of clinical decision-making during a simulation experience. 2. Methods A pilot observational study was designed to explore the feasibility of using eye-tracking during high fidelity clinical simulation scenarios and the impact on students’ learning experience. The objective was to identify undergraduate healthcare students’ area of visual interest and its relationship to performance ratings. 2.1. Sample
Fig. 2. The eye tracking video output. Note the pupil tracking (red) overlaid with the PoV capture.
A convenience sample of 39 final year undergraduate degree students was recruited from three Australian universities in 2013. The universities were chosen in order to recruit a sample of students studying in two disciplines and from two Australian states. Nineteen participants were enrolled in a Paramedic Bachelor programme and 20 in a Nursing Bachelor programme. Ethical approval was ob-
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
tained from all participating universities. Data were collected at various times over the course of 2013. 2.2. Procedures In university simulation laboratories, each student individually participated in three scenarios with a standardised patient (actor) in bed and portraying a rapidly deteriorating patient. On arrival, participants were orientated to the simulation session and the study procedures. Eye tracking glasses were fitted and calibrated to suit each individual, who were then instructed how to look through the glasses to enable optimal video capture. The eye tracking glasses were calibrated before each scenario. The three 8-minute scenarios depicted common medical conditions: acute myocardial infarction, hypovolemic shock and chronic obstructive pulmonary disease. The simulation scenarios used were previously trailed with undergraduate nursing students (Bogossian et al. 2013) Participants’ ability to assess and respond to the deteriorating patient was rated by a clinician using an observational structured clinical examination (OSCE) developed in the above named programme. Participant’s eye movements and PoV were recorded via Tobii glasses with the whole performance separately video-recorded by another fixed video-camera. After each scenario, the participant was taken to an interview room and debriefed by a trained clinician using the PoV replay with eye-tracking overlay. Over approximately 20 minutes, the participant and clinician engaged in an analysis and learning debrief using the video record and replay options. At programme completion feedback tools were completed. Details of the study protocol and the outcome measures of clinical performance and situation awareness have been reported elsewhere (Cooper et al., 2013; O’Meara et al. 2015) although Area of Interest is examined for the first time in this current report.
45
looked ‘around’ the glasses (that is, past the frame), often when they were up close to a patient and looking down at an object. In this situation, and based on the Tobii guidelines, four eye-tracking recordings (of 39) were excluded where eye-tracking was recorded for less than 60% of the 8 minutes. Survey data were analysed using IBM-SPSS version 19 (IBM Corporation, 2011) using summary and inferential statistics. Parametric tests were used to analyse performance scores. Categorical data were compared with AoI using correlation coefficients and nonparametric tests. Statistical significance was set at P < 0.05. 3. Results The dataset represented all 20 nursing students and 15 of 19 paramedicine students. The mean total gaze on designated targets in the environment for the three 8 minute scenarios was 40–77%. 3.1. Gaze on areas of interest (AoI) Graphs 1–3 display gaze on three visual targets as a proportion of the total gaze by discipline for scenarios 1, 2 and 3 respectively. Over all 3 scenarios, the participants’ gaze focused mainly on the patient’s head (34%), followed by the patient’s trunk (24%) and the clinical assistant (5%). The two additional situation awareness object targets were omitted from the analysis because gaze was too limited in duration to be reliably reported (as previously discussed).
2.3. Eye-tracking measures The eye-tracking video camera overlaid the participants’ pupil fixation on a simultaneous point-of-view recording. Coloured dots on the video depicted the participant’s pupil fixation during each scenario, with the size of the coloured dot approximating the pupil fixation time (Figs 2 and 3). A number of metrics can be used to analyse eye-tracking. ‘Gaze’ (the sum of all recorded fixations) was the measure used to compare attention that was distributed amongst targets in the environment (e.g. the patient, a defibrillator or another staff member) (Poole and Ball, 2004). Gaze recorded in milliseconds was collated by an external provider who extracted and logged five designated target areas. These targets were determined by clinical experts on the research team to be:
• • • •
Graph 1. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 1.
the patient’s head the patient’s trunk (defined as the patient’s upper body; chest and abdomen, excluding extremities the scenario assistant (doctor/senior nurse or second paramedic) two objects in the room (e.g. a picture or flowers) which were used as measures of situation awareness of the wider (or global) environment.
2.4. Analysis Following the Tobii eye-tracking protocol, minimum gaze on an area of interest (AoI) was set at ≥150 ms (Tobii Technology, 2013). Below this, gaze data are too minimal for the human brain to record, and too short to be extracted and logged by researchers – and were thus excluded. In addition gaze was not captured where participants
Graph 2. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 2.
46
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
Graph 3. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 3.
Mann–Whitney U tests identified a significant difference in gaze between disciplines in eight of the nine AoI (see graphs above). Paramedicine students had more gaze fixed on the patient’s head and upper body as compared to nursing students. Conversely, nursing students’ gaze on the scenario assistant was almost twice that of paramedicine students for the first two of three scenarios (see graphs above). 3.2. Performance There was a significant improvement in the observed clinical performance ratings, between the first (Cardiac) scenario (M = 48.72, SD = 10.73) and the last (Respiratory) scenario (M = 65.64, SD = 9.61) (t(38) = −8.77; P = 0.000, 95% CI [−20.83, −13.01]) with a moderate effect size (Cohens’ d = 1.66). Spearman’s Rank Order Correlation tests were performed to determine whether performance ratings were associated with gaze on AoIs identifying that a focus on the ‘Patient’s body’ correlated to the ‘OSCE’ score in scenarios 1 (p(30) = 0.401, P = 0.028) and 3 (p(32) = 0.378, P = 0.033). Feedback comments from both professional groups indicated an increase in awareness of their situation and benefits from mapping gaze path in reflective debriefing. There were no negative comments made. 4. Discussion As far as is known, this is the first time mobile eye tracking has been used in dynamic high-fidelity scenarios in nursing. In line with the study objectives, eye-tracking was seen by participants as assisting their feedback, and there were strong indications that the ‘eye-tracking’ component improved learning. Importantly paramedicine students and nursing students were significantly different in the focus of their attention on three patient-centred areas of interest, which may reflect the degree of emergency training and how students in each discipline are taught to assess and manage a patient. For example paramedic students spent considerably more time focusing on the patient’s head and body whilst nursing students spent more time gazing at their assistant and appealing for guidance. We found significant barriers, however, to the application of eyetracking glasses in a naturalistic setting. Participants working close to a patient or doing ancillary tasks (e.g. reading an ECG or making notes) may drop their gaze below the glasses and outside the capture area. On average, 4.4 minutes of the 8-minute scenarios were captured despite participants being coached on how to optimise gaze capture.
Hermens et al. (2013) also noted that eye-tracking equipment can miss eye movement when there are large field-of-view areas. Alternatively, eye-tracking in two-dimensional fields within immediate proximity (such as when viewing a computer screen or looking at a supermarket shelf display) may be more reliable. In a small 2D area of a computer screen, a below-screen scanner can successfully capture views (Horsley et al., 2014). However, decreasing dynamism and fidelity in a scenario, in order to increase eye tracking capture, may not simulate the true clinical situation and performance. We reported total gaze duration as the main measurement for visual targets. Hermens et al. (2013) report that gaze fixation duration are appropriate in scenario type situations as it reduces the possible variation between participants. Other studies have used alternative eye-tracking measurements such as transitions, eye movement scans and patterns, the time to first fixation and also pupil diameter (Hermens et al., 2013; Holmqvist et al., 2011; Poole and Ball, 2004). Each of these approaches requires matching of instrumentation with the study objectives to optimise processes. In the current study there were benefits to study participants with significant improvement in performance assessments between the first and third scenario, and positive reports for the inclusion of eye tracking records in the debriefing process. Such outcomes are supported by Wilson et al. (2011) who identified that surgeons who received gaze training made the greatest progress in training. However AoI analyses do need to take into account the situation as a whole, for example, identifying a focus on the patient’s trunk and through reflective video review identifying why this occurred. The convenience nature of the study sample was a limitation as it is likely that students who were most interested in simulation and emergency care were recruited. As a pilot study with limited resources, the study was also underpowered to detect differences between disciplines but does demonstrate proof of concept. Future studies should expand the evidence for best practice by examining visual scanning patterns amongst disciplines and between novices and experts, to enable novice and expert development.
5. Conclusion This study demonstrates the potential for eye-tracking to objectively capture a unique perspective on clinical practice. Paramedicine and nursing students had different approaches to simulated clinical situations which were identified by variations in gaze patterns. Whilst the study outcomes were positive they must be balanced against the resources’ intense nature of eye-tracking in clinical simulation. As a stand-alone training programme, the time required, a high staff to student ratio and call on university resources preclude this model as training for individual students. To overcome such resource limitations, eye-tracking or point-of-view cameras could feasibly be included in scenarios that are based on teams of students. In future studies an understanding of experts’ performance would also be of benefit. Further study of this application is warranted.
Acknowledgements Financial support for the study was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government Department of Education, Employment and Workplace Relations.
Conflicts of interest None declared.
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
References Bogossian, F., Cooper, S., Cant, R., Beauchamp, A., Porter, J., Kain, V., et al., 2014. Undergraduate nursing students’ performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: an Australian multi-centre study. Nurse Education Today. 34, 691–696. Cooper, S., Black, B., Bogossian, F., Browning, M., Munro, G., O’Meara, P., et al., 2013. Can Eye-tracking Technology Improve Situational Awareness and Student Feedback during Simulation? Office for Learning and Teaching, Innovation, Science, Research and Tertiary Education, Sydney. Duchowski, A.T., 2007. Eye Tracking Methodology: Theory and Practice. Springer-Verlag. Henneman, P.L., Fisher, D.L., Henneman, E.A., Pham, T.A., Campbell, M.M., Nathanson, B.H., 2010. Patient identification errors are common in a simulated setting. Annals of Emergency Medicine. 55 (6), 503–509. Hermens, F., Flin, R., Ahmed, I., 2013. Eye movements in surgery: a literature review. Journal of Eye Movement Research. 6 (4), 1–11. Holmqvist, K., NystrÃm, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J., 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press. Horsley, M., Eliot, M., Knight, B.A., Reilly, R., Broadbent, M., Horsley, M., et al., 2014. Comparing novice and expert nurses in analysing electrocardiographs (ECGs) containing critical diagnostic information: an eye tracking study of the
47
development of complex nursing visual cognitive skills, in: Current Trends in Eye Tracking Research. Springer International, Switzerland, pp. 297–315. IBM Corporation, 2011. IBM SPSS Statistics for Windows. Version 19. IBM Corp, Armonk, New York. Levett-Jones, T., Hoffman, K., Dempsey, J., Jeong, S.Y.-S., Noble, D., Norton, C.A., et al., 2010. The ‘five rights’ of clinical reasoning: an educational model to enhance nursing students’ ability to identify and manage clinically ‘at risk’ patients. Nurse Education Today. 30 (6), 515–520. O’Meara, P., Munro, G., Cooper, S., Williams, B., Bogossian, F., Ross, L., et al., 2015. Developing situation awareness amongst nursing and paramedicine students utilizing eye tracking technology and video debriefing techniques: a proof of concept paper. International Emergency Nursing. 23 (2), 94–99. Poole, A., Ball, L., 2004. Eye Tracking in Human-computer Interaction and Usability Research: Current Status and Future Prospects: Psychology Department. Lancaster University, UK. Tobii Technology, (2013). Mobile eyetracking – Tobii eyetracking glasses. . Wilson, M.R., Vine, S.J., Bright, E., Masters, R.S., Defriend, D., McGrath, J.S., 2011. Gaze training enhances laproscopic technical skill acquisition and multi-tasking performance: a randomized controlled study. Surgical Endoscopy. 25 (120), 3731–3739. Ze, H. (2011). Methods to study nurses’ visual scanning patterns during the medication administration process. (Thesis) Master of Science: University of Massachusetts Amherst, USA.
Contents lists available at ScienceDirect
International Emergency Nursing j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / a a e n
The use and limits of eye-tracking in high-fidelity clinical scenarios: A pilot study Mark Browning BN (hon), RN, BHlthSc (Lecturer, Course Coordinator) a,*, Simon Cooper PhD, MEd, DipSocRes, BA, RGN (Professor, Emergency Care and Research Development) a, Robyn Cant PhD, MHlthSc (Research Fellow) a, Louise Sparkes MN, GCertEmerg Nsg, RN (Lecturer) a, Fiona Bogossian PhD MPH BAppSc DipAppSc(Nurs Ed) RN Midwife (Associate Professor, Director of Research) b, Brett Williams BAdultVocEd, GradCertIntensiveCareParamed, GradDipEmergHlth, MHlthSc, PhD, FPA (Associate Professor, Head of Paramedicine Department) c, Peter O’Meara BHA, MPP, PhD, FPA (Professor, Head of Paramedicine, Public & Community Health) d, Linda Ross BTeach, GradDipPhysEd, DipHlth(Amb), BParaStud, MHlthProfessEd (Lecturer) c, Graham Munro BHS, MHSM (Lecturer) d, Barbara Black RN,Grad Cert Peri-op Nursing, MHlthProfessEd (Lecturer) b a
School of Nursing and Midwifery, Monash University, Berwick, Vic. 3806, Australia School of Nursing & Midwifery, The University of Queensland, Herston Campus, QLD, Australia c Department of Community Emergency Health & Paramedic Practice, Monash University, Vic., Australia d La Trobe Rural Health School, La Trobe University, Bendigo, Vic., Australia b
A R T I C L E
I N F O
Article history: Received 22 January 2015 Received in revised form 30 July 2015 Accepted 4 August 2015 Keywords: Debriefing Eye-tracking Education Clinical Simulation Point of view (PoV) Gaze
A B S T R A C T
Aim: To explore the potential of mobile eye-tracking to identify healthcare students’ area of visual interest and its relationship to performance ratings. Background: Eye-tracking identifies an individual’s visual attention focus, and has been used as a training technique in medicine and in nursing. In this study participants wore a point of view (PoV) camera within a spectacle frame during simulation education experiences. Methods: Thirty-nine final year nursing and paramedicine students individually participated in three 8 minute clinical simulations with debriefing using videoed eye-tracking recordings. Coloured dots on the video depicted the participant’s pupil fixation on five targeted areas. Data extracted from the video camera were collated to report time spent on each target (their ‘gaze’). Results: The mean total gaze of expert designated targets in the environment for three 8 minute scenarios was 40–77%. Of 35 participants’ focus on three main areas of interest, their priority was the patient’s head (34%), the patient’s trunk (24%) and their clinical assistant (5%), with significant differences between nursing and paramedic disciplines (P < 0.05). Objectively rated clinical performance improved significantly by the third scenario (P ≤ 0.001). Participants were positive regarding use of eye tracking during debriefing. Conclusions: Eye tracking has the potential to enhance debriefing and educational outcomes, although there are limitations to gaze capture in high fidelity environments and resource cost is high. Further study is warranted to enable better understanding of how expert clinicians achieve high levels of performance. © 2016 Elsevier Ltd. All rights reserved.
1. Introduction
* Corresponding author. School of Nursing and Midwifery, Monash University, PO Box 1071, Narre Warren, Vic. 3805, Australia. Tel.: +61 3 9904 7128; fax: +61 3 9904 4655. E-mail address: [email protected] (M. Browning). http://dx.doi.org/10.1016/j.ienj.2015.08.002 1755-599X/© 2016 Elsevier Ltd. All rights reserved.
Eye movement tracking or eye-tracking is a technology that has emerged from the field of marketing, whilst more recently being trialled as a technique for the training of healthcare professionals. Eye-tracking can identify the focus of an individual’s attention and
44
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
Box 1. Definitions used Eye-tracking: measurement of eye movements based on where the eye’s pupil is focused, using video technology AoI (area of interest): visual attention tracked by camera Gaze: the total time an individual’s attention is given to visual target/s in the environment (i.e. fixation time) Gaze capture: timed attention to a visual target of more than 150 milliseconds – anything less is not noted by the human brain. PoV glasses: point of view glasses worn by participant contain a camera that is calibrated to each individual’s right eye retina.
measure eye movements based on where their pupil is focused (Poole and Ball, 2004). These measurements have been used in conjunction with other training assessments to examine how well clinicians perform procedures (Hermens et al., 2013; Poole and Ball, 2004). Some examples include investigation of nurses’ scanning patterns during medication administration (Ze, 2011); the use of eyetracking to explore patient identification errors in nursing (Henneman et al., 2010); and examination of electrocardiograph interpretation by novice expert and nurses (Horsley et al., 2014). We hypothesised that recording performance whilst wearing a point of view (PoV) camera within a spectacle frame during simulation education experiences would enhance debriefing for health professional students. Eye-tracking recordings may further enable detailed analysis of the student’s area of attention, inform their reflective review and, ultimately, assist decision-making and enhance performance (Box 1). Fig. 3. The eye tracking video output. Note the pupil tracking (red) overlaid with the PoV capture.
1.1. Eye-tracking technology
Fig. 1. The Tobii eye tracking headset worn by participants.
As humans, we make between three and five eye movements per second to examine and make sense of visual information (Holmqvist et al., 2011). Visual attention is the first step in making sense of complex information (Horsley et al., 2014). In clinical nursing, individuals need to collect and prioritise patient cues and make decisions about patient management using visual cues (Levett-Jones et al., 2010). Video technology incorporated into a pair of hightechnology glasses and computerised software developed by Tobii Technology (Fig. 1) enables timed data replay. The Glasses capture PoV video and the pupil movement is mapped over this (Figs 2 and 3). With the assumption that gaze fixation is normally the focus of thought (Duchowski, 2007), it follows that eye-tracking could substantiate the timing and sequence of clinical decision-making during a simulation experience. 2. Methods A pilot observational study was designed to explore the feasibility of using eye-tracking during high fidelity clinical simulation scenarios and the impact on students’ learning experience. The objective was to identify undergraduate healthcare students’ area of visual interest and its relationship to performance ratings. 2.1. Sample
Fig. 2. The eye tracking video output. Note the pupil tracking (red) overlaid with the PoV capture.
A convenience sample of 39 final year undergraduate degree students was recruited from three Australian universities in 2013. The universities were chosen in order to recruit a sample of students studying in two disciplines and from two Australian states. Nineteen participants were enrolled in a Paramedic Bachelor programme and 20 in a Nursing Bachelor programme. Ethical approval was ob-
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
tained from all participating universities. Data were collected at various times over the course of 2013. 2.2. Procedures In university simulation laboratories, each student individually participated in three scenarios with a standardised patient (actor) in bed and portraying a rapidly deteriorating patient. On arrival, participants were orientated to the simulation session and the study procedures. Eye tracking glasses were fitted and calibrated to suit each individual, who were then instructed how to look through the glasses to enable optimal video capture. The eye tracking glasses were calibrated before each scenario. The three 8-minute scenarios depicted common medical conditions: acute myocardial infarction, hypovolemic shock and chronic obstructive pulmonary disease. The simulation scenarios used were previously trailed with undergraduate nursing students (Bogossian et al. 2013) Participants’ ability to assess and respond to the deteriorating patient was rated by a clinician using an observational structured clinical examination (OSCE) developed in the above named programme. Participant’s eye movements and PoV were recorded via Tobii glasses with the whole performance separately video-recorded by another fixed video-camera. After each scenario, the participant was taken to an interview room and debriefed by a trained clinician using the PoV replay with eye-tracking overlay. Over approximately 20 minutes, the participant and clinician engaged in an analysis and learning debrief using the video record and replay options. At programme completion feedback tools were completed. Details of the study protocol and the outcome measures of clinical performance and situation awareness have been reported elsewhere (Cooper et al., 2013; O’Meara et al. 2015) although Area of Interest is examined for the first time in this current report.
45
looked ‘around’ the glasses (that is, past the frame), often when they were up close to a patient and looking down at an object. In this situation, and based on the Tobii guidelines, four eye-tracking recordings (of 39) were excluded where eye-tracking was recorded for less than 60% of the 8 minutes. Survey data were analysed using IBM-SPSS version 19 (IBM Corporation, 2011) using summary and inferential statistics. Parametric tests were used to analyse performance scores. Categorical data were compared with AoI using correlation coefficients and nonparametric tests. Statistical significance was set at P < 0.05. 3. Results The dataset represented all 20 nursing students and 15 of 19 paramedicine students. The mean total gaze on designated targets in the environment for the three 8 minute scenarios was 40–77%. 3.1. Gaze on areas of interest (AoI) Graphs 1–3 display gaze on three visual targets as a proportion of the total gaze by discipline for scenarios 1, 2 and 3 respectively. Over all 3 scenarios, the participants’ gaze focused mainly on the patient’s head (34%), followed by the patient’s trunk (24%) and the clinical assistant (5%). The two additional situation awareness object targets were omitted from the analysis because gaze was too limited in duration to be reliably reported (as previously discussed).
2.3. Eye-tracking measures The eye-tracking video camera overlaid the participants’ pupil fixation on a simultaneous point-of-view recording. Coloured dots on the video depicted the participant’s pupil fixation during each scenario, with the size of the coloured dot approximating the pupil fixation time (Figs 2 and 3). A number of metrics can be used to analyse eye-tracking. ‘Gaze’ (the sum of all recorded fixations) was the measure used to compare attention that was distributed amongst targets in the environment (e.g. the patient, a defibrillator or another staff member) (Poole and Ball, 2004). Gaze recorded in milliseconds was collated by an external provider who extracted and logged five designated target areas. These targets were determined by clinical experts on the research team to be:
• • • •
Graph 1. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 1.
the patient’s head the patient’s trunk (defined as the patient’s upper body; chest and abdomen, excluding extremities the scenario assistant (doctor/senior nurse or second paramedic) two objects in the room (e.g. a picture or flowers) which were used as measures of situation awareness of the wider (or global) environment.
2.4. Analysis Following the Tobii eye-tracking protocol, minimum gaze on an area of interest (AoI) was set at ≥150 ms (Tobii Technology, 2013). Below this, gaze data are too minimal for the human brain to record, and too short to be extracted and logged by researchers – and were thus excluded. In addition gaze was not captured where participants
Graph 2. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 2.
46
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
Graph 3. Comparison of gaze on AoI (as a proportion of total AOI gaze) by discipline in scenario 3.
Mann–Whitney U tests identified a significant difference in gaze between disciplines in eight of the nine AoI (see graphs above). Paramedicine students had more gaze fixed on the patient’s head and upper body as compared to nursing students. Conversely, nursing students’ gaze on the scenario assistant was almost twice that of paramedicine students for the first two of three scenarios (see graphs above). 3.2. Performance There was a significant improvement in the observed clinical performance ratings, between the first (Cardiac) scenario (M = 48.72, SD = 10.73) and the last (Respiratory) scenario (M = 65.64, SD = 9.61) (t(38) = −8.77; P = 0.000, 95% CI [−20.83, −13.01]) with a moderate effect size (Cohens’ d = 1.66). Spearman’s Rank Order Correlation tests were performed to determine whether performance ratings were associated with gaze on AoIs identifying that a focus on the ‘Patient’s body’ correlated to the ‘OSCE’ score in scenarios 1 (p(30) = 0.401, P = 0.028) and 3 (p(32) = 0.378, P = 0.033). Feedback comments from both professional groups indicated an increase in awareness of their situation and benefits from mapping gaze path in reflective debriefing. There were no negative comments made. 4. Discussion As far as is known, this is the first time mobile eye tracking has been used in dynamic high-fidelity scenarios in nursing. In line with the study objectives, eye-tracking was seen by participants as assisting their feedback, and there were strong indications that the ‘eye-tracking’ component improved learning. Importantly paramedicine students and nursing students were significantly different in the focus of their attention on three patient-centred areas of interest, which may reflect the degree of emergency training and how students in each discipline are taught to assess and manage a patient. For example paramedic students spent considerably more time focusing on the patient’s head and body whilst nursing students spent more time gazing at their assistant and appealing for guidance. We found significant barriers, however, to the application of eyetracking glasses in a naturalistic setting. Participants working close to a patient or doing ancillary tasks (e.g. reading an ECG or making notes) may drop their gaze below the glasses and outside the capture area. On average, 4.4 minutes of the 8-minute scenarios were captured despite participants being coached on how to optimise gaze capture.
Hermens et al. (2013) also noted that eye-tracking equipment can miss eye movement when there are large field-of-view areas. Alternatively, eye-tracking in two-dimensional fields within immediate proximity (such as when viewing a computer screen or looking at a supermarket shelf display) may be more reliable. In a small 2D area of a computer screen, a below-screen scanner can successfully capture views (Horsley et al., 2014). However, decreasing dynamism and fidelity in a scenario, in order to increase eye tracking capture, may not simulate the true clinical situation and performance. We reported total gaze duration as the main measurement for visual targets. Hermens et al. (2013) report that gaze fixation duration are appropriate in scenario type situations as it reduces the possible variation between participants. Other studies have used alternative eye-tracking measurements such as transitions, eye movement scans and patterns, the time to first fixation and also pupil diameter (Hermens et al., 2013; Holmqvist et al., 2011; Poole and Ball, 2004). Each of these approaches requires matching of instrumentation with the study objectives to optimise processes. In the current study there were benefits to study participants with significant improvement in performance assessments between the first and third scenario, and positive reports for the inclusion of eye tracking records in the debriefing process. Such outcomes are supported by Wilson et al. (2011) who identified that surgeons who received gaze training made the greatest progress in training. However AoI analyses do need to take into account the situation as a whole, for example, identifying a focus on the patient’s trunk and through reflective video review identifying why this occurred. The convenience nature of the study sample was a limitation as it is likely that students who were most interested in simulation and emergency care were recruited. As a pilot study with limited resources, the study was also underpowered to detect differences between disciplines but does demonstrate proof of concept. Future studies should expand the evidence for best practice by examining visual scanning patterns amongst disciplines and between novices and experts, to enable novice and expert development.
5. Conclusion This study demonstrates the potential for eye-tracking to objectively capture a unique perspective on clinical practice. Paramedicine and nursing students had different approaches to simulated clinical situations which were identified by variations in gaze patterns. Whilst the study outcomes were positive they must be balanced against the resources’ intense nature of eye-tracking in clinical simulation. As a stand-alone training programme, the time required, a high staff to student ratio and call on university resources preclude this model as training for individual students. To overcome such resource limitations, eye-tracking or point-of-view cameras could feasibly be included in scenarios that are based on teams of students. In future studies an understanding of experts’ performance would also be of benefit. Further study of this application is warranted.
Acknowledgements Financial support for the study was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government Department of Education, Employment and Workplace Relations.
Conflicts of interest None declared.
M. Browning et al./International Emergency Nursing 25 (2016) 43–47
References Bogossian, F., Cooper, S., Cant, R., Beauchamp, A., Porter, J., Kain, V., et al., 2014. Undergraduate nursing students’ performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: an Australian multi-centre study. Nurse Education Today. 34, 691–696. Cooper, S., Black, B., Bogossian, F., Browning, M., Munro, G., O’Meara, P., et al., 2013. Can Eye-tracking Technology Improve Situational Awareness and Student Feedback during Simulation? Office for Learning and Teaching, Innovation, Science, Research and Tertiary Education, Sydney. Duchowski, A.T., 2007. Eye Tracking Methodology: Theory and Practice. Springer-Verlag. Henneman, P.L., Fisher, D.L., Henneman, E.A., Pham, T.A., Campbell, M.M., Nathanson, B.H., 2010. Patient identification errors are common in a simulated setting. Annals of Emergency Medicine. 55 (6), 503–509. Hermens, F., Flin, R., Ahmed, I., 2013. Eye movements in surgery: a literature review. Journal of Eye Movement Research. 6 (4), 1–11. Holmqvist, K., NystrÃm, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J., 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press. Horsley, M., Eliot, M., Knight, B.A., Reilly, R., Broadbent, M., Horsley, M., et al., 2014. Comparing novice and expert nurses in analysing electrocardiographs (ECGs) containing critical diagnostic information: an eye tracking study of the
47
development of complex nursing visual cognitive skills, in: Current Trends in Eye Tracking Research. Springer International, Switzerland, pp. 297–315. IBM Corporation, 2011. IBM SPSS Statistics for Windows. Version 19. IBM Corp, Armonk, New York. Levett-Jones, T., Hoffman, K., Dempsey, J., Jeong, S.Y.-S., Noble, D., Norton, C.A., et al., 2010. The ‘five rights’ of clinical reasoning: an educational model to enhance nursing students’ ability to identify and manage clinically ‘at risk’ patients. Nurse Education Today. 30 (6), 515–520. O’Meara, P., Munro, G., Cooper, S., Williams, B., Bogossian, F., Ross, L., et al., 2015. Developing situation awareness amongst nursing and paramedicine students utilizing eye tracking technology and video debriefing techniques: a proof of concept paper. International Emergency Nursing. 23 (2), 94–99. Poole, A., Ball, L., 2004. Eye Tracking in Human-computer Interaction and Usability Research: Current Status and Future Prospects: Psychology Department. Lancaster University, UK. Tobii Technology, (2013). Mobile eyetracking – Tobii eyetracking glasses. . Wilson, M.R., Vine, S.J., Bright, E., Masters, R.S., Defriend, D., McGrath, J.S., 2011. Gaze training enhances laproscopic technical skill acquisition and multi-tasking performance: a randomized controlled study. Surgical Endoscopy. 25 (120), 3731–3739. Ze, H. (2011). Methods to study nurses’ visual scanning patterns during the medication administration process. (Thesis) Master of Science: University of Massachusetts Amherst, USA.
