Nurse Education Today 34 (2014) 691–696
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
Undergraduate nursing students' performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: An Australian multi-centre study☆ Fiona Bogossian a,⁎, Simon Cooper b, Robyn Cant b, Alison Beauchamp b,e, Joanne Porter c, Victoria Kain a, Tracey Bucknall d, Nicole M. Phillips d, The FIRST2ACT™ Research Team a
The School of Nursing & Midwifery, The University of Queensland, Herston Campus, QLD, Australia The School of Nursing & Midwifery, Monash University, Berwick Campus, VIC, Australia c The School of Nursing & Midwifery, Monash University, Gippsland Campus, VIC, Australia d School of Nursing and Midwifery Deakin University, Burwood Campus, VIC, Australia e Population Health Strategic Research Centre, Deakin University, Burwood Campus, VIC, Australia b
a r t i c l e
i n f o
Article history: Accepted 18 September 2013
Keywords: Education Nursing Patient deterioration Simulation Clinical performance Clinical decision making Situation awareness Team work
s u m m a r y Background: Early recognition and situation awareness of sudden patient deterioration, a timely appropriate clinical response, and teamwork are critical to patient outcomes. High fidelity simulated environments provide the opportunity for undergraduate nursing students to develop and refine recognition and response skills. Objectives: This paper reports the quantitative findings of the first phase of a larger program of ongoing research: Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends (FIRST2ACTTM). It specifically aims to identify the characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients. Design: Mixed-method multi-centre study. Setting: High fidelity simulated acute clinical environment in three Australian universities. Participants: A convenience sample of 97 final year nursing students enrolled in an undergraduate Bachelor of Nursing or combined Bachelor of Nursing degree were included in the study. Method: In groups of three, participants proceeded through three phases: (i) pre-briefing and completion of a multi-choice question test, (ii) three video-recorded simulated clinical scenarios where actors substituted real patients with deteriorating conditions, and (iii) post-scenario debriefing. Clinical performance, teamwork and situation awareness were evaluated, using a validated standard checklist (OSCE), Team Emergency Assessment Measure (TEAM) score sheet and Situation Awareness Global Assessment Technique (SAGAT). A Modified Angoff technique was used to establish cut points for clinical performance. Results: Student teams engaged in 97 simulation experiences across the three scenarios and achieved a level of clinical performance consistent with the experts' identified pass level point in only 9 (1%) of the simulation experiences. Knowledge was significantly associated with overall teamwork (p = .034), overall situation awareness (p = .05) and clinical performance in two of the three scenarios (p = .032 cardiac and p = .006 shock). Situation awareness scores of scenario team leaders were low overall, with an average total score of 41%. Conclusions: Final year undergraduate nursing students may have difficulty recognising and responding appropriately to patient deterioration. Improving pre-requisite knowledge, rehearsal of first response and team management strategies need to be a key component of undergraduate nursing students' education and ought to specifically address clinical performance, teamwork and situation awareness. © 2013 Elsevier Ltd. All rights reserved.
Introduction The prevalence of deteriorating hospital patients measured as those who fulfill Medical Emergency Team (MET) criteria has been ☆ Ethical approval: Monash University CF11/3414-201101825, Deakin University 2012-030 and The University of Queensland 201200011. ⁎ Corresponding author. E-mail address: [email protected] (F. Bogossian). 0260-6917/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.nedt.2013.09.015
demonstrated to range from 3.3% in a recent Australian study (Bucknall et al., 2013) up to 18% in a Danish study (Fuhrmann et al., 2008). Studies consistently demonstrate significant findings that patients who fulfill MET criteria for rapid response have in excess of double the risk of mortality both in hospital and at 30 days (Bell et al., 2006; Bucknall et al., 2013; Fuhrmann et al., 2008). Moreover hospital patients who are deteriorating physiologically are frequently mismanaged, leading to international concerns over patient safety (Agency for Healthcare Research and Quality, 2009; Cooper et al., 2011a,b; DeVita et al., 2010).
692
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
Nurses as front line care providers are in a position to detect and respond to changes in vital signs and other cues reflecting patient deterioration and in doing so directly impact patient mortality. Following a series of pilot studies examining the performance of nursing students and qualified staff in simulated settings we have developed a program of learning called FIRST2ACT™ (Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends) (Buykx et al., 2011). In recent trials the program demonstrated a significant impact on learning, and for hospital nurses, a significant impact on clinical practice (Buykx et al., 2011). Following the intervention an audit of patients' medical records showed improvements in the charting of vital signs, in pain score recording and in the correct delivery of oxygen therapy. The current study builds on the previous research (Buykx et al., 2011) and forms the first phase of a larger program of research funded by the Australian Learning and Teaching Council, the protocol of which has been published elsewhere (Cooper et al., 2012). This phase focuses on understanding clinical performance, teamwork, situation awareness and decision-making in undergraduate nursing students and this paper reports quantitative findings from the first phase. Aims The aims of this study were firstly to identify characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients. Secondly, to explore interactions between the outcome measures in order to identify factors potentially amenable to modification for the improvement of patient safety. Method A mixed method multi-centre study was designed to focus on undergraduate nursing students and their experiences in caring for deteriorating patients. Undergraduate nursing students were recruited from three Australian universities, two from Victoria and one from Queensland. In order to be able to recognise and respond to sudden patient deterioration, students required some level of exposure to relevant theory and clinical practice, through a standard educational program on emergency care, thus only final year students were considered eligible. Although instruments and design had been previously tested in a group of registered nurses (Cooper et al., 2011a,b) the current research is predominantly descriptive and thus the determination of an a priori sample size is unnecessary. Based on previous studies, in which response rates varied from 46% for nursing students to 82% for registered nurses, we anticipated a response rate in excess of 50%. Following ethical approval from each of the three universities final year nursing students were invited to participate in the study. Students were provided with standardised information about the project via email in the first instance and then via a 5-minute power point presentation conducted by a faculty member who was not a part of the research team. Students responded in the form of an expression of interest, if they wished to participate. Study inclusion was on a first come, first served, basis at each site and students reconfirmed their participation when allocated a scheduled session time and date. These strategies were employed to maximize participation within the physical and human resource limitations on the number of students who could participate at each university site. The intervention and data collection were conducted over 2 days at each site over a 3-week period. In order to ensure the highest possible level of reliability and validity over the multiple sites and days, the intervention was delivered by a core roving team including three members of the research team (SC, JP & AB) and one trained actor who were supported by other members of the research team based on the relevant university site. At each site a clinical simulation space that replicated a ward-like environment and standardised equipment was assembled to provide a
consistent student experience across all sites. Students proceeded in groups of three through the research stages (i) pre-intervention briefing, (ii) the simulation intervention and (iii) post-intervention debriefing (see Table 1). Pre-intervention Briefing The pre-intervention briefing included written participant information and a standard verbal explanation of the project prior to written consent. This was followed by the completion of a questionnaire in order to collect demographic and clinical experience data and an 11 item multiple-choice questionnaire (MCQ). Each participant was assigned a participant number during the pre-intervention stage and all data collected (written and video) from the individual participant was assigned this number. The assignment of the team leader role order for the intervention within each trio of students was randomly determined. The Simulation Intervention Each trio of students then moved to the clinical simulation space waiting area. For each scenario the assigned team leader was given a brief handover and was instructed to enter the room. Video filming began and the team leader responded to the cues of the environment and the patient actor and was able to call for assistance from other team members at any time. The total duration of each simulation scenario was 8 min, with subtle deterioration cues evident in the first 4 min prior to more obvious and significant signs of deterioration in the final 4 min (Cooper et al., 2011a,b). Two assessors assessed the clinical and non-technical skills performance of the participant trio, and then agreed a final score. Blinding was not possible. At the conclusion of each scenario the team leader was taken aside and asked a series of rapid-fire situation awareness questions by a research team member to ascertain their understanding and awareness of the situation as well as their prediction of future events in relation to the simulation. Post Intervention Debriefing Following completion of the clinical simulation component participants underwent debriefing which consisted of video review, performance feedback and written participant evaluation of the experience. Participant trios under the guidance of member of the research team undertook video review of their performance who encouraged participants to self-evaluate and give a reflective account of their performance and decision making in a process called ‘photo elicitation’ (DiGiuseppe et al., 2002; Harper, 1994). Following this, participants were asked to complete a written evaluation of the experience.
Table 1 Phases and related activities in First2Act simulation. Pre-intervention
Intervention Scenario
Roles
Measures
Consent Briefing
Cardiac
Leader A All team
SAGAT OSCE TEAM
Participant ID assigned Participant information questionnaire demographic and clinical experience Multiple choice questionnaire (MCQ) 11 items to test knowledge
Shock
Leader B All team
SAGAT OSCE TEAM
Respiratory
Leader C
SAGAT
All team
OSCE TEAM
Postintervention Debriefing Photo elicitation and video review Self evaluation Instructor feedback
Written evaluation of simulation experience
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
The total duration of the experience for participants was 2 h, and participants were able to use this experience as a component of their clinical learning activities and as evidence of research engagement. Participants were issued with a certificate of participation for inclusion in their professional portfolios.
Instruments and Measures Clinical knowledge was measured using an 11 item MCQ adapted from previously validated instruments (Cooper, 2010; Endacott et al., 2009) and verified by a panel of clinical experts, which was designed to test knowledge of the deteriorating patient. Clinical performance was assessed using a standardized Objective Structured Clinical Examination (OSCE) score sheet (the number of items varied n = 24, 28 and 26 according to the scenario). In order to set the ‘pass mark’ for the OSCE score we followed a Modified Angoff technique (Ricker, 2006) which is subsequently described. A group of clinical nursing experts, with 7–27 years clinical experience (mean 15 years), who also had teaching experience, reviewed the OSCE scoring for each scenario. The group was asked to recall students whom they had taught to manage patients experiencing similar deterioration and to consider those that ‘bordered on mastery’, or put another way met the minimum safe level of competency. These students were then discussed focusing on what they were able to achieve and what they were not able to achieve in order to develop ‘a common notion of a borderline candidate’. Individual experts identified the score items on each OSCE that represented a borderline pass i.e. which nominal (yes/no) criteria must (and are likely to) be achieved by a borderline candidate. The mean of their summed scores was calculated to provide a cut point indicative of a pass mark for each OSCE scenario. Non-technical skills of each student trio were assessed using the Team Emergency Assessment Measure (TEAM). The TEAM instrument includes ratings of non-technical skills (11 items on a 0–4 point scale) that include subscales for leadership, team working and task management with possible scores of 8, 28 and 8 respectively. The instrument also yields an overall global rating (0–10 point scale) and has previously been demonstrated to be valid and reliable (Cooper et al., 2010a). Situation awareness of the team leader was assessed using Situation Awareness Global Assessment Technique (SAGAT) designed to identify the leader's awareness of the patient's physiologic state as well as awareness of the wider situation (Cooper et al., 2010b). The 12-item instrument provides subscale measures of four components of situation awareness, namely physiological perception (3 items), global situation perception (3 items), comprehension (2 items) and projection (4 items). The SAGAT instrument has been previously validated (Bell et al., 2006).
Analysis Data for individual participants across all sites was pooled and entered into a database, and statistical analyses undertaken using SPSS (IBM Corporation, Released, 2011). A significance level of 0.05 was determined for all descriptive and inferential tests. No values were imputed for missing data. Scores for clinical knowledge, clinical performance, teamwork and situation awareness (MCQ, OSCE, TEAM and SAGAT instruments respectively) were summed, and analyzed as continuous variables in the first instance and descriptive summary statistics employed. Inferential statistics were employed to test associations between variables. To further explore predictors of student success in managing the deteriorating patient clinical performance (OSCE) scores were dichotomized according to the determined Angoff cut points and knowledge by MCQ scores divided into quartiles to enable subsequent bivariate analysis of categorical variables.
693
Results A total of 570 final year nursing students studying a Bachelor of Nursing or combined degrees from University A (n = 120); University B (n = 350); and University C (n = 100) were eligible for inclusion. Ninety-seven students participated in the study from University A (n = 34), University B (n = 32) and University C (n = 31) yielding participation rates of 28%, 9% and 31% by institution. The majority of participants (92%) were enrolled in a Bachelor of Nursing degree with the remainder enrolled in dual degree and graduate entry nursing programs leading to registration as a nurse. Three unintentionally recruited Bachelor of Midwifery students and one registered nurse were ineligible for inclusion and were subsequently omitted from analysis of performance; however as they had been allocated to a trio they participated in the scenarios in non-leadership roles and in debriefing. Of the 97 participants the majority (93%) were female and the median age was 21 years (mean 22.59 ± 5.47; range 18–52). Clinical exposure was assessed by the frequency and type of clinical placements students had experienced throughout their degree. A total of 95% had at least 3 clinical placements and most frequently this included general medical surgical units (48%). Approximately half (52%) of the students had previous experience in health care related employment in assistant roles. A total of fifty students (52%) reported exposure to the real life experience of a deteriorating patient. Of those, the predominant roles were as observers (n = 29) or recorder scribes (n = 10). Only 5 students (5%) reported they had previously been first responders to a deteriorating patient. Clinical knowledge assessed prior to the intervention using the 11-item MCQ revealed a range of scores from 4 to 10 and a mean of 7.25 (SD 1.48, CI 6.93–7.56). A significant difference (p = .003) was observed between mean scores knowledge for students by universities. The mean score for University B was 6.50 (SD 1.37, CI 6.01–6.99) lower than University A, 7.74 (SD 1.42, CI 7.24–8.23) and University C, 7.48 (SD1.63, CI 6.89–8.08). Performance Assessment Three measures of performance — clinical performance, teamwork and situation awareness are reported, the former two relate to group performance and the latter relates to performance of the individual scenario leader. Summary scores for each measure are in Table 2. Clinical performance for each team of three students was assessed by OSCE. Using the Modified Angoff technique indicative pass marks were established for the OSCE scores and set by scenarios at 59% cardiac, 70% shock, and 67% respiratory. Only 9 student teams passed an OSCE based on the cut points and the remainder (91%) failed to achieve the set level of clinical performance. When total score was examined by university there was a significant difference (p = .027) in mean OSCE scores in the cardiac scenario with University B (mean 11.0) exhibiting lower scores than the other institutions (University A mean 14.1 and University C mean 12.3). Nontechnical skills were assessed using the TEAM instrument. The average total score for the 11 items was 38% and the average global score was 37% (see Table 2), with strong item-to-total associations (n = 92, r = .848, p b .001). The summary results for each subscale were; leadership — mean 2.76, SD 1.53, team working — mean 11.13, SD 4.42 and task management — mean 2.71, SD 1.46. All components of the TEAM instrument demonstrated significant positive associations, namely: item-to-total scores (range r = .599–.838, p = .000); item to Item (range r = .212–.838, p = .000); and item to global score (range r = .422–.838). Global scores were strongly associated with the three subscales; leadership (r = .708, p = .000), team working (r = .960, p = .000) and task management (r = .800, p = .000). A Cronbach's alpha coefficient (α) of .912 for the TEAM scale of 11 items confirmed adequate reliability (expected N .70). Alpha for each of the three subscales
694
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
Table 2 Summary scores of clinical performance, teamwork and situation awareness by scenario. Clinical performance — OSCE
Teamwork — TEAM
Situation awareness — SAGAT
Scenario (n = groups)
Mean score (% of total score)
Range/total (sd)
95% CI
Mean score (% of total score)
Range/total (sd)
95% CI
Cardiac n = 35 Shock n = 32 Respiratory n = 30 Total
12.65 (45%) 12.66 (49%) 12.96 (54%) 12.75 (49%)
7–18/28 (2.83) 8–19/26 (3.34) 7–18/24 (2.61) 7–19/25.3 (2.92)
11.68–13.63 11.45–13.86 11.99–13.91 12.16–13.34
17.54 (40%) 3–32/44 (6.66) 15.25–19.83 14.59 (33%) 3–33/44 (6.74) 12.16–17.03 17.29 (39%) 8–31/44 (5.99) 15.55–20.29 Non-technical performance total (n = 94) 16.60 (38%) 3–33/44 (6.60) 15.30–18.00 Global rating (n = 94) 3.65 (37%) 1–7/10 (1.35) 3.37–3.93
Mean score (% of total score)
Range/total (sd)
95% CI
4.94 (41%) 5.00 (42%) 4.97 (41%) 4.96 (41%)
2–9/12 (1.68) 1–9 (2.02) 3–8 (1.16) 1–9/12 (1.65)
4.37–5.52 4.27–5.73 4.53–5.40 4.64–5.30
Note: sample may vary owing to incomplete/missing data.
was confirmed as adequate: leadership α = .818 (2 items); team working α = .881 (7 items); and task management α = .842 (2 items). Considering two of the subscales comprised only 2 items, the reliability was regarded as being high. Situation awareness scores (see Table 3) were low overall with an average score of 41% and within the subscales of physiological perception 26%, global perception 32% and comprehension 44%. However students were better at forecasting likely medical consequences with a projection average score of 59%. When examined by university a significant difference was identified for mean situation awareness scores (p = .03). University C scored 5.61 (SD 1.70, CI 4.99–6.24), University A scored 4.91 (SD 1.68, CI 4.33–.49) and University B scored 4.41 (SD1.39, CI 3.91–4.91). Experience and Knowledge Related to Performance The relationship between students' experience and knowledge (age, previous employment in nursing or health care related field, having experience of caring for the deteriorating patient and MCQ) and clinical performance, teamwork and situation awareness was examined. Significant correlations are reported in Table 4. Knowledge as measured by MCQ scores was significantly associated (p ≤ .05) with total teamwork, total situation awareness and clinical performance in cardiac and shock scenarios. Having previously cared for a patient who suddenly deteriorated was associated with overall teamwork performance. Teamwork performance was correlated with clinical performance in each of the scenarios. Although clinical performance (OSCE) scores were dichotomized according to the determined Angoff cut points and knowledge by MCQ scores divided into quartiles, we were unable to proceed to bivariate analysis of categorical variables because of the small numbers of students who passed the clinical performance OSCE. Discussion Overall the results of the study indicate that final year nursing students lacked the knowledge, clinical skills, team work and situation awareness required to competently manage a deteriorating patient either as first response leaders or team members. Only a small proportion of student groups performed to the clinical standard determined by expert clinicians.
An audit of the curricula from the participating universities revealed that students had been taught the relevant skills and knowledge prior to participating in these simulation intervention, including; primary survey, oxygen therapy, electrocardiogram, acute coronary syndrome, respiratory distress, shock, altered conscious state, surgical emergencies, MET criteria for rapid response, principles and practice of working in a team and giving a patient handover using ISBAR (Identify, Situation, Background, Assessment and Recommendation) guidelines (Finnigan et al., 2010). It is noteworthy that each curriculum had been accredited by the professional accrediting authority — the Australian Nursing and Midwifery Council. Successful completion of accredited program requirements enables a graduate to be eligible for registration as a nurse. The conduct of healthcare education has come under the spotlight as both in nursing (Australian Nursing and Midwifery Accreditation Council (ANMAC), 2011) and in medicine (Australian Medical Council, 2009) there is a requirement that students learn teamwork and collaborative practices as a prescription for competence. This arises from concerns and evidence that effective teamwork and collaboration between health professionals can improve patient safety (Agency for Healthcare Research and Quality, 2009; Australian Commission on Safety and Quality in Healthcare, 2010; Greenfield et al., 2011). Our study findings support that final year nursing students would benefit from teamwork practice in the simulated or real settings and that this should be instituted. The conduct of deteriorating patient ‘rescue’ events by student nursing teams may bridge the current knowledge–practice gap. Despite preparation and an awareness that the simulation exercises would focus on patient deterioration, students' knowledge varied. No student received a full score, and on average students demonstrated two thirds of the requisite knowledge to manage the deteriorating patient as determined by the MCQ prior to the simulation exercise. As higher knowledge MCQ scores were associated with higher clinical performance OSCE scores in two scenarios this suggests that knowledge review is important prior to clinical placement. Compounding the application of underlying knowledge and experience to a clinical simulation scenario is the finding that approximately half the students had no exposure (and only 5% have been first responders), to the deteriorating patient and thus had limited opportunity to develop or hone potentially lifesaving clinical skills. However, our results suggest that previous experience in caring for a suddenly deteriorating patient improved team performance. This is in keeping with the body of evidence that skills should be the subject of deliberate practice, until an acceptable standard
Table 3 Situation awareness subscale and total scores by scenarios. Scenario (n = leader)
Cardiac n = 35 Shock n = 32 Respiratory n = 30 All Scenarios N = 97
Physiological perception 3 items
Global perception 3 items
Comprehension 2 items
Projection 4 items
Total SAGAT 12 items
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
0.60 (.60) 0.81 (.78) 0.93 (.69) 0.77 (sd .70)
0–2 (20%) 0–2 (27%) 0–2 (31%) 0–2 (26%)
0.97 (.75) 1.16 (.95) 0.73 (.78) 0.96 (sd .84)
0–3 (32%) 0–3 (39%) 0–3 (24%) 0–3 (32%)
0.88 (.76) 0.94 (.67) 0.83 (.53) 0.89 (sd .66)
0–2 (44%) 0–2 (47%) 0–2 (41%) 0–2 (44%)
2.48 (.82) 2.09 (.89) 2.47 (.63) 2.35 (sd .80)
1–4 (62%) 1–4 (52%) 2–4 (62%) 1–4 (59%)
4.94 (1.68) 5.00 (2.02) 4.97 (1.16) 4.97 (sd 1.65)
2–9 (41%) 1–9 (42%) 3–8 (41%) 1–9 (41%)
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
695
Table 4 Pattern of correlations between student experience and knowledge and clinical performance, teamwork and situation awareness by scenario.a Experience and knowledge Age
Age Ever worked as an employee in nursing or healthcare related field Ever cared for a patient who has suddenly deteriorated MCQ total score TEAM total SA total Cardiac OSCE Shock OSCE Respiratory OSCE a
Ever worked as an employee in nursing or healthcare related field
Ever cared for a patient who has suddenly deteriorated
MCQ total score
Team work and situation awareness
Scenario scores
Team total
Cardiac OSCE
SA total
Shock OSCE
Respiratory OSCE
−.398 (.024)
.226 (027) .217 (.034) .226 (.027)
−.398 (.024)
.217 (.034) .200 (.050) .363 (.032) .475 (.006)
.337 (.047) .638 (.000) .448 (.015)
.200 (.050)
.363 (.032) .337 (.047)
.475 (.006) .638 (.000) .474 (.006)
.448 (.015)
.474 (.006)
Spearman's rank order correlation and p-value; significant difference if p ≤ .05.
of performance is attained, and that skills become ‘second nature’ or are able to be performed smoothly and without apparent effort (Ericcson, 2004). An example of this is the accreditation in basic and advanced life support education that must be repeated within set time periods (Australian Resuscitation Council and New Zealand Resuscitation Council, 2012). Following this line of thinking, it is not surprising that nursing students need more practice of team-based clinical simulation for learning. A large intervention trial of a Safer Patient Initiative in the UK (Benning et al., 2011) that included tools for monitoring patients' condition reported difficulty in achieving improvements in a number of patient safety indices. It may be that improvements in patient safety now and in the future are dependent upon better team training for our graduates: student nurses and medical staff. Overall measures of clinical performance, teamwork and situation awareness indicated that students achieved half the possible total scores that were indicative of ideal performance. Results of the analysis for clinical performance when Modified Angoff cut points were applied indicated that approximately one in ten students were able to respond to the deteriorating patient with a level of clinical technical skill deemed acceptable by expert clinicians. This may or may not be typical of student performance at this stage of their education and experience. However, comparison of students' performance with those of registered nurses (Endacott et al., in review) suggests that the level of technical skills performance OSCE is similar between registered and student nurses. Our data analysis indicated that participants from University B had significantly lower mean scores for knowledge and situation awareness, and clinical performance in relation to one of the three scenarios; the cardiac scenario compared with the other two universities. This performance differential may reflect any number of factors including differing program entry requirements, curricula and pedagogy and levels of exposure to simulation experiences across institutions. Although it is beyond the scope of this paper to detail specific changes which ought to be made to individual programs, our findings point to the need for universities to examine and rectify factors which may be impeding knowledge, situation awareness and clinical performance. Measures of team working were consistently low across both the nontechnical and global subscales. This finding may reflect the relative lack of exposure to, or inclusion in, teamwork models experienced by students in clinical environments. This may be particularly so in clinical placement models that do not offer extended placement in one unit. However our results suggest that knowledge was significantly correlated with overall team score in students. It is therefore not surprising that significantly higher scores for RNs compared with student teams have been reported for all three subscales of the TEAM instrument (Endacott et al., in review).
TEAM instrument correlations confirmed moderate to strong relationships within the instrument, between the 11-item scale and global rating, and the reliability of the instrument was also confirmed. These claims to validity and reliability have recently been independently corroborated (McKay et al., 2012) and the TEAM instrument has been identified as being quick and easy to use to assess team working and nontechnical skills in resuscitation contexts and simulated environments and for global assessment of team performance (McKay et al., 2012) (Wollaston et al., 2004). In terms of situation awareness our findings suggest that this may be a poorly developed skill in nursing students. Situation awareness demands perception of the elements in the environment within a volume of time and space, and making subsequent meaning of these. It is a skill that develops with experience, although it is subject to individual differences (Tsang and Vidulich, 2006). Situation awareness is identified in the literature as the first step in decision making, allowing individuals to understand a situation and project what will occur next (Australian Commission on Safety and Quality in Healthcare, 2010). While experienced nurses can easily draw on a depth of knowledge and apply patterns of recognition to be highly aware, novice nurses can be placed under pressure with a high mental workload when faced with complex situations; leading to low situation awareness scores (Tsang and Vidulich, 2006). Situation awareness is a skill that is not explicitly taught or practiced in our three undergraduate programs however a significant difference was identified by institution. This suggests that there may be particular hidden curriculum influences that impact on the development of situation awareness skills (McKenna et al., in review). In our study students' knowledge was significantly correlated with situation awareness and this suggests that if students' knowledge was improved so might their situation awareness as proposed by Tsang and Vidulich (2006). How student nurses gain tacit knowledge of situation awareness is worthy of further investigation. Strengths and Limitations This study is, to our knowledge, one of the largest, multi-centre studies of nursing students' competence in managing deteriorating patients. Although in each institution students had been taught the requisite knowledge and skills to manage the deteriorating patient, the learning experiences in this study provided, for most, their first opportunity to integrate knowledge and skills in a safe, high fidelity simulated environment. Although student participation was based on a voluntary convenience sample whose performance may not be representative of all final year undergraduate nursing students, we assert that sampling of 97 students was adequate to reflect a broad range of student performances.
696
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
The use of a roving research team enhanced the reliability of the three simulation scenarios between the universities, and this enabled comparisons to be made across universities that will inform teaching quality improvement in relation to management of the deteriorating patient. Additionally the assessment of clinical performance (OSCE) and team work (TEAM) was scored by two observers, moderated and discussed at the completion of each scenario, to enhance inter-rater reliability. Further the use of validated and reliable instruments TEAM, and SAGAT was a strength of our study. The OSCE instruments to measure clinical performance specific to each scenario, were developed with an expert team of clinicians and academics to ensure face and content validity. Recognising the need to determine an appropriate level of clinical performance we used a Modified Angoff technique and the opinion of expert clinicians. While on clinical placement student performance is often assessed formally and informally by clinicians, it may be that the expert clinicians in our study had higher expectations of student performance that are unrealistic or different to those of clinicians who do not have specific expertise in relation to managing the deteriorating patient. The articulation and translation of the expert clinician's almost intuitive understanding of global performance into benchmarking objective performance criteria are complex tasks, however the Angoff technique provides a process to enhance rigor. We did not measure any potential intervening or confounding variables such as student confidence or leadership attributes at baseline or completion of the student experience. Anecdotally, students reported during the process of photo elicitation and self-evaluation that the simulation exercise made them more confident to lead responses and manage the deteriorating patient. A full analysis of the post intervention debriefing and evaluation is the subject of a separate paper yet to be published. Recommendations & Conclusions It is apparent from our study that nursing students are not able to recognise and manage the deteriorating patient to the extent expected by experienced clinicians. Our response to these findings is to recommend strategies to improve pre-requisite knowledge and to provide opportunities for students to integrate this knowledge, through curriculum renewal. Regular rehearsal of first response and team management of the deteriorating patient should become a mandatory component of students' clinical preparation, and ought to specifically address clinical performance, teamwork and situation awareness. While high fidelity simulation exercises such as we have undertaken are resource intensive, our future work will explore making these experiences accessible to all nursing students. Acknowledgments The Australian Government, Office for Learning and Teaching funded this project. The research team are especially grateful to the students who participated in the study. The authors acknowledge the contributions of other members of the FIRST2ACT™ research team: Lisa McKenna, Leigh Kinsmen, Ruth Endacott, Brett Devries, Helen Forbes and Robyn Hill and Research Assistant Karen MIssen. References Agency for Healthcare Research and Quality, 2009. TeamStepps: Strategies and Tools to Enhance Performance and Patient Safety. U.S. Department of Health and Human Services (http://teamstepps.ahrq.gov/index.htm. Accessed April 2011). Australian Commission on Safety and Quality in Healthcare, 2010. Consensus Statement: Essential Elements for Recognising and Responding to Clinical Deterioration. Australian Commission on Safety and Quality in Healthcare, Canberra (http://www.health.gov.au/
internet/safety/publishing.nsf/Content/EB5349066738C24CCA2575E70026C32A/$File/ national_consensus_statement.pdf). Australian Medical Council, 2009. Assessment and Accreditation for Medical Schools: Standards and Procedure. AMC, Canberra (http://www.amc.org.au/images/Medschool/ standards.pdf). Australian Nursing and Midwifery Accreditation Council (ANMAC), 2011. National Accreditation Framework. Australian Resuscitation Council and New Zealand Resuscitation Council, 2012. Changes to Basic Life Support. http://www.resus.org.au/files/summary-of-majorchangestobls-guidelines-nov10.pdf. Bell, M.B., Konrad, D., Granath, F., Ekbom, A., Martling, C.R., 2006. Prevalence and Sensitivity of MET-criteria in a Scandinavian University Hospital. Resuscitation 70 (1), 66–73. Benning, A., Ghaleb, M., Suokas, A., Dixon-Woods, M., Dawson, J., Barber, N., Al, E., 2011. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 342, d195. http://dx.doi.org/10.1136/bmj.d1195PMCID: PMC3033440. Bucknall, T.K., Jones, D., Bellomo, R., Staples, M., 2013. The RESCUE investigators. Responding to medical emergencies: system characteristics under examination (RESCUE). A prospective multi-site point prevalence study. Resuscitation 84, 179–183. Buykx, P., Kinsman, L., Cooper, S., McConnell-Henry, T., Cant, R., Endacott, R., Scholes, J., 2011. FIRST2 ACT: educating nurses to identify patient deterioration — a theory-based model for best practice simulation education. Nurse Educ. Today. http://dx.doi.org/ 10.1016/j.nedt.2011.03.006 (published on-line 8-APR-2011). Cooper, S., Kinsman, L., Buykx, P., McConnell-Henry, T., Endacott, R., Scholes, J., 2010a. Managing the deteriorating patient in a simulated environment: nursing students' knowledge, skill and situation awareness. J. Clin. Nurs. 19, 2309–2318. Cooper, S., C., R., Sellick, K., Porter, J., Somers, G., Kinsman, L., Nestel, D., 2010b. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM). Resuscitation 81, 446–452. http://dx.doi.org/10.1016/ j.resuscitation.2009.1011.1027 (Resuscitation 81: 446–52). Cooper, S., McConnell-Henry, T., Cant, R., Porter, J., Missen, K., Kinsman, L., Endacott, R., Scholes, J., 2011a. Managing deteriorating patients: registered nurses' performance in a simulated setting. Open Nurs. J. 5, 120–126. http://dx.doi.org/10.2174/1874434 6011050100120. Cooper, S., M.-H., T., Cant, R., et al., 2011b. Managing deteriorating patients: registered nurses' performance in a simulated setting. Sim Health 2011 — Patient Centered Simulation. Hilton Hotel, Sydney, pp. 12–15 (Sept 2011). Cooper, S., Beauchamp, A., Bogossian, F., Bucknall, T., Cant, R., Devries, B., Endacott, R., Forbes, H., Hill, R., Kinsman, L., Kain, V.J., McKenna, L., Porter, J., Phillips, N., Young, S., 2012. Managing patient deterioration: a protocol for enhancing undergraduate nursing students' competence through web-based simulation and feedback techniques. BMC Nurs. 11 (1), 18. DeVita, M.A., Smith, G.B., Adam, S.K., Adams-Pizarro, I., Buist, M., Bellomo, R., Bonello, R., Cerchiari, E., Farlow, B., Goldsmith, D., Haskell, H., Hillman, K., Howell, M., Hravnak, M., Hunt, E.A., Hvarfner, A., Kellett, J., Lighthall, G.K., Lippert, A., Lippert, F.K., Mahroof, R., Myers, J.S., Rosen, M., Reynolds, S., Rotondi, A., Rubulotta, F., Winters, B., 2010. “Identifying the hospitalised patient in crisis” — a consensus conference on the afferent limb of rapid response systems. Resuscitation 81 (4), 375–382. DiGiuseppe, D.L., Aron, D.C., Ranbom, L., Harper, D.L., Rosenthal, G.E., 2002. Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern. Child Health J. 6 (3), 169–179. Endacott, R., Jevon, P., Cooper, S., 2009. Clinical Nursing Skills: Core And Advanced. Oxford University Press, Oxford. Endacott, R., Bogossian, F., Cooper, S., Forbes, H., Kain, V., Young, S., Porter, J., 2013w. Leadership and teamwork in medical emergencies: performance of nursing students and Registered nurses in simulated patient scenarios. Simul. Healthc. (in review). Ericcson, K., 2004. Deliberate practice and the acquisition and maintenance of expert performance in medical and related domains. Acad. Med. 79 (10), S70–S81. Finnigan, M.A., Marshall, S.D., Flanagan, B.T., 2010. ISBAR for clear communication: one hospital's experience spreading the message. Aust. Health Rev. 34 (4), 400–404. Fuhrmann, L., Lippert, A., Perner, A., Ostergaard, D., 2008. Incidence, staff awareness and mortality of patients at risk on general wards. Resuscitation 77 (3), 325–330. Greenfield, D., Nugus, P., Travaglia, J., et al., 2011. Auditing an organisation's interprofessional learning and interprofessional practice; the interprofessional praxis audit framework. J. Interprof. Care 24, 436–449. Harper, D., 1994. On the Authority of the Image: Visual Methods at the Crossroads. Sage, Thousand Oaks 403–412. IBM Corporation, Released, 2011. IBM SPSS Statistics for Windows. IBM Corp., Armonk, New York. McKay, A., Walkerb, S., Brettc, S., Vincent, C., Sevdalis, N., 2012. Team performance in resuscitation teams: comparison and critique of two recently developed scoring tools. Resuscitation 83, 1478–1483. McKenna, L., Missen, K., Cooper, S., Bogossian, F., Cant, R., 2013w. Situation awareness in undergraduate students during a simulated patient scenario. Nurse Educ. Today (in review). Ricker, K., 2006. Setting cut scores: a critical review of the Angoff and Modified Angoff methods. Alberta J. Educ. Res. 52 (1), 53–63. Tsang, P., Vidulich, M., 2006. Mental workload and situation awareness [chapter 9], In: Salvendy, G. (Ed.), Handbook of Human Factors and Ergonomics, Third ed. John Wiley & Sons, Inc., New York, p. pp. 243. Wollaston, A., Fahey, P., McKay, M., Hegney, D., Miller, P., Wollaston, J., 2004. Reliability and validity of the Toowoomba adult trauma triage tool: a Queensland, Australia study. Accid. Emerg. Nurs. 12 (4), 230–237.
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
Undergraduate nursing students' performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: An Australian multi-centre study☆ Fiona Bogossian a,⁎, Simon Cooper b, Robyn Cant b, Alison Beauchamp b,e, Joanne Porter c, Victoria Kain a, Tracey Bucknall d, Nicole M. Phillips d, The FIRST2ACT™ Research Team a
The School of Nursing & Midwifery, The University of Queensland, Herston Campus, QLD, Australia The School of Nursing & Midwifery, Monash University, Berwick Campus, VIC, Australia c The School of Nursing & Midwifery, Monash University, Gippsland Campus, VIC, Australia d School of Nursing and Midwifery Deakin University, Burwood Campus, VIC, Australia e Population Health Strategic Research Centre, Deakin University, Burwood Campus, VIC, Australia b
a r t i c l e
i n f o
Article history: Accepted 18 September 2013
Keywords: Education Nursing Patient deterioration Simulation Clinical performance Clinical decision making Situation awareness Team work
s u m m a r y Background: Early recognition and situation awareness of sudden patient deterioration, a timely appropriate clinical response, and teamwork are critical to patient outcomes. High fidelity simulated environments provide the opportunity for undergraduate nursing students to develop and refine recognition and response skills. Objectives: This paper reports the quantitative findings of the first phase of a larger program of ongoing research: Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends (FIRST2ACTTM). It specifically aims to identify the characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients. Design: Mixed-method multi-centre study. Setting: High fidelity simulated acute clinical environment in three Australian universities. Participants: A convenience sample of 97 final year nursing students enrolled in an undergraduate Bachelor of Nursing or combined Bachelor of Nursing degree were included in the study. Method: In groups of three, participants proceeded through three phases: (i) pre-briefing and completion of a multi-choice question test, (ii) three video-recorded simulated clinical scenarios where actors substituted real patients with deteriorating conditions, and (iii) post-scenario debriefing. Clinical performance, teamwork and situation awareness were evaluated, using a validated standard checklist (OSCE), Team Emergency Assessment Measure (TEAM) score sheet and Situation Awareness Global Assessment Technique (SAGAT). A Modified Angoff technique was used to establish cut points for clinical performance. Results: Student teams engaged in 97 simulation experiences across the three scenarios and achieved a level of clinical performance consistent with the experts' identified pass level point in only 9 (1%) of the simulation experiences. Knowledge was significantly associated with overall teamwork (p = .034), overall situation awareness (p = .05) and clinical performance in two of the three scenarios (p = .032 cardiac and p = .006 shock). Situation awareness scores of scenario team leaders were low overall, with an average total score of 41%. Conclusions: Final year undergraduate nursing students may have difficulty recognising and responding appropriately to patient deterioration. Improving pre-requisite knowledge, rehearsal of first response and team management strategies need to be a key component of undergraduate nursing students' education and ought to specifically address clinical performance, teamwork and situation awareness. © 2013 Elsevier Ltd. All rights reserved.
Introduction The prevalence of deteriorating hospital patients measured as those who fulfill Medical Emergency Team (MET) criteria has been ☆ Ethical approval: Monash University CF11/3414-201101825, Deakin University 2012-030 and The University of Queensland 201200011. ⁎ Corresponding author. E-mail address: [email protected] (F. Bogossian). 0260-6917/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.nedt.2013.09.015
demonstrated to range from 3.3% in a recent Australian study (Bucknall et al., 2013) up to 18% in a Danish study (Fuhrmann et al., 2008). Studies consistently demonstrate significant findings that patients who fulfill MET criteria for rapid response have in excess of double the risk of mortality both in hospital and at 30 days (Bell et al., 2006; Bucknall et al., 2013; Fuhrmann et al., 2008). Moreover hospital patients who are deteriorating physiologically are frequently mismanaged, leading to international concerns over patient safety (Agency for Healthcare Research and Quality, 2009; Cooper et al., 2011a,b; DeVita et al., 2010).
692
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
Nurses as front line care providers are in a position to detect and respond to changes in vital signs and other cues reflecting patient deterioration and in doing so directly impact patient mortality. Following a series of pilot studies examining the performance of nursing students and qualified staff in simulated settings we have developed a program of learning called FIRST2ACT™ (Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends) (Buykx et al., 2011). In recent trials the program demonstrated a significant impact on learning, and for hospital nurses, a significant impact on clinical practice (Buykx et al., 2011). Following the intervention an audit of patients' medical records showed improvements in the charting of vital signs, in pain score recording and in the correct delivery of oxygen therapy. The current study builds on the previous research (Buykx et al., 2011) and forms the first phase of a larger program of research funded by the Australian Learning and Teaching Council, the protocol of which has been published elsewhere (Cooper et al., 2012). This phase focuses on understanding clinical performance, teamwork, situation awareness and decision-making in undergraduate nursing students and this paper reports quantitative findings from the first phase. Aims The aims of this study were firstly to identify characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients. Secondly, to explore interactions between the outcome measures in order to identify factors potentially amenable to modification for the improvement of patient safety. Method A mixed method multi-centre study was designed to focus on undergraduate nursing students and their experiences in caring for deteriorating patients. Undergraduate nursing students were recruited from three Australian universities, two from Victoria and one from Queensland. In order to be able to recognise and respond to sudden patient deterioration, students required some level of exposure to relevant theory and clinical practice, through a standard educational program on emergency care, thus only final year students were considered eligible. Although instruments and design had been previously tested in a group of registered nurses (Cooper et al., 2011a,b) the current research is predominantly descriptive and thus the determination of an a priori sample size is unnecessary. Based on previous studies, in which response rates varied from 46% for nursing students to 82% for registered nurses, we anticipated a response rate in excess of 50%. Following ethical approval from each of the three universities final year nursing students were invited to participate in the study. Students were provided with standardised information about the project via email in the first instance and then via a 5-minute power point presentation conducted by a faculty member who was not a part of the research team. Students responded in the form of an expression of interest, if they wished to participate. Study inclusion was on a first come, first served, basis at each site and students reconfirmed their participation when allocated a scheduled session time and date. These strategies were employed to maximize participation within the physical and human resource limitations on the number of students who could participate at each university site. The intervention and data collection were conducted over 2 days at each site over a 3-week period. In order to ensure the highest possible level of reliability and validity over the multiple sites and days, the intervention was delivered by a core roving team including three members of the research team (SC, JP & AB) and one trained actor who were supported by other members of the research team based on the relevant university site. At each site a clinical simulation space that replicated a ward-like environment and standardised equipment was assembled to provide a
consistent student experience across all sites. Students proceeded in groups of three through the research stages (i) pre-intervention briefing, (ii) the simulation intervention and (iii) post-intervention debriefing (see Table 1). Pre-intervention Briefing The pre-intervention briefing included written participant information and a standard verbal explanation of the project prior to written consent. This was followed by the completion of a questionnaire in order to collect demographic and clinical experience data and an 11 item multiple-choice questionnaire (MCQ). Each participant was assigned a participant number during the pre-intervention stage and all data collected (written and video) from the individual participant was assigned this number. The assignment of the team leader role order for the intervention within each trio of students was randomly determined. The Simulation Intervention Each trio of students then moved to the clinical simulation space waiting area. For each scenario the assigned team leader was given a brief handover and was instructed to enter the room. Video filming began and the team leader responded to the cues of the environment and the patient actor and was able to call for assistance from other team members at any time. The total duration of each simulation scenario was 8 min, with subtle deterioration cues evident in the first 4 min prior to more obvious and significant signs of deterioration in the final 4 min (Cooper et al., 2011a,b). Two assessors assessed the clinical and non-technical skills performance of the participant trio, and then agreed a final score. Blinding was not possible. At the conclusion of each scenario the team leader was taken aside and asked a series of rapid-fire situation awareness questions by a research team member to ascertain their understanding and awareness of the situation as well as their prediction of future events in relation to the simulation. Post Intervention Debriefing Following completion of the clinical simulation component participants underwent debriefing which consisted of video review, performance feedback and written participant evaluation of the experience. Participant trios under the guidance of member of the research team undertook video review of their performance who encouraged participants to self-evaluate and give a reflective account of their performance and decision making in a process called ‘photo elicitation’ (DiGiuseppe et al., 2002; Harper, 1994). Following this, participants were asked to complete a written evaluation of the experience.
Table 1 Phases and related activities in First2Act simulation. Pre-intervention
Intervention Scenario
Roles
Measures
Consent Briefing
Cardiac
Leader A All team
SAGAT OSCE TEAM
Participant ID assigned Participant information questionnaire demographic and clinical experience Multiple choice questionnaire (MCQ) 11 items to test knowledge
Shock
Leader B All team
SAGAT OSCE TEAM
Respiratory
Leader C
SAGAT
All team
OSCE TEAM
Postintervention Debriefing Photo elicitation and video review Self evaluation Instructor feedback
Written evaluation of simulation experience
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
The total duration of the experience for participants was 2 h, and participants were able to use this experience as a component of their clinical learning activities and as evidence of research engagement. Participants were issued with a certificate of participation for inclusion in their professional portfolios.
Instruments and Measures Clinical knowledge was measured using an 11 item MCQ adapted from previously validated instruments (Cooper, 2010; Endacott et al., 2009) and verified by a panel of clinical experts, which was designed to test knowledge of the deteriorating patient. Clinical performance was assessed using a standardized Objective Structured Clinical Examination (OSCE) score sheet (the number of items varied n = 24, 28 and 26 according to the scenario). In order to set the ‘pass mark’ for the OSCE score we followed a Modified Angoff technique (Ricker, 2006) which is subsequently described. A group of clinical nursing experts, with 7–27 years clinical experience (mean 15 years), who also had teaching experience, reviewed the OSCE scoring for each scenario. The group was asked to recall students whom they had taught to manage patients experiencing similar deterioration and to consider those that ‘bordered on mastery’, or put another way met the minimum safe level of competency. These students were then discussed focusing on what they were able to achieve and what they were not able to achieve in order to develop ‘a common notion of a borderline candidate’. Individual experts identified the score items on each OSCE that represented a borderline pass i.e. which nominal (yes/no) criteria must (and are likely to) be achieved by a borderline candidate. The mean of their summed scores was calculated to provide a cut point indicative of a pass mark for each OSCE scenario. Non-technical skills of each student trio were assessed using the Team Emergency Assessment Measure (TEAM). The TEAM instrument includes ratings of non-technical skills (11 items on a 0–4 point scale) that include subscales for leadership, team working and task management with possible scores of 8, 28 and 8 respectively. The instrument also yields an overall global rating (0–10 point scale) and has previously been demonstrated to be valid and reliable (Cooper et al., 2010a). Situation awareness of the team leader was assessed using Situation Awareness Global Assessment Technique (SAGAT) designed to identify the leader's awareness of the patient's physiologic state as well as awareness of the wider situation (Cooper et al., 2010b). The 12-item instrument provides subscale measures of four components of situation awareness, namely physiological perception (3 items), global situation perception (3 items), comprehension (2 items) and projection (4 items). The SAGAT instrument has been previously validated (Bell et al., 2006).
Analysis Data for individual participants across all sites was pooled and entered into a database, and statistical analyses undertaken using SPSS (IBM Corporation, Released, 2011). A significance level of 0.05 was determined for all descriptive and inferential tests. No values were imputed for missing data. Scores for clinical knowledge, clinical performance, teamwork and situation awareness (MCQ, OSCE, TEAM and SAGAT instruments respectively) were summed, and analyzed as continuous variables in the first instance and descriptive summary statistics employed. Inferential statistics were employed to test associations between variables. To further explore predictors of student success in managing the deteriorating patient clinical performance (OSCE) scores were dichotomized according to the determined Angoff cut points and knowledge by MCQ scores divided into quartiles to enable subsequent bivariate analysis of categorical variables.
693
Results A total of 570 final year nursing students studying a Bachelor of Nursing or combined degrees from University A (n = 120); University B (n = 350); and University C (n = 100) were eligible for inclusion. Ninety-seven students participated in the study from University A (n = 34), University B (n = 32) and University C (n = 31) yielding participation rates of 28%, 9% and 31% by institution. The majority of participants (92%) were enrolled in a Bachelor of Nursing degree with the remainder enrolled in dual degree and graduate entry nursing programs leading to registration as a nurse. Three unintentionally recruited Bachelor of Midwifery students and one registered nurse were ineligible for inclusion and were subsequently omitted from analysis of performance; however as they had been allocated to a trio they participated in the scenarios in non-leadership roles and in debriefing. Of the 97 participants the majority (93%) were female and the median age was 21 years (mean 22.59 ± 5.47; range 18–52). Clinical exposure was assessed by the frequency and type of clinical placements students had experienced throughout their degree. A total of 95% had at least 3 clinical placements and most frequently this included general medical surgical units (48%). Approximately half (52%) of the students had previous experience in health care related employment in assistant roles. A total of fifty students (52%) reported exposure to the real life experience of a deteriorating patient. Of those, the predominant roles were as observers (n = 29) or recorder scribes (n = 10). Only 5 students (5%) reported they had previously been first responders to a deteriorating patient. Clinical knowledge assessed prior to the intervention using the 11-item MCQ revealed a range of scores from 4 to 10 and a mean of 7.25 (SD 1.48, CI 6.93–7.56). A significant difference (p = .003) was observed between mean scores knowledge for students by universities. The mean score for University B was 6.50 (SD 1.37, CI 6.01–6.99) lower than University A, 7.74 (SD 1.42, CI 7.24–8.23) and University C, 7.48 (SD1.63, CI 6.89–8.08). Performance Assessment Three measures of performance — clinical performance, teamwork and situation awareness are reported, the former two relate to group performance and the latter relates to performance of the individual scenario leader. Summary scores for each measure are in Table 2. Clinical performance for each team of three students was assessed by OSCE. Using the Modified Angoff technique indicative pass marks were established for the OSCE scores and set by scenarios at 59% cardiac, 70% shock, and 67% respiratory. Only 9 student teams passed an OSCE based on the cut points and the remainder (91%) failed to achieve the set level of clinical performance. When total score was examined by university there was a significant difference (p = .027) in mean OSCE scores in the cardiac scenario with University B (mean 11.0) exhibiting lower scores than the other institutions (University A mean 14.1 and University C mean 12.3). Nontechnical skills were assessed using the TEAM instrument. The average total score for the 11 items was 38% and the average global score was 37% (see Table 2), with strong item-to-total associations (n = 92, r = .848, p b .001). The summary results for each subscale were; leadership — mean 2.76, SD 1.53, team working — mean 11.13, SD 4.42 and task management — mean 2.71, SD 1.46. All components of the TEAM instrument demonstrated significant positive associations, namely: item-to-total scores (range r = .599–.838, p = .000); item to Item (range r = .212–.838, p = .000); and item to global score (range r = .422–.838). Global scores were strongly associated with the three subscales; leadership (r = .708, p = .000), team working (r = .960, p = .000) and task management (r = .800, p = .000). A Cronbach's alpha coefficient (α) of .912 for the TEAM scale of 11 items confirmed adequate reliability (expected N .70). Alpha for each of the three subscales
694
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
Table 2 Summary scores of clinical performance, teamwork and situation awareness by scenario. Clinical performance — OSCE
Teamwork — TEAM
Situation awareness — SAGAT
Scenario (n = groups)
Mean score (% of total score)
Range/total (sd)
95% CI
Mean score (% of total score)
Range/total (sd)
95% CI
Cardiac n = 35 Shock n = 32 Respiratory n = 30 Total
12.65 (45%) 12.66 (49%) 12.96 (54%) 12.75 (49%)
7–18/28 (2.83) 8–19/26 (3.34) 7–18/24 (2.61) 7–19/25.3 (2.92)
11.68–13.63 11.45–13.86 11.99–13.91 12.16–13.34
17.54 (40%) 3–32/44 (6.66) 15.25–19.83 14.59 (33%) 3–33/44 (6.74) 12.16–17.03 17.29 (39%) 8–31/44 (5.99) 15.55–20.29 Non-technical performance total (n = 94) 16.60 (38%) 3–33/44 (6.60) 15.30–18.00 Global rating (n = 94) 3.65 (37%) 1–7/10 (1.35) 3.37–3.93
Mean score (% of total score)
Range/total (sd)
95% CI
4.94 (41%) 5.00 (42%) 4.97 (41%) 4.96 (41%)
2–9/12 (1.68) 1–9 (2.02) 3–8 (1.16) 1–9/12 (1.65)
4.37–5.52 4.27–5.73 4.53–5.40 4.64–5.30
Note: sample may vary owing to incomplete/missing data.
was confirmed as adequate: leadership α = .818 (2 items); team working α = .881 (7 items); and task management α = .842 (2 items). Considering two of the subscales comprised only 2 items, the reliability was regarded as being high. Situation awareness scores (see Table 3) were low overall with an average score of 41% and within the subscales of physiological perception 26%, global perception 32% and comprehension 44%. However students were better at forecasting likely medical consequences with a projection average score of 59%. When examined by university a significant difference was identified for mean situation awareness scores (p = .03). University C scored 5.61 (SD 1.70, CI 4.99–6.24), University A scored 4.91 (SD 1.68, CI 4.33–.49) and University B scored 4.41 (SD1.39, CI 3.91–4.91). Experience and Knowledge Related to Performance The relationship between students' experience and knowledge (age, previous employment in nursing or health care related field, having experience of caring for the deteriorating patient and MCQ) and clinical performance, teamwork and situation awareness was examined. Significant correlations are reported in Table 4. Knowledge as measured by MCQ scores was significantly associated (p ≤ .05) with total teamwork, total situation awareness and clinical performance in cardiac and shock scenarios. Having previously cared for a patient who suddenly deteriorated was associated with overall teamwork performance. Teamwork performance was correlated with clinical performance in each of the scenarios. Although clinical performance (OSCE) scores were dichotomized according to the determined Angoff cut points and knowledge by MCQ scores divided into quartiles, we were unable to proceed to bivariate analysis of categorical variables because of the small numbers of students who passed the clinical performance OSCE. Discussion Overall the results of the study indicate that final year nursing students lacked the knowledge, clinical skills, team work and situation awareness required to competently manage a deteriorating patient either as first response leaders or team members. Only a small proportion of student groups performed to the clinical standard determined by expert clinicians.
An audit of the curricula from the participating universities revealed that students had been taught the relevant skills and knowledge prior to participating in these simulation intervention, including; primary survey, oxygen therapy, electrocardiogram, acute coronary syndrome, respiratory distress, shock, altered conscious state, surgical emergencies, MET criteria for rapid response, principles and practice of working in a team and giving a patient handover using ISBAR (Identify, Situation, Background, Assessment and Recommendation) guidelines (Finnigan et al., 2010). It is noteworthy that each curriculum had been accredited by the professional accrediting authority — the Australian Nursing and Midwifery Council. Successful completion of accredited program requirements enables a graduate to be eligible for registration as a nurse. The conduct of healthcare education has come under the spotlight as both in nursing (Australian Nursing and Midwifery Accreditation Council (ANMAC), 2011) and in medicine (Australian Medical Council, 2009) there is a requirement that students learn teamwork and collaborative practices as a prescription for competence. This arises from concerns and evidence that effective teamwork and collaboration between health professionals can improve patient safety (Agency for Healthcare Research and Quality, 2009; Australian Commission on Safety and Quality in Healthcare, 2010; Greenfield et al., 2011). Our study findings support that final year nursing students would benefit from teamwork practice in the simulated or real settings and that this should be instituted. The conduct of deteriorating patient ‘rescue’ events by student nursing teams may bridge the current knowledge–practice gap. Despite preparation and an awareness that the simulation exercises would focus on patient deterioration, students' knowledge varied. No student received a full score, and on average students demonstrated two thirds of the requisite knowledge to manage the deteriorating patient as determined by the MCQ prior to the simulation exercise. As higher knowledge MCQ scores were associated with higher clinical performance OSCE scores in two scenarios this suggests that knowledge review is important prior to clinical placement. Compounding the application of underlying knowledge and experience to a clinical simulation scenario is the finding that approximately half the students had no exposure (and only 5% have been first responders), to the deteriorating patient and thus had limited opportunity to develop or hone potentially lifesaving clinical skills. However, our results suggest that previous experience in caring for a suddenly deteriorating patient improved team performance. This is in keeping with the body of evidence that skills should be the subject of deliberate practice, until an acceptable standard
Table 3 Situation awareness subscale and total scores by scenarios. Scenario (n = leader)
Cardiac n = 35 Shock n = 32 Respiratory n = 30 All Scenarios N = 97
Physiological perception 3 items
Global perception 3 items
Comprehension 2 items
Projection 4 items
Total SAGAT 12 items
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
Mean (sd)
Range (%)
0.60 (.60) 0.81 (.78) 0.93 (.69) 0.77 (sd .70)
0–2 (20%) 0–2 (27%) 0–2 (31%) 0–2 (26%)
0.97 (.75) 1.16 (.95) 0.73 (.78) 0.96 (sd .84)
0–3 (32%) 0–3 (39%) 0–3 (24%) 0–3 (32%)
0.88 (.76) 0.94 (.67) 0.83 (.53) 0.89 (sd .66)
0–2 (44%) 0–2 (47%) 0–2 (41%) 0–2 (44%)
2.48 (.82) 2.09 (.89) 2.47 (.63) 2.35 (sd .80)
1–4 (62%) 1–4 (52%) 2–4 (62%) 1–4 (59%)
4.94 (1.68) 5.00 (2.02) 4.97 (1.16) 4.97 (sd 1.65)
2–9 (41%) 1–9 (42%) 3–8 (41%) 1–9 (41%)
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
695
Table 4 Pattern of correlations between student experience and knowledge and clinical performance, teamwork and situation awareness by scenario.a Experience and knowledge Age
Age Ever worked as an employee in nursing or healthcare related field Ever cared for a patient who has suddenly deteriorated MCQ total score TEAM total SA total Cardiac OSCE Shock OSCE Respiratory OSCE a
Ever worked as an employee in nursing or healthcare related field
Ever cared for a patient who has suddenly deteriorated
MCQ total score
Team work and situation awareness
Scenario scores
Team total
Cardiac OSCE
SA total
Shock OSCE
Respiratory OSCE
−.398 (.024)
.226 (027) .217 (.034) .226 (.027)
−.398 (.024)
.217 (.034) .200 (.050) .363 (.032) .475 (.006)
.337 (.047) .638 (.000) .448 (.015)
.200 (.050)
.363 (.032) .337 (.047)
.475 (.006) .638 (.000) .474 (.006)
.448 (.015)
.474 (.006)
Spearman's rank order correlation and p-value; significant difference if p ≤ .05.
of performance is attained, and that skills become ‘second nature’ or are able to be performed smoothly and without apparent effort (Ericcson, 2004). An example of this is the accreditation in basic and advanced life support education that must be repeated within set time periods (Australian Resuscitation Council and New Zealand Resuscitation Council, 2012). Following this line of thinking, it is not surprising that nursing students need more practice of team-based clinical simulation for learning. A large intervention trial of a Safer Patient Initiative in the UK (Benning et al., 2011) that included tools for monitoring patients' condition reported difficulty in achieving improvements in a number of patient safety indices. It may be that improvements in patient safety now and in the future are dependent upon better team training for our graduates: student nurses and medical staff. Overall measures of clinical performance, teamwork and situation awareness indicated that students achieved half the possible total scores that were indicative of ideal performance. Results of the analysis for clinical performance when Modified Angoff cut points were applied indicated that approximately one in ten students were able to respond to the deteriorating patient with a level of clinical technical skill deemed acceptable by expert clinicians. This may or may not be typical of student performance at this stage of their education and experience. However, comparison of students' performance with those of registered nurses (Endacott et al., in review) suggests that the level of technical skills performance OSCE is similar between registered and student nurses. Our data analysis indicated that participants from University B had significantly lower mean scores for knowledge and situation awareness, and clinical performance in relation to one of the three scenarios; the cardiac scenario compared with the other two universities. This performance differential may reflect any number of factors including differing program entry requirements, curricula and pedagogy and levels of exposure to simulation experiences across institutions. Although it is beyond the scope of this paper to detail specific changes which ought to be made to individual programs, our findings point to the need for universities to examine and rectify factors which may be impeding knowledge, situation awareness and clinical performance. Measures of team working were consistently low across both the nontechnical and global subscales. This finding may reflect the relative lack of exposure to, or inclusion in, teamwork models experienced by students in clinical environments. This may be particularly so in clinical placement models that do not offer extended placement in one unit. However our results suggest that knowledge was significantly correlated with overall team score in students. It is therefore not surprising that significantly higher scores for RNs compared with student teams have been reported for all three subscales of the TEAM instrument (Endacott et al., in review).
TEAM instrument correlations confirmed moderate to strong relationships within the instrument, between the 11-item scale and global rating, and the reliability of the instrument was also confirmed. These claims to validity and reliability have recently been independently corroborated (McKay et al., 2012) and the TEAM instrument has been identified as being quick and easy to use to assess team working and nontechnical skills in resuscitation contexts and simulated environments and for global assessment of team performance (McKay et al., 2012) (Wollaston et al., 2004). In terms of situation awareness our findings suggest that this may be a poorly developed skill in nursing students. Situation awareness demands perception of the elements in the environment within a volume of time and space, and making subsequent meaning of these. It is a skill that develops with experience, although it is subject to individual differences (Tsang and Vidulich, 2006). Situation awareness is identified in the literature as the first step in decision making, allowing individuals to understand a situation and project what will occur next (Australian Commission on Safety and Quality in Healthcare, 2010). While experienced nurses can easily draw on a depth of knowledge and apply patterns of recognition to be highly aware, novice nurses can be placed under pressure with a high mental workload when faced with complex situations; leading to low situation awareness scores (Tsang and Vidulich, 2006). Situation awareness is a skill that is not explicitly taught or practiced in our three undergraduate programs however a significant difference was identified by institution. This suggests that there may be particular hidden curriculum influences that impact on the development of situation awareness skills (McKenna et al., in review). In our study students' knowledge was significantly correlated with situation awareness and this suggests that if students' knowledge was improved so might their situation awareness as proposed by Tsang and Vidulich (2006). How student nurses gain tacit knowledge of situation awareness is worthy of further investigation. Strengths and Limitations This study is, to our knowledge, one of the largest, multi-centre studies of nursing students' competence in managing deteriorating patients. Although in each institution students had been taught the requisite knowledge and skills to manage the deteriorating patient, the learning experiences in this study provided, for most, their first opportunity to integrate knowledge and skills in a safe, high fidelity simulated environment. Although student participation was based on a voluntary convenience sample whose performance may not be representative of all final year undergraduate nursing students, we assert that sampling of 97 students was adequate to reflect a broad range of student performances.
696
F. Bogossian et al. / Nurse Education Today 34 (2014) 691–696
The use of a roving research team enhanced the reliability of the three simulation scenarios between the universities, and this enabled comparisons to be made across universities that will inform teaching quality improvement in relation to management of the deteriorating patient. Additionally the assessment of clinical performance (OSCE) and team work (TEAM) was scored by two observers, moderated and discussed at the completion of each scenario, to enhance inter-rater reliability. Further the use of validated and reliable instruments TEAM, and SAGAT was a strength of our study. The OSCE instruments to measure clinical performance specific to each scenario, were developed with an expert team of clinicians and academics to ensure face and content validity. Recognising the need to determine an appropriate level of clinical performance we used a Modified Angoff technique and the opinion of expert clinicians. While on clinical placement student performance is often assessed formally and informally by clinicians, it may be that the expert clinicians in our study had higher expectations of student performance that are unrealistic or different to those of clinicians who do not have specific expertise in relation to managing the deteriorating patient. The articulation and translation of the expert clinician's almost intuitive understanding of global performance into benchmarking objective performance criteria are complex tasks, however the Angoff technique provides a process to enhance rigor. We did not measure any potential intervening or confounding variables such as student confidence or leadership attributes at baseline or completion of the student experience. Anecdotally, students reported during the process of photo elicitation and self-evaluation that the simulation exercise made them more confident to lead responses and manage the deteriorating patient. A full analysis of the post intervention debriefing and evaluation is the subject of a separate paper yet to be published. Recommendations & Conclusions It is apparent from our study that nursing students are not able to recognise and manage the deteriorating patient to the extent expected by experienced clinicians. Our response to these findings is to recommend strategies to improve pre-requisite knowledge and to provide opportunities for students to integrate this knowledge, through curriculum renewal. Regular rehearsal of first response and team management of the deteriorating patient should become a mandatory component of students' clinical preparation, and ought to specifically address clinical performance, teamwork and situation awareness. While high fidelity simulation exercises such as we have undertaken are resource intensive, our future work will explore making these experiences accessible to all nursing students. Acknowledgments The Australian Government, Office for Learning and Teaching funded this project. The research team are especially grateful to the students who participated in the study. The authors acknowledge the contributions of other members of the FIRST2ACT™ research team: Lisa McKenna, Leigh Kinsmen, Ruth Endacott, Brett Devries, Helen Forbes and Robyn Hill and Research Assistant Karen MIssen. References Agency for Healthcare Research and Quality, 2009. TeamStepps: Strategies and Tools to Enhance Performance and Patient Safety. U.S. Department of Health and Human Services (http://teamstepps.ahrq.gov/index.htm. Accessed April 2011). Australian Commission on Safety and Quality in Healthcare, 2010. Consensus Statement: Essential Elements for Recognising and Responding to Clinical Deterioration. Australian Commission on Safety and Quality in Healthcare, Canberra (http://www.health.gov.au/
internet/safety/publishing.nsf/Content/EB5349066738C24CCA2575E70026C32A/$File/ national_consensus_statement.pdf). Australian Medical Council, 2009. Assessment and Accreditation for Medical Schools: Standards and Procedure. AMC, Canberra (http://www.amc.org.au/images/Medschool/ standards.pdf). Australian Nursing and Midwifery Accreditation Council (ANMAC), 2011. National Accreditation Framework. Australian Resuscitation Council and New Zealand Resuscitation Council, 2012. Changes to Basic Life Support. http://www.resus.org.au/files/summary-of-majorchangestobls-guidelines-nov10.pdf. Bell, M.B., Konrad, D., Granath, F., Ekbom, A., Martling, C.R., 2006. Prevalence and Sensitivity of MET-criteria in a Scandinavian University Hospital. Resuscitation 70 (1), 66–73. Benning, A., Ghaleb, M., Suokas, A., Dixon-Woods, M., Dawson, J., Barber, N., Al, E., 2011. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 342, d195. http://dx.doi.org/10.1136/bmj.d1195PMCID: PMC3033440. Bucknall, T.K., Jones, D., Bellomo, R., Staples, M., 2013. The RESCUE investigators. Responding to medical emergencies: system characteristics under examination (RESCUE). A prospective multi-site point prevalence study. Resuscitation 84, 179–183. Buykx, P., Kinsman, L., Cooper, S., McConnell-Henry, T., Cant, R., Endacott, R., Scholes, J., 2011. FIRST2 ACT: educating nurses to identify patient deterioration — a theory-based model for best practice simulation education. Nurse Educ. Today. http://dx.doi.org/ 10.1016/j.nedt.2011.03.006 (published on-line 8-APR-2011). Cooper, S., Kinsman, L., Buykx, P., McConnell-Henry, T., Endacott, R., Scholes, J., 2010a. Managing the deteriorating patient in a simulated environment: nursing students' knowledge, skill and situation awareness. J. Clin. Nurs. 19, 2309–2318. Cooper, S., C., R., Sellick, K., Porter, J., Somers, G., Kinsman, L., Nestel, D., 2010b. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM). Resuscitation 81, 446–452. http://dx.doi.org/10.1016/ j.resuscitation.2009.1011.1027 (Resuscitation 81: 446–52). Cooper, S., McConnell-Henry, T., Cant, R., Porter, J., Missen, K., Kinsman, L., Endacott, R., Scholes, J., 2011a. Managing deteriorating patients: registered nurses' performance in a simulated setting. Open Nurs. J. 5, 120–126. http://dx.doi.org/10.2174/1874434 6011050100120. Cooper, S., M.-H., T., Cant, R., et al., 2011b. Managing deteriorating patients: registered nurses' performance in a simulated setting. Sim Health 2011 — Patient Centered Simulation. Hilton Hotel, Sydney, pp. 12–15 (Sept 2011). Cooper, S., Beauchamp, A., Bogossian, F., Bucknall, T., Cant, R., Devries, B., Endacott, R., Forbes, H., Hill, R., Kinsman, L., Kain, V.J., McKenna, L., Porter, J., Phillips, N., Young, S., 2012. Managing patient deterioration: a protocol for enhancing undergraduate nursing students' competence through web-based simulation and feedback techniques. BMC Nurs. 11 (1), 18. DeVita, M.A., Smith, G.B., Adam, S.K., Adams-Pizarro, I., Buist, M., Bellomo, R., Bonello, R., Cerchiari, E., Farlow, B., Goldsmith, D., Haskell, H., Hillman, K., Howell, M., Hravnak, M., Hunt, E.A., Hvarfner, A., Kellett, J., Lighthall, G.K., Lippert, A., Lippert, F.K., Mahroof, R., Myers, J.S., Rosen, M., Reynolds, S., Rotondi, A., Rubulotta, F., Winters, B., 2010. “Identifying the hospitalised patient in crisis” — a consensus conference on the afferent limb of rapid response systems. Resuscitation 81 (4), 375–382. DiGiuseppe, D.L., Aron, D.C., Ranbom, L., Harper, D.L., Rosenthal, G.E., 2002. Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern. Child Health J. 6 (3), 169–179. Endacott, R., Jevon, P., Cooper, S., 2009. Clinical Nursing Skills: Core And Advanced. Oxford University Press, Oxford. Endacott, R., Bogossian, F., Cooper, S., Forbes, H., Kain, V., Young, S., Porter, J., 2013w. Leadership and teamwork in medical emergencies: performance of nursing students and Registered nurses in simulated patient scenarios. Simul. Healthc. (in review). Ericcson, K., 2004. Deliberate practice and the acquisition and maintenance of expert performance in medical and related domains. Acad. Med. 79 (10), S70–S81. Finnigan, M.A., Marshall, S.D., Flanagan, B.T., 2010. ISBAR for clear communication: one hospital's experience spreading the message. Aust. Health Rev. 34 (4), 400–404. Fuhrmann, L., Lippert, A., Perner, A., Ostergaard, D., 2008. Incidence, staff awareness and mortality of patients at risk on general wards. Resuscitation 77 (3), 325–330. Greenfield, D., Nugus, P., Travaglia, J., et al., 2011. Auditing an organisation's interprofessional learning and interprofessional practice; the interprofessional praxis audit framework. J. Interprof. Care 24, 436–449. Harper, D., 1994. On the Authority of the Image: Visual Methods at the Crossroads. Sage, Thousand Oaks 403–412. IBM Corporation, Released, 2011. IBM SPSS Statistics for Windows. IBM Corp., Armonk, New York. McKay, A., Walkerb, S., Brettc, S., Vincent, C., Sevdalis, N., 2012. Team performance in resuscitation teams: comparison and critique of two recently developed scoring tools. Resuscitation 83, 1478–1483. McKenna, L., Missen, K., Cooper, S., Bogossian, F., Cant, R., 2013w. Situation awareness in undergraduate students during a simulated patient scenario. Nurse Educ. Today (in review). Ricker, K., 2006. Setting cut scores: a critical review of the Angoff and Modified Angoff methods. Alberta J. Educ. Res. 52 (1), 53–63. Tsang, P., Vidulich, M., 2006. Mental workload and situation awareness [chapter 9], In: Salvendy, G. (Ed.), Handbook of Human Factors and Ergonomics, Third ed. John Wiley & Sons, Inc., New York, p. pp. 243. Wollaston, A., Fahey, P., McKay, M., Hegney, D., Miller, P., Wollaston, J., 2004. Reliability and validity of the Toowoomba adult trauma triage tool: a Queensland, Australia study. Accid. Emerg. Nurs. 12 (4), 230–237.
