531458 research-article2014
WJNXXX10.1177/0193945914531458Western Journal of Nursing ResearchKalisch et al.
Intervention Studies
An Intervention to Improve Nursing Teamwork Using Virtual Simulation
Western Journal of Nursing Research 2015, Vol. 37(2) 164–179 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0193945914531458 wjn.sagepub.com
Beatrice J. Kalisch1, Michelle Aebersold1, Margaret McLaughlin1, Dana Tschannen1, and Sarah Lane1
Abstract The purpose of this study was to test the use of virtual simulation to improve teamwork among nursing staff. Using a quasi-experimental design, nursing staff (n = 43) from one patient care unit participated in a 1-hr session, which focused on common nursing teamwork problems. The overall mean teamwork scores improved from pre- (M = 3.25, SD = 0.58) to postintervention (M = 3.49, SD = 0.67, p < .012). The intervention also had large (0.60 ≤ d ≤ 0.97) and significant effects on the measures of three teamwork subscales (i.e., trust, team orientation, and backup). Keywords systems/management/leadership, nurses as subjects, nurses, nurses as subjects, nursing education, nurses as subjects, statistical analysis, methods, acute care, location of care Although larger investigations comparing this approach to more traditional training are needed, this study demonstrates the feasibility of implementing virtual simulation training with nursing staff members. 1University
of Michigan, Ann Arbor, USA
Corresponding Author: Beatrice J. Kalisch, Titus Professor and Director, Nursing Business and Health Systems, University of Michigan, School of Nursing, 400 N. Ingalls Street, Ann Arbor, MI 48109, USA. Email: [email protected]
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
165
Kalisch et al.
Nursing teams are essential to high-quality nursing care in inpatient hospital settings. No one nurse has all of the skills and knowledge, not to mention time and availability, to meet all of the needs of a patient or a group of patients. By its very nature, nursing care requires a team of nursing staff members (B. Kalisch & Schoville, 2012). There are an estimated 50,000 inpatient nursing teams (Registered Nurses [RNs], Licensed Practical Nurses [LPNs], and nursing assistants [NAs], working on any given unit) functioning in the United States alone, highlighting the major influence these health care providers have on the overall quality, safety and cost of patient care. Effective teamwork in nursing makes a significant contribution to improving health care quality through avoidance of errors (Catchpole et al., 2007; Pronovost et al., 2006), decreased missed nursing care (B. J. Kalisch & Lee, 2010), reductions in procedure time (Wiegmann, ElBardissi, Dearani, Daly, & Sundt, 2007), and improved communication among patients, families, and providers (Agarwal, Sands, & Schneider, 2010). Better teamwork in the intensive care unit was found to be related to lower patient mortality rates (Wheelan, Burchill, & Tilin, 2003). Brewer showed that a group-type hospital culture predicted fewer patient falls with injury (Brewer, 2006). Furthermore, improved teamwork has also been associated with greater job satisfaction (Amos, Hu, & Herrick, 2005; Chang, Ma, Chiu, Lin, & Lee, 2009; Collette, 2004; B. J. Kalisch, Lee, & Rochman, 2010; Rafferty, Ball, & Aiken, 2001) and decreased turnover (Blegen, Vaughn, & Vojir, 2008). However, nursing staff have not routinely received formalized teamwork training either in their basic professional preparation or on the job (Rosen et al., 2008). The need for teamwork training has been increasingly recognized (Corrigan, Kohn, & Donaldson, 2000; “Safe Practices for Better Healthcare–2010 Update: A Consensus Report,” 2010). Yet, there has been scant research on methods of teaching and fostering teamwork with nursing staff (B. J. Kalisch & Lee, 2009; B. J. Kalisch, Weaver, & Salas, 2009). The purpose of this study was to determine the feasibility and effectiveness of a newly developed virtual intervention for improving teamwork among nursing staff.
Approaches to Teamwork Training in Nursing A qualitative study was conducted applying a theoretically based model of teamwork to determine relevant team processes among nurses. Nurses from five patient care units participated in focus groups, describing team processes in their daily work. Responses were analyzed using the Salas framework (Figure 1) to develop a concrete conceptualization of teamwork within
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
166
Western Journal of Nursing Research 37(2)
Figure 1. Conceptual framework (Burke, Sims, & Salas, 2005).
nursing teams (Burke, Sims, & Salas, 2005). Results support the framework as a relevant and appropriate means for describing teamwork among nurses (B. J. Kalisch et al., 2009). Following up on this qualitative work, a quantitative tool to measure nursing teamwork in inpatient settings, the Nursing Teamwork Survey (NTS), was developed and tested (B. J. Kalisch, Lee, & Salas, 2010). The NTS was administered to nursing staff on 52 patient care units in five hospitals (77% RNs and LPNs, 12% NAs, and 8% unit secretaries). Teamwork varied considerably across units and service types, with the highest scores (e.g., greater teamwork) occurring in pediatrics and maternity and lower scores on the medical–surgical and emergency units. Staff working 8- or 10-hr shifts (as opposed to 12 hr), part-time staff (as opposed to full time), and those working on night shift had the highest teamwork scores. The highest teamwork scores were associated with little or no overtime (B. J. Kalisch & Lee, 2009). Another study showed that the level of nursing teamwork on inpatient acute care hospital patient units affects the amount of missed nursing care (B. J. Kalisch & Lee, 2010). Specifically, controlling for occupation of staff
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
167
Kalisch et al.
members (e.g., RN, LPN, NA) and staff characteristics (e.g., education, shift worked, experience), teamwork alone accounted for 11% of missed nursing care. When teamwork was stronger, there was less missed nursing care. Following up on a study of the amount and type of missed nursing care on 110 patient care units, the 5 patient care units with the most missed nursing care and the 5 with the least missed nursing care were selected for a qualitative study. Focus groups were conducted with the staff on these 10 units (located in five hospitals) to determine how these units differed. The key difference among the units was that the least missed units reported and described a much higher level of teamwork (B. J. Kalisch, Gosselin, & Choi, 2012). Studies to test interventions that would increase teamwork among nursing staff are limited. One study that tested an intervention (a combination of training, an engaged guiding team, and coaching) to improve teamwork resulted in a significant decrease in patient falls, staff vacancy, and turnover rates, and a significant rise in staff evaluations of the level of teamwork (B. J. Kalisch, Curley, & Stefanov, 2007). Although the intervention yielded promising and important results, it involved considerable time and the high use of resources (i.e., staff time). For widespread implementation, a shorter, more economical intervention utilizing less staff time was felt to be needed. Simulation is a technique that is gaining use in both academic and health care settings to train students and practicing staff. High-fidelity simulation has been a successful educational tool for both skill development and knowledge transfer from the simulated environment to the actual clinical environment (Day, Iles, & Griffiths, 2009; Maithel et al., 2006; Marnie et al., 2006; Tschannen, Aebersold, McLaughlin, Bowen, & Fairchild, 2012; Van Sickle, McClusky, Gallagher, & Smith, 2005; Wayne et al., 2008). Various simulation techniques have been used to specifically target team training. The results of one study showed that obstetric teams participating in a team training using high-fidelity simulation demonstrated a significantly higher teamwork score than the intervention group (Fransen et al., 2012). In another study, a modified TeamSTEPPS training program that included simulation, resulted in an increase in teamwork knowledge and attitudes and participants were able to correctly identify effective team skills. (Robertson et al., 2010) Successive exposure to team training simulations has been found to also increase teamwork skills (Falcone et al., 2008; Messmer, 2008). Effective team training using simulation is also found in military combat trauma (Holcomb et al., 2002) and trauma teams in emergency departments (Shapiro et al., 2004). Training in virtual environments is emerging as a novel approach to various types of skill and knowledge-based education programs. Studies have tested multi-user virtual environments (MUVE) such as Second Life™ (SL), which is an open access virtual environment created by Linden Labs. Learning
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
168
Western Journal of Nursing Research 37(2)
outcomes were evaluated in a study comparing didactic lecture and virtual patients. The study found that students exposed to the virtual training environment (SL) demonstrated significantly higher levels of knowledge acquisition and knowledge application than those students who received just the lecture. Knowledge acquisition was measured through a 10-item multiple choice test and knowledge application was measured through an objective structured clinical examination. Both were based on course content (LeFlore et al., 2012). Aebersold, Tschannen, & Bathish (2012) found that students who participated in virtual simulations targeting teamwork behaviors in an on-line virtual environment demonstrated significantly higher skills in communication and professional behavior after successive virtual simulations. Another team found students who participated in a series of three virtual scenarios (SL) over an academic year demonstrated higher overall performance in a mannequin-based simulation when compared with the control group (who received the usual classroom education; Tschannen et al., 2012). Kidd and colleagues discovered that students in a mental health undergraduate nursing course rated the virtual environment (SL) as a moderately effective teaching strategy (Kidd, Knisley, & Morgan, 2012). In summary, approaches to teamwork training have been effective but tend to involve considerable time and resources (B. J. Kalisch et al., 2007). Shorter training programs that have involved mannequin-based or high-fidelity simulation strategies have also been effective in improving teamwork but they require access to a simulation center. Virtual simulation such as SL has the potential to effectively deliver education to nursing staff as they can participate from off-site provided, they have access to a computer and internet connection. There is a cost involved in setting up the virtual environment if using SL. This cost involves the purchase and maintenance of “land” or space in SL to set up the virtual hospital environment. The cost to setting up the virtual hospital can be done in a less expensive mode if using pre-built items that can be purchased for a few dollars. These items will not be interactive but will allow the hospital environment to be set up to provide the setting to run the simulations. This option becomes more cost-effective if access to SL exists to the group planning the learning experience. There are also places available in SL for “rent” to groups wanting to use the area to run simulations. Alternatively, companies can be hired to develop the environment, which is more expensive but still less costly that setting up a physical simulation center. This type of distance learning is an attractive option for training in non-technical skills such as teamwork. The conceptual framework for this study was Salas and colleague’s teamwork model (Figure 1). The Salas Framework specifies five core components
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
169
Kalisch et al.
of teamwork: (a) team leadership (i.e., structure, direction, and support provided by both the formal leader and/or by other members of the team), (b) collective orientation (i.e., cohesiveness, individuals see the team’s success as taking precedence over individual needs and performance), (c) mutual performance monitoring (i.e., observation and awareness of team members, understanding team roles), (d) backup behavior (i.e., helping one another with their tasks and responsibilities), and (e) adaptability (i.e., ability to adjust strategies and resource allocation based on information gathered from the environment). In terms of relationships, the framework posits that leadership directly affects orientation, performance monitoring, and backup behavior. Both orientation and backup behavior influence performance monitoring. In turn, performance monitoring and backup generate adaptability (Burke et al., 2005). These relationships are fostered by three coordinating mechanisms according to the framework: (a) shared mental models (SMMs; that is, mutual conceptualizations of the task, roles, strengths/weaknesses, processes, and strategy necessary to attain interdependent goal), (b) closed-loop communication (i.e., active information exchange in which the receiver verifies receipt and the sender verifies intended message was received), and (c) mutual trust (i.e., shared perception that members will perform actions necessary to reach interdependent goals and act in the interest of the team). As previously indicated, a study on five units substantiated the applicability of the Salas model to nursing teamwork on inpatient units (B. J. Kalisch et al., 2009).
Research Questions The research questions for this study are as follows: Research Questions 1: Does an intervention to increase nursing teamwork using virtual simulation result in improved teamwork? Research Questions 2: Does an intervention to increase teamwork result in a higher level of knowledge about teamwork? Research Questions 3: Does the level of computer and/or virtual environment experience influence the effectiveness of the intervention?
Method Design This study is a quasi-experimental design. The aim was to test a virtual simulation method of increasing teamwork among the nursing staff on inpatient hospital units.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
170
Western Journal of Nursing Research 37(2)
Sample and Setting The setting for the study was a medical–surgical patient care unit in an academic health center hospital in the U.S. Midwest. After Institutional Review Board (IRB) approval, nursing staff who provide direct care to patients were invited to be a part of this study. Of the 80 nursing staff members on the unit (RNs and NAs), 54% (n = 43) agreed to participate in the study. Of the 43 staff members in the study, 88% were RNs and 12% were NAs. No LPNs were employed on the unit.
Intervention The virtual simulation environment we used was SL which is a 3D MUVE where each participant assumes a “real person,” using am avatar. Prior to this study, an eight-bed virtual hospital in SL had been developed. The virtual unit has four semi-private patient care rooms, a central nursing station, and a conference room. The trainees were assigned to assume one of the four avatars (roles) set up for the training. If four staff were not available for any given training session, one of the trainers assumed the role of an avatar. In some cases, the trainers deliberately played a role to create a specific scenario. Each patient room in SL had beds, medical equipment, and objects the avatars interacted with (e.g., patients, assessment tools, etc.) during the simulation scenarios. The intervention for this study involved two steps: (a) the provision of podcasts on the elements of teamwork and an introduction to SL and how to navigate the virtual world, which were placed on the unit’s computers for the staff to view on their own and (b) a 1-hr virtual simulation in which participants were exposed to three scenarios with debriefing sessions at the end of each. The SL podcast, 5 min in length, was designed to give participants an overview of the SL environment and to demonstrate skills needed. The podcast included a virtual tour of the training environment (virtual hospital) and displayed navigation (walk, interact with objects, sit) and communication skills (chat and voice) the participants would be expected to use. The teamwork podcast, 30 min in length, defined the eight teamwork behaviors (from the Salas model) and gave examples from situations that occur on most patient care units (e.g., an RN searches all over the unit for an NA to put a patient on a bedpan when she could have done it herself much faster; staff not answering the patient’s call light because it is not their patient, etc.). As indicated above, the 1-hr teamwork training sessions involved three role-plays based on typical teamwork problems that occur in nursing teams (B. J. Kalisch et al., 2009). Each scenario had roles for three RNs and one
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
171
Kalisch et al.
NA. Specifically, the scenarios focused on the following situations: a challenging assignment requiring teamwork and delegation, over-delegation to an NA, and a miscommunication between an RN and an NA (Table 1). Each scenario was designed to have the nursing staff members play a particular role (e.g., RN, NA, charge nurse, nurse manager, etc.). After the role-play, the trainees analyzed whether the eight teamwork behaviors in the Salas model were present or absent. As feedback from one staff member to another is required to resolve the issue in all three conflicts, the basic approach to effective feedback was presented and reinforced throughout the training sessions. The final aspect of each of the debriefing sessions (for each of the three scenarios) was to redo or explain how it might be played out in a more effective way. The nursing staff were scheduled in training groups during their work shifts around the clock. Coverage for patient care for those staff members in training was provided by the nurse managers, nurse educator, and charge nurse. If the shift was especially busy, the training was rescheduled. This happened an estimated 10% of the time. Computers were set up ahead of time with the avatars to be used by the trainees in their role-plays in SL. Although the nursing staff and trainers wore headsets, which were required to permit the use of voice to “talk” in SL during the virtual scenarios, trainees had to be placed in different conference rooms and offices. Otherwise, they would be distracted by the others talking. Each scenario ran 10 to 15 min followed by a 5 to 10 min debriefing. During the training session, participants were given specific instructions for their role through the use of cue cards (given prior to the scenario) and notecards passed in SL during the scenario. The notecards contained a description of the role the participant was playing. The cue cards contained information about the patient the participant was interacting with in SL. The participants were then instructed to play out the scenario as they would if it were occurring in real life on their unit. During the debriefing conducted by the trainers, participants were asked to identify the presence or absence of teamwork behaviors (e.g., backup, leadership, shared mental models, etc.). They were also instructed as to how to give effective feedback, which was needed in all three scenarios. Finally, they were asked what they might do differently in the future as a result of the simulated learning experience.
Measures The NTS and a Teamwork Knowledge Survey were used to measure teamwork. These were administered to the study participants 3 weeks before the intervention and 3 weeks after all of the staff had undergone the training. In
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
172
Western Journal of Nursing Research 37(2)
Table 1. Virtual Scenarios Utilized in Training. Scenario
Teamwork Elements
Challenging Team leadership, assignment mutual performance requiring teamwork monitoring, backup and delegation behavior, adaptability, team orientation, shared mental models.
Over-delegation to nursing assistant
Team leadership, mutual performance monitoring, backup behavior, adaptability, team orientation, shared mental models.
RN and NA miscommunication and follow up
Mutual performance monitoring, shared mental model, closedloop communication interpersonal relationships.
Scenario Nurses begin the day by getting report on the unit and then they are directed to go and visit their patients. The charge nurse is off the unit at a meeting. When Nurse 1 arrives in her patient’s room the patient in bed 1 states “I think my roommate just fell out of bed.” The patient in bed 1 is also complaining of severe dizziness (rule out gastrointestinal bleed). RNs and NAs begin the day by getting report on the unit and then they are directed to go and visit their patients. When the NA arrives the patient (r/o myocardial infarction) in Bed 3 is complaining of severe chest pain. When the NA tries to relay this to the RN caring for the patient (played by the trainer) she is told, “I am not worried about it.” After receiving report, the nurse responds to a patient call light. The patient relays to the nurse that yesterday she did not receive aspects of nursing care (no glucose check, did not walk). The NA that is assigned to care for her was the same one as yesterday and she (the patient) does not want that NA taking care of her again.
Note. RN = Registered Nurses; NA = nursing assistants.
addition, the participants were asked five questions about computer proficiency and experience using computer simulation and virtual worlds in the pre-intervention data collection.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
173
Kalisch et al.
The NTS Nursing teamwork was measured by the NTS, a 33-item questionnaire with a 5-point Likert-type scale from rarely (1) to always (5). Total score ranges from 33 to 165 with higher scores indicating higher nursing teamwork. The NTS consists of five subscales: (a) trust (i.e., shared perception that members will perform actions necessary to reach interdependent goals and act in the interest of the team), (b) team orientation (i.e., cohesiveness, the extent to which individuals see the team’s success taking precedence over individual needs and performance), (c) backup (i.e., helping one another with their tasks and responsibilities), (d) shared mental model (i.e., mutual conceptualizations of the task, roles, strengths/weaknesses, and processes and strategy necessary to attain interdependent goals), and (e) team leadership (i.e., structure, direction, and support). Internal reliability (Cronbach’s α = .94), test–retest reliability (r = .92), and validity (concurrent, convergent, and contrast validity) of the NTS have been reported elsewhere (B. J. Kalisch, Lee, & Salas, 2010).
Teamwork Knowledge Test Knowledge about teamwork was tested with an eight-item test, in which the percent of correct responses for each was calculated. The questions for this survey were drawn from those contained in the training manual for TeamSTEPPS (King et al., 2008). TeamSTEPPS is based on the Salas model of teamwork, which was used as the conceptual framework for this study. Computer and virtual world experience. Computer experience was measured via questionnaire on a scale of 1 (very low) to 5 (very high). Respondents were asked whether they had heard of virtual worlds. They were then asked whether they had participated in any computer virtual worlds.
Procedures Each person participating in the study completed a consent form. All staff on the unit were given a survey packet both before and after the intervention containing a large candy bar as an incentive to participate. To maintain anonymity and consistency in comparing pre- and post-training questionnaires, each respondent provided an anonymous identifier with birth month, mother’s first name, and mother’s birth month. Completed surveys were deposited into a locked box on the unit.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
174
Western Journal of Nursing Research 37(2)
Table 2. Teamwork Scores Between Pre- and Post-Intervention. Pre-Intervention, Post-Intervention, M ± SD M ± SD Teamwork overall Trust Team orientation Backup Shared mental model Team leadership
3.25 ± 0.58 2.94 ± 0.64 2.80 ± 0.61 3.18 ± 0.71 3.67 ± 0.62 3.19 ± 0.92
3.49 ± 0.67 3.22 ± 0.73 3.13 ± 0.73 3.38 ± 0.72 3.93 ± 0.48 3.58 ± 0.78
p Effect Size
t −3.07 −2.24 −3.45 −2.22 −1.73 −2.11
.012 .042 .004 .045 .109 .052
0.96 0.96 0.97 0.60 0.49 0.53
Data Analysis Sixteen participants completed both the pre- and the post-test. Data were analyzed by descriptive statistics (means, standard deviation, and percentages) and paired t test. Effect sizes were calculated to use Cohen’s d as the difference between the means for two groups divided by the standard deviation of either group (Cohen, 1988). The PASW Statistics software (version 18.0; SPSS, Inc., Chicago, IL) was used for the analyses.
Results A total of 43 nursing staff RNs (88.4%) and NAs (11.6%) participated in this study. Among the participants, 81.4% were female and 41.9% had a bachelor’s degree or higher. Of the staff that participated, 60% rated their overall computer proficiency as high or very high and the rest of participants (40%) rated their computer proficiency as very low, low, or medium. Completing both pre- and post-intervention were 16 nursing staff members (37.2%).
Teamwork Behaviors As seen in Table 2, the overall mean scores of teamwork at pre-intervention was 3.25 (SD = 0.58) and at post-intervention it was 3.49 (0.67). More explicitly, the teamwork overall score at post-intervention was significantly higher than those at pre-test (t = −3.07, p < .012). According to Cohen’s effect size (Cohen, 1988), the intervention had a large (d = .96) and statistically significant effect on the measure of overall teamwork. Three out of five teamwork subscale scores (trust, team orientation, and backup) were significantly higher from pre- to post-intervention. The intervention also had large (0.60 ≤ d ≤ 0.97) and significant effects on the three teamwork subscales.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
175
Kalisch et al.
Table 3. Changes in Teamwork Behaviors and Knowledge by Level of Computer Proficiency. Computer Proficiency Very Low to Medium, High to Very High, M ± SD M ± SD
Teamwork overall Trust Team orientation Backup SMM Team leadership Teamwork knowledge
3.07 ± 0.50 3.07 ± 0.49 2.91 ± 0.51 3.28 ± 0.67 3.86 ± 0.61 3.58 ± 0.66 6.88 ± 1.25
3.46 ± 0.75 3.34 ± 0.88 3.25 ± 0.92 3.40 ± 0.76 3.84 ± 0.53 3.59 ± 0.91 6.75 ± 1.04
t
p
−0.27 −0.75 −0.95 −0.34 0.06 −0.03 0.22
.792 .466 .356 .737 .951 .979 .830
Note. SMM = shared mental model.
Teamwork Knowledge Scores Teamwork knowledge scores were not significantly different between preand post-intervention (t = −1.08, p < .301).
Computer Proficiency and Previous Virtual World Experience Nursing staff reporting higher levels of computer experience did not score higher on teamwork behaviors and knowledge after the intervention than did those with less computer experience (Table 3). We also found that experience with virtual worlds did not influence overall teamwork scores, the five teamwork subscales, or the level of teamwork knowledge. Of the five teamwork subscales, “team orientation” and “trust” scores were slightly higher for those who had participated in virtual world simulations, but those differences were not statistically significant (t = 1.27, p = .225; t = 0.29, p = .775, respectively). Higher scores on backup behavior (t = −0.35, p = .733), shared mental models (t = −0.58, p = .569), team leadership (t = −0.39, p = .702), and teamwork knowledge (t = −0.30, p = .766) for those with no experience with virtual world simulations did not reach statistical significance.
Discussion The need for effective teamwork in nursing has been demonstrated. Yet, capacity building for teamwork skills has been inadequate. The purpose of this study was to determine the feasibility of a virtually based intervention for
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
176
Western Journal of Nursing Research 37(2)
improving teamwork among nursing staff. Through the use of virtual simulations, nurses were given opportunities to practice skills related to teamwork and deal with difficult situations (e.g., when team members do not function effectively as team members). The basic teamwork behaviors (team orientation, backup, etc.) were reviewed after each scenario, asking the trainees to identify whether this element of teamwork was present or not. This study was effective because it used scenarios that reflect the everyday experiences of nursing staff working together as a team (e.g., RN and NA miscommunication and follow up; over-delegation to the NA). A comparative study with other approaches (e.g., traditional classroom, face-to-face role-playing, highfidelity simulation, etc.), to enhancing nursing teamwork is needed to fully evaluate the effectiveness of virtual simulation. Findings from the study showed that overall teamwork scores did improve pre- and post-intervention. In addition, scores on the subscales of trust, team orientation, backup, and team leadership significantly improved after participation in the virtual simulations. Despite improvement in teamwork behaviors, actual knowledge of teamwork did not improve significantly from pre- to post-intervention. In this study, nurses were not able to articulate team behaviors, but they were able to practice them, as noted by improved teamwork behavior scores post-implementation. Other notable findings included the lack of significance between changes in teamwork behaviors and knowledge and computer proficiency and participation in virtual worlds. All of the nurses were able to participate in the intervention with very limited training in virtual environments. This finding suggests that a staff member would be able to effectively participate in virtual, computer-based training, despite his or her level of computer experience and exposure to virtual training. There are limitations of this study that may have affected the results. Specifically, in a single-group design, this study took place on one unit with a small sample of nursing staff; thus generalizability is limited. The intervention should be retested with a larger sample of patient care unit nursing teams. Another limitation was the participation rate in the intervention. Not all of the staff participated, thus some did not have the opportunity to practice effective teamwork behaviors. Also, there was a large attrition rate from pre- to posttest response levels. Attrition, common in pre- and post-studies, was compounded in this study as it coincided with nursing staff vacations. This study described a new approach to team training using the virtual environment. This approach allows for greater access, as participants only need to be present “virtually.” In addition, the virtual environment provides a safe environment to practice non-technical skills such as teamwork. In the
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
177
Kalisch et al.
nursing environment, it is difficult to get staff away from work for training. Use of virtual simulations to train the teams may be a suitable alternative. Use of the virtual environment has the potential to disseminate information/ evidence rapidly to participants, thus breaking down barriers of the traditional classroom or training facility. Further work is needed to explore this new innovative approach to training nursing staff members in teamwork. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded in part by the Blue Cross Blue Shield Foundation of Michigan.
References Aebersold, M., Tschannen, D., & Bathish, M. (2012). Innovative simulation strategies in education. Nursing Research and Practice, 2012, Article 765212. Agarwal, R., Sands, D. Z., & Schneider, J. D. (2010). Quantifying the economic impact of communication inefficiencies in U.S. hospitals. Journal of Healthcare Management, 55, 265-281; discussion 281-262. Amos, M. A., Hu, J., & Herrick, C. A. (2005). The impact of team building on communication and job satisfaction of nursing staff. Journal for Nurses in Professional Development, 21, 10-16. Blegen, M. A., Vaughn, T., & Vojir, C. P. (2008). Nurse staffing levels: Impact of organizational characteristics and registered nurse supply. Health Services Research, 43(1, Pt. 1), 154-173. Brewer, B. B. (2006). Relationships among teams, culture, safety, and cost outcomes. Western Journal of Nursing Research, 28, 641-653. Burke, C. S., Sims, D. E., & Salas, E. (2005). Is there a “big five” in teamwork? Small Group Research, 36, 555-599. Catchpole, K. R., Giddings, A. E., Wilkinson, M., Hirst, G., Dale, T., & de Leval, M. R. (2007). Improving patient safety by identifying latent failures in successful operations. Surgery, 142, 102-110. doi:10.1016/j.surg.2007.01.033 Chang, W.-Y., Ma, J.-C., Chiu, H.-T., Lin, K.-C., & Lee, P.-H. (2009). Job satisfaction and perceptions of quality of patient care, collaboration and teamwork in acute care hospitals. Journal of Advanced Nursing, 65, 1946-1945. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Collette, J. E. (2004). Retention of nursing staff—A team-based approach. Australian Health Review, 28, 349-356.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
178
Western Journal of Nursing Research 37(2)
Corrigan, J. M., Kohn, L. T., & Donaldson, M. S. (2000). To err is human. Washington, DC: National Academies Press. Day, T., Iles, N., & Griffiths, P. (2009). Effect of performance feedback on tracheal suctioning knowledge and skills: Randomized controlled trial. Journal of Advanced Nursing, 65, 1423-1431. Falcone, J. R. A., Daugherty, M., Schweer, L., Patterson, M., Brown, R. L., & Garcia, V. F. (2008). Multidisciplinary pediatric trauma team training using high-fidelity trauma simulation. Journal of Pediatric Surgery, 43, 1065-1071. Fransen, A. F., van de Ven, J., Merién, A. E. R., de Wit-Zuurendonk, L. D., Houterman, S., Mol, B. W., & Oei, S. G. (2012). Effect of obstetric team training on team performance and medical technical skills: A randomised controlled trial. BJOG, 119, 1387-1393. Holcomb, J. B., Dumire, R. D., Crommett, J. W., Stamateris, C. E., Fagert, M. A., Cleveland, J. A., . . . Mattox, K. L. (2002). Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. Journal of Trauma, 52, 1078-1085; discussion 1085-1076. Kalisch, B., & Schoville, R. (2012). It takes a team. American Journal of Nursing, 112(10), 50-54. doi:10.1097/01.NAJ.0000421024.08037.0d Kalisch, B. J., Curley, M., & Stefanov, S. (2007). An intervention to enhance nursing staff teamwork and engagement. Journal of Nursing Administration, 37, 77-84. Kalisch, B. J., Gosselin, K., & Choi, S. H. (2012). A comparison of patient care units with high versus low levels of missed nursing care. Health Care Management Review, 37, 320-328. Kalisch, B. J., & Lee, H. (2009). Nursing teamwork, staff characteristics, work schedules, and staffing. Health Care Management Review, 34, 323-333. Kalisch, B. J., & Lee, K. H. (2010). The impact of teamwork on missed nursing care. Nursing Outlook, 58, 233-241. doi:10.1016/j.outlook.2010.06.004 Kalisch, B. J., Lee, H., & Rochman, M. (2010). Nursing staff teamwork and job satisfaction. Journal of Nursing Management, 18, 938-947. Kalisch, B. J., Lee, H., & Salas, E. (2010). The development and testing of the nursing teamwork survey. Nursing Research, 59, 42-50. Kalisch, B. J., Weaver, S. J., & Salas, E. (2009). What does nursing teamwork look like? A qualitative study. Journal of Nursing Care Quality, 24, 298-307. Kidd, L. I., Knisley, S. J., & Morgan, K. I. (2012). Effectiveness of a Second Life(®) simulation as a teaching strategy for undergraduate mental health nursing students. Journal of Psychosocial Nursing and Mental Health Services, 50(7), 28-37. King, H., Battles, J., Baker, D., Alonso, A., Salas, E., Webster, J., . . . Salisbury, M. L. (2008). TeamSTEPPS™: Team strategies and tools to enhance performance and patient safety. In K. Henriksen, J. B. Battles, M. A. Keyes, & M. L. Grady (Eds.), Advances in patient safety: New directions and alternative approaches (Vol. 3, Performance and tools). Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from http://www.ncbi.nlm.nih.gov/books/NBK43686/ LeFlore, J. L., Anderson, M., Zielke, M. A., Nelson, K. A., Thomas, P. E., Hardee, G., & John, L. D. (2012). Can a virtual patient trainer teach student nurses how
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
179
Kalisch et al.
to save lives-teaching nursing students about pediatric respiratory diseases. Simulation in Healthcare, 7, 10-17. Maithel, S., Sierra, R., Korndorffer, J., Neumann, P., Dawson, S., Callery, M., . . . Scott, D. (2006). Construct and face validity of MIST-VR, Endotower, and CELTS. Surgical Endoscopy, 20, 104-112. Marnie, E. R., Mari, A., Vanessa, R., Mueck, J. T., Roslyn, F. S., & Paul, H. M. (2006). Achieving housestaff competence in emergency airway management using scenario based simulation training. Chest, 129, 1453-1458. Messmer, P. R. (2008). Enhancing nurse-physician collaboration using pediatric simulation. Journal of Continuing Education in Nursing, 39, 319-327. Pronovost, P. J., Thompson, D. A., Holzmueller, C. G., Lubomski, L. H., Dorman, T., Dickman, F., . . . Morlock, L. L. (2006). Toward learning from patient safety reporting systems. Journal of Critical Care, 21, 305-315. doi:10.1016/j. jcrc.2006.07.001 Rafferty, A. M., Ball, J., & Aiken, L. H. (2001). Are teamwork and professional autonomy compatible, and do they result in improved hospital care? Quality in Health Care, 10(Suppl. 2), ii32-ii37. Robertson, B., Kaplan, B., Atallah, H., Higgins, M., Lewitt, M. J., & Ander, D. S. (2010). The use of simulation and a modified TeamSTEPPS curriculum for medical and nursing student team training. Simulation in Healthcare, 5, 332-337. Rosen, M. A., Salas, E., Wilson, K. A., King, H. B., Salisbury, M., Augenstein, J. S., . . . Birnbach, D. J. (2008). Measuring team performance in simulation-based training: Adopting best practices for healthcare. Simulation in Healthcare, 3, 33-41. Safe practices for better healthcare–2010 update: A consensus report. (2010). Washington, DC: National Quality Forum. Shapiro, M. J., Morey, J. C., Small, S. D., Langford, V., Kaylor, C. J., Jagminas, L., . . . Jay, G. D. (2004). Simulation based teamwork training for emergency department staff: Does it improve clinical team performance when added to an existing didactic teamwork curriculum? Quality and Safety in Health Care, 13, 417-421. Tschannen, D., Aebersold, M., McLaughlin, E., Bowen, J., & Fairchild, J. (2012). Use of virtual simulations for improving knowledge transfer among baccalaureate nursing students. Journal of Nursing Education and Practice, 2(3), 15-24. Van Sickle, K. R., McClusky, R. D. A., Gallagher, A. G., & Smith, C. D. (2005). Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surgical Endoscopy, 19, 1227-1231. Wayne, D. B., Didwania, A., Feinglass, J., Fudala, M. J., Barsuk, J. H., & McGaghie, W. C. (2008). Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: A case-control study. Chest, 133, 56-61. Wheelan, S. A., Burchill, C. N., & Tilin, F. (2003). The link between teamwork and patients’ outcomes in intensive care units. American Journal of Critical Care, 12, 527-534. Wiegmann, D. A., ElBardissi, A. W., Dearani, J. A., Daly, R. C., & Sundt, T. M., III. (2007). Disruptions in surgical flow and their relationship to surgical errors: An exploratory investigation. Surgery, 142, 658-665. doi:10.1016/j.surg.2007.07.034
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
WJNXXX10.1177/0193945914531458Western Journal of Nursing ResearchKalisch et al.
Intervention Studies
An Intervention to Improve Nursing Teamwork Using Virtual Simulation
Western Journal of Nursing Research 2015, Vol. 37(2) 164–179 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0193945914531458 wjn.sagepub.com
Beatrice J. Kalisch1, Michelle Aebersold1, Margaret McLaughlin1, Dana Tschannen1, and Sarah Lane1
Abstract The purpose of this study was to test the use of virtual simulation to improve teamwork among nursing staff. Using a quasi-experimental design, nursing staff (n = 43) from one patient care unit participated in a 1-hr session, which focused on common nursing teamwork problems. The overall mean teamwork scores improved from pre- (M = 3.25, SD = 0.58) to postintervention (M = 3.49, SD = 0.67, p < .012). The intervention also had large (0.60 ≤ d ≤ 0.97) and significant effects on the measures of three teamwork subscales (i.e., trust, team orientation, and backup). Keywords systems/management/leadership, nurses as subjects, nurses, nurses as subjects, nursing education, nurses as subjects, statistical analysis, methods, acute care, location of care Although larger investigations comparing this approach to more traditional training are needed, this study demonstrates the feasibility of implementing virtual simulation training with nursing staff members. 1University
of Michigan, Ann Arbor, USA
Corresponding Author: Beatrice J. Kalisch, Titus Professor and Director, Nursing Business and Health Systems, University of Michigan, School of Nursing, 400 N. Ingalls Street, Ann Arbor, MI 48109, USA. Email: [email protected]
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
165
Kalisch et al.
Nursing teams are essential to high-quality nursing care in inpatient hospital settings. No one nurse has all of the skills and knowledge, not to mention time and availability, to meet all of the needs of a patient or a group of patients. By its very nature, nursing care requires a team of nursing staff members (B. Kalisch & Schoville, 2012). There are an estimated 50,000 inpatient nursing teams (Registered Nurses [RNs], Licensed Practical Nurses [LPNs], and nursing assistants [NAs], working on any given unit) functioning in the United States alone, highlighting the major influence these health care providers have on the overall quality, safety and cost of patient care. Effective teamwork in nursing makes a significant contribution to improving health care quality through avoidance of errors (Catchpole et al., 2007; Pronovost et al., 2006), decreased missed nursing care (B. J. Kalisch & Lee, 2010), reductions in procedure time (Wiegmann, ElBardissi, Dearani, Daly, & Sundt, 2007), and improved communication among patients, families, and providers (Agarwal, Sands, & Schneider, 2010). Better teamwork in the intensive care unit was found to be related to lower patient mortality rates (Wheelan, Burchill, & Tilin, 2003). Brewer showed that a group-type hospital culture predicted fewer patient falls with injury (Brewer, 2006). Furthermore, improved teamwork has also been associated with greater job satisfaction (Amos, Hu, & Herrick, 2005; Chang, Ma, Chiu, Lin, & Lee, 2009; Collette, 2004; B. J. Kalisch, Lee, & Rochman, 2010; Rafferty, Ball, & Aiken, 2001) and decreased turnover (Blegen, Vaughn, & Vojir, 2008). However, nursing staff have not routinely received formalized teamwork training either in their basic professional preparation or on the job (Rosen et al., 2008). The need for teamwork training has been increasingly recognized (Corrigan, Kohn, & Donaldson, 2000; “Safe Practices for Better Healthcare–2010 Update: A Consensus Report,” 2010). Yet, there has been scant research on methods of teaching and fostering teamwork with nursing staff (B. J. Kalisch & Lee, 2009; B. J. Kalisch, Weaver, & Salas, 2009). The purpose of this study was to determine the feasibility and effectiveness of a newly developed virtual intervention for improving teamwork among nursing staff.
Approaches to Teamwork Training in Nursing A qualitative study was conducted applying a theoretically based model of teamwork to determine relevant team processes among nurses. Nurses from five patient care units participated in focus groups, describing team processes in their daily work. Responses were analyzed using the Salas framework (Figure 1) to develop a concrete conceptualization of teamwork within
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
166
Western Journal of Nursing Research 37(2)
Figure 1. Conceptual framework (Burke, Sims, & Salas, 2005).
nursing teams (Burke, Sims, & Salas, 2005). Results support the framework as a relevant and appropriate means for describing teamwork among nurses (B. J. Kalisch et al., 2009). Following up on this qualitative work, a quantitative tool to measure nursing teamwork in inpatient settings, the Nursing Teamwork Survey (NTS), was developed and tested (B. J. Kalisch, Lee, & Salas, 2010). The NTS was administered to nursing staff on 52 patient care units in five hospitals (77% RNs and LPNs, 12% NAs, and 8% unit secretaries). Teamwork varied considerably across units and service types, with the highest scores (e.g., greater teamwork) occurring in pediatrics and maternity and lower scores on the medical–surgical and emergency units. Staff working 8- or 10-hr shifts (as opposed to 12 hr), part-time staff (as opposed to full time), and those working on night shift had the highest teamwork scores. The highest teamwork scores were associated with little or no overtime (B. J. Kalisch & Lee, 2009). Another study showed that the level of nursing teamwork on inpatient acute care hospital patient units affects the amount of missed nursing care (B. J. Kalisch & Lee, 2010). Specifically, controlling for occupation of staff
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
167
Kalisch et al.
members (e.g., RN, LPN, NA) and staff characteristics (e.g., education, shift worked, experience), teamwork alone accounted for 11% of missed nursing care. When teamwork was stronger, there was less missed nursing care. Following up on a study of the amount and type of missed nursing care on 110 patient care units, the 5 patient care units with the most missed nursing care and the 5 with the least missed nursing care were selected for a qualitative study. Focus groups were conducted with the staff on these 10 units (located in five hospitals) to determine how these units differed. The key difference among the units was that the least missed units reported and described a much higher level of teamwork (B. J. Kalisch, Gosselin, & Choi, 2012). Studies to test interventions that would increase teamwork among nursing staff are limited. One study that tested an intervention (a combination of training, an engaged guiding team, and coaching) to improve teamwork resulted in a significant decrease in patient falls, staff vacancy, and turnover rates, and a significant rise in staff evaluations of the level of teamwork (B. J. Kalisch, Curley, & Stefanov, 2007). Although the intervention yielded promising and important results, it involved considerable time and the high use of resources (i.e., staff time). For widespread implementation, a shorter, more economical intervention utilizing less staff time was felt to be needed. Simulation is a technique that is gaining use in both academic and health care settings to train students and practicing staff. High-fidelity simulation has been a successful educational tool for both skill development and knowledge transfer from the simulated environment to the actual clinical environment (Day, Iles, & Griffiths, 2009; Maithel et al., 2006; Marnie et al., 2006; Tschannen, Aebersold, McLaughlin, Bowen, & Fairchild, 2012; Van Sickle, McClusky, Gallagher, & Smith, 2005; Wayne et al., 2008). Various simulation techniques have been used to specifically target team training. The results of one study showed that obstetric teams participating in a team training using high-fidelity simulation demonstrated a significantly higher teamwork score than the intervention group (Fransen et al., 2012). In another study, a modified TeamSTEPPS training program that included simulation, resulted in an increase in teamwork knowledge and attitudes and participants were able to correctly identify effective team skills. (Robertson et al., 2010) Successive exposure to team training simulations has been found to also increase teamwork skills (Falcone et al., 2008; Messmer, 2008). Effective team training using simulation is also found in military combat trauma (Holcomb et al., 2002) and trauma teams in emergency departments (Shapiro et al., 2004). Training in virtual environments is emerging as a novel approach to various types of skill and knowledge-based education programs. Studies have tested multi-user virtual environments (MUVE) such as Second Life™ (SL), which is an open access virtual environment created by Linden Labs. Learning
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
168
Western Journal of Nursing Research 37(2)
outcomes were evaluated in a study comparing didactic lecture and virtual patients. The study found that students exposed to the virtual training environment (SL) demonstrated significantly higher levels of knowledge acquisition and knowledge application than those students who received just the lecture. Knowledge acquisition was measured through a 10-item multiple choice test and knowledge application was measured through an objective structured clinical examination. Both were based on course content (LeFlore et al., 2012). Aebersold, Tschannen, & Bathish (2012) found that students who participated in virtual simulations targeting teamwork behaviors in an on-line virtual environment demonstrated significantly higher skills in communication and professional behavior after successive virtual simulations. Another team found students who participated in a series of three virtual scenarios (SL) over an academic year demonstrated higher overall performance in a mannequin-based simulation when compared with the control group (who received the usual classroom education; Tschannen et al., 2012). Kidd and colleagues discovered that students in a mental health undergraduate nursing course rated the virtual environment (SL) as a moderately effective teaching strategy (Kidd, Knisley, & Morgan, 2012). In summary, approaches to teamwork training have been effective but tend to involve considerable time and resources (B. J. Kalisch et al., 2007). Shorter training programs that have involved mannequin-based or high-fidelity simulation strategies have also been effective in improving teamwork but they require access to a simulation center. Virtual simulation such as SL has the potential to effectively deliver education to nursing staff as they can participate from off-site provided, they have access to a computer and internet connection. There is a cost involved in setting up the virtual environment if using SL. This cost involves the purchase and maintenance of “land” or space in SL to set up the virtual hospital environment. The cost to setting up the virtual hospital can be done in a less expensive mode if using pre-built items that can be purchased for a few dollars. These items will not be interactive but will allow the hospital environment to be set up to provide the setting to run the simulations. This option becomes more cost-effective if access to SL exists to the group planning the learning experience. There are also places available in SL for “rent” to groups wanting to use the area to run simulations. Alternatively, companies can be hired to develop the environment, which is more expensive but still less costly that setting up a physical simulation center. This type of distance learning is an attractive option for training in non-technical skills such as teamwork. The conceptual framework for this study was Salas and colleague’s teamwork model (Figure 1). The Salas Framework specifies five core components
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
169
Kalisch et al.
of teamwork: (a) team leadership (i.e., structure, direction, and support provided by both the formal leader and/or by other members of the team), (b) collective orientation (i.e., cohesiveness, individuals see the team’s success as taking precedence over individual needs and performance), (c) mutual performance monitoring (i.e., observation and awareness of team members, understanding team roles), (d) backup behavior (i.e., helping one another with their tasks and responsibilities), and (e) adaptability (i.e., ability to adjust strategies and resource allocation based on information gathered from the environment). In terms of relationships, the framework posits that leadership directly affects orientation, performance monitoring, and backup behavior. Both orientation and backup behavior influence performance monitoring. In turn, performance monitoring and backup generate adaptability (Burke et al., 2005). These relationships are fostered by three coordinating mechanisms according to the framework: (a) shared mental models (SMMs; that is, mutual conceptualizations of the task, roles, strengths/weaknesses, processes, and strategy necessary to attain interdependent goal), (b) closed-loop communication (i.e., active information exchange in which the receiver verifies receipt and the sender verifies intended message was received), and (c) mutual trust (i.e., shared perception that members will perform actions necessary to reach interdependent goals and act in the interest of the team). As previously indicated, a study on five units substantiated the applicability of the Salas model to nursing teamwork on inpatient units (B. J. Kalisch et al., 2009).
Research Questions The research questions for this study are as follows: Research Questions 1: Does an intervention to increase nursing teamwork using virtual simulation result in improved teamwork? Research Questions 2: Does an intervention to increase teamwork result in a higher level of knowledge about teamwork? Research Questions 3: Does the level of computer and/or virtual environment experience influence the effectiveness of the intervention?
Method Design This study is a quasi-experimental design. The aim was to test a virtual simulation method of increasing teamwork among the nursing staff on inpatient hospital units.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
170
Western Journal of Nursing Research 37(2)
Sample and Setting The setting for the study was a medical–surgical patient care unit in an academic health center hospital in the U.S. Midwest. After Institutional Review Board (IRB) approval, nursing staff who provide direct care to patients were invited to be a part of this study. Of the 80 nursing staff members on the unit (RNs and NAs), 54% (n = 43) agreed to participate in the study. Of the 43 staff members in the study, 88% were RNs and 12% were NAs. No LPNs were employed on the unit.
Intervention The virtual simulation environment we used was SL which is a 3D MUVE where each participant assumes a “real person,” using am avatar. Prior to this study, an eight-bed virtual hospital in SL had been developed. The virtual unit has four semi-private patient care rooms, a central nursing station, and a conference room. The trainees were assigned to assume one of the four avatars (roles) set up for the training. If four staff were not available for any given training session, one of the trainers assumed the role of an avatar. In some cases, the trainers deliberately played a role to create a specific scenario. Each patient room in SL had beds, medical equipment, and objects the avatars interacted with (e.g., patients, assessment tools, etc.) during the simulation scenarios. The intervention for this study involved two steps: (a) the provision of podcasts on the elements of teamwork and an introduction to SL and how to navigate the virtual world, which were placed on the unit’s computers for the staff to view on their own and (b) a 1-hr virtual simulation in which participants were exposed to three scenarios with debriefing sessions at the end of each. The SL podcast, 5 min in length, was designed to give participants an overview of the SL environment and to demonstrate skills needed. The podcast included a virtual tour of the training environment (virtual hospital) and displayed navigation (walk, interact with objects, sit) and communication skills (chat and voice) the participants would be expected to use. The teamwork podcast, 30 min in length, defined the eight teamwork behaviors (from the Salas model) and gave examples from situations that occur on most patient care units (e.g., an RN searches all over the unit for an NA to put a patient on a bedpan when she could have done it herself much faster; staff not answering the patient’s call light because it is not their patient, etc.). As indicated above, the 1-hr teamwork training sessions involved three role-plays based on typical teamwork problems that occur in nursing teams (B. J. Kalisch et al., 2009). Each scenario had roles for three RNs and one
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
171
Kalisch et al.
NA. Specifically, the scenarios focused on the following situations: a challenging assignment requiring teamwork and delegation, over-delegation to an NA, and a miscommunication between an RN and an NA (Table 1). Each scenario was designed to have the nursing staff members play a particular role (e.g., RN, NA, charge nurse, nurse manager, etc.). After the role-play, the trainees analyzed whether the eight teamwork behaviors in the Salas model were present or absent. As feedback from one staff member to another is required to resolve the issue in all three conflicts, the basic approach to effective feedback was presented and reinforced throughout the training sessions. The final aspect of each of the debriefing sessions (for each of the three scenarios) was to redo or explain how it might be played out in a more effective way. The nursing staff were scheduled in training groups during their work shifts around the clock. Coverage for patient care for those staff members in training was provided by the nurse managers, nurse educator, and charge nurse. If the shift was especially busy, the training was rescheduled. This happened an estimated 10% of the time. Computers were set up ahead of time with the avatars to be used by the trainees in their role-plays in SL. Although the nursing staff and trainers wore headsets, which were required to permit the use of voice to “talk” in SL during the virtual scenarios, trainees had to be placed in different conference rooms and offices. Otherwise, they would be distracted by the others talking. Each scenario ran 10 to 15 min followed by a 5 to 10 min debriefing. During the training session, participants were given specific instructions for their role through the use of cue cards (given prior to the scenario) and notecards passed in SL during the scenario. The notecards contained a description of the role the participant was playing. The cue cards contained information about the patient the participant was interacting with in SL. The participants were then instructed to play out the scenario as they would if it were occurring in real life on their unit. During the debriefing conducted by the trainers, participants were asked to identify the presence or absence of teamwork behaviors (e.g., backup, leadership, shared mental models, etc.). They were also instructed as to how to give effective feedback, which was needed in all three scenarios. Finally, they were asked what they might do differently in the future as a result of the simulated learning experience.
Measures The NTS and a Teamwork Knowledge Survey were used to measure teamwork. These were administered to the study participants 3 weeks before the intervention and 3 weeks after all of the staff had undergone the training. In
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
172
Western Journal of Nursing Research 37(2)
Table 1. Virtual Scenarios Utilized in Training. Scenario
Teamwork Elements
Challenging Team leadership, assignment mutual performance requiring teamwork monitoring, backup and delegation behavior, adaptability, team orientation, shared mental models.
Over-delegation to nursing assistant
Team leadership, mutual performance monitoring, backup behavior, adaptability, team orientation, shared mental models.
RN and NA miscommunication and follow up
Mutual performance monitoring, shared mental model, closedloop communication interpersonal relationships.
Scenario Nurses begin the day by getting report on the unit and then they are directed to go and visit their patients. The charge nurse is off the unit at a meeting. When Nurse 1 arrives in her patient’s room the patient in bed 1 states “I think my roommate just fell out of bed.” The patient in bed 1 is also complaining of severe dizziness (rule out gastrointestinal bleed). RNs and NAs begin the day by getting report on the unit and then they are directed to go and visit their patients. When the NA arrives the patient (r/o myocardial infarction) in Bed 3 is complaining of severe chest pain. When the NA tries to relay this to the RN caring for the patient (played by the trainer) she is told, “I am not worried about it.” After receiving report, the nurse responds to a patient call light. The patient relays to the nurse that yesterday she did not receive aspects of nursing care (no glucose check, did not walk). The NA that is assigned to care for her was the same one as yesterday and she (the patient) does not want that NA taking care of her again.
Note. RN = Registered Nurses; NA = nursing assistants.
addition, the participants were asked five questions about computer proficiency and experience using computer simulation and virtual worlds in the pre-intervention data collection.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
173
Kalisch et al.
The NTS Nursing teamwork was measured by the NTS, a 33-item questionnaire with a 5-point Likert-type scale from rarely (1) to always (5). Total score ranges from 33 to 165 with higher scores indicating higher nursing teamwork. The NTS consists of five subscales: (a) trust (i.e., shared perception that members will perform actions necessary to reach interdependent goals and act in the interest of the team), (b) team orientation (i.e., cohesiveness, the extent to which individuals see the team’s success taking precedence over individual needs and performance), (c) backup (i.e., helping one another with their tasks and responsibilities), (d) shared mental model (i.e., mutual conceptualizations of the task, roles, strengths/weaknesses, and processes and strategy necessary to attain interdependent goals), and (e) team leadership (i.e., structure, direction, and support). Internal reliability (Cronbach’s α = .94), test–retest reliability (r = .92), and validity (concurrent, convergent, and contrast validity) of the NTS have been reported elsewhere (B. J. Kalisch, Lee, & Salas, 2010).
Teamwork Knowledge Test Knowledge about teamwork was tested with an eight-item test, in which the percent of correct responses for each was calculated. The questions for this survey were drawn from those contained in the training manual for TeamSTEPPS (King et al., 2008). TeamSTEPPS is based on the Salas model of teamwork, which was used as the conceptual framework for this study. Computer and virtual world experience. Computer experience was measured via questionnaire on a scale of 1 (very low) to 5 (very high). Respondents were asked whether they had heard of virtual worlds. They were then asked whether they had participated in any computer virtual worlds.
Procedures Each person participating in the study completed a consent form. All staff on the unit were given a survey packet both before and after the intervention containing a large candy bar as an incentive to participate. To maintain anonymity and consistency in comparing pre- and post-training questionnaires, each respondent provided an anonymous identifier with birth month, mother’s first name, and mother’s birth month. Completed surveys were deposited into a locked box on the unit.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
174
Western Journal of Nursing Research 37(2)
Table 2. Teamwork Scores Between Pre- and Post-Intervention. Pre-Intervention, Post-Intervention, M ± SD M ± SD Teamwork overall Trust Team orientation Backup Shared mental model Team leadership
3.25 ± 0.58 2.94 ± 0.64 2.80 ± 0.61 3.18 ± 0.71 3.67 ± 0.62 3.19 ± 0.92
3.49 ± 0.67 3.22 ± 0.73 3.13 ± 0.73 3.38 ± 0.72 3.93 ± 0.48 3.58 ± 0.78
p Effect Size
t −3.07 −2.24 −3.45 −2.22 −1.73 −2.11
.012 .042 .004 .045 .109 .052
0.96 0.96 0.97 0.60 0.49 0.53
Data Analysis Sixteen participants completed both the pre- and the post-test. Data were analyzed by descriptive statistics (means, standard deviation, and percentages) and paired t test. Effect sizes were calculated to use Cohen’s d as the difference between the means for two groups divided by the standard deviation of either group (Cohen, 1988). The PASW Statistics software (version 18.0; SPSS, Inc., Chicago, IL) was used for the analyses.
Results A total of 43 nursing staff RNs (88.4%) and NAs (11.6%) participated in this study. Among the participants, 81.4% were female and 41.9% had a bachelor’s degree or higher. Of the staff that participated, 60% rated their overall computer proficiency as high or very high and the rest of participants (40%) rated their computer proficiency as very low, low, or medium. Completing both pre- and post-intervention were 16 nursing staff members (37.2%).
Teamwork Behaviors As seen in Table 2, the overall mean scores of teamwork at pre-intervention was 3.25 (SD = 0.58) and at post-intervention it was 3.49 (0.67). More explicitly, the teamwork overall score at post-intervention was significantly higher than those at pre-test (t = −3.07, p < .012). According to Cohen’s effect size (Cohen, 1988), the intervention had a large (d = .96) and statistically significant effect on the measure of overall teamwork. Three out of five teamwork subscale scores (trust, team orientation, and backup) were significantly higher from pre- to post-intervention. The intervention also had large (0.60 ≤ d ≤ 0.97) and significant effects on the three teamwork subscales.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
175
Kalisch et al.
Table 3. Changes in Teamwork Behaviors and Knowledge by Level of Computer Proficiency. Computer Proficiency Very Low to Medium, High to Very High, M ± SD M ± SD
Teamwork overall Trust Team orientation Backup SMM Team leadership Teamwork knowledge
3.07 ± 0.50 3.07 ± 0.49 2.91 ± 0.51 3.28 ± 0.67 3.86 ± 0.61 3.58 ± 0.66 6.88 ± 1.25
3.46 ± 0.75 3.34 ± 0.88 3.25 ± 0.92 3.40 ± 0.76 3.84 ± 0.53 3.59 ± 0.91 6.75 ± 1.04
t
p
−0.27 −0.75 −0.95 −0.34 0.06 −0.03 0.22
.792 .466 .356 .737 .951 .979 .830
Note. SMM = shared mental model.
Teamwork Knowledge Scores Teamwork knowledge scores were not significantly different between preand post-intervention (t = −1.08, p < .301).
Computer Proficiency and Previous Virtual World Experience Nursing staff reporting higher levels of computer experience did not score higher on teamwork behaviors and knowledge after the intervention than did those with less computer experience (Table 3). We also found that experience with virtual worlds did not influence overall teamwork scores, the five teamwork subscales, or the level of teamwork knowledge. Of the five teamwork subscales, “team orientation” and “trust” scores were slightly higher for those who had participated in virtual world simulations, but those differences were not statistically significant (t = 1.27, p = .225; t = 0.29, p = .775, respectively). Higher scores on backup behavior (t = −0.35, p = .733), shared mental models (t = −0.58, p = .569), team leadership (t = −0.39, p = .702), and teamwork knowledge (t = −0.30, p = .766) for those with no experience with virtual world simulations did not reach statistical significance.
Discussion The need for effective teamwork in nursing has been demonstrated. Yet, capacity building for teamwork skills has been inadequate. The purpose of this study was to determine the feasibility of a virtually based intervention for
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
176
Western Journal of Nursing Research 37(2)
improving teamwork among nursing staff. Through the use of virtual simulations, nurses were given opportunities to practice skills related to teamwork and deal with difficult situations (e.g., when team members do not function effectively as team members). The basic teamwork behaviors (team orientation, backup, etc.) were reviewed after each scenario, asking the trainees to identify whether this element of teamwork was present or not. This study was effective because it used scenarios that reflect the everyday experiences of nursing staff working together as a team (e.g., RN and NA miscommunication and follow up; over-delegation to the NA). A comparative study with other approaches (e.g., traditional classroom, face-to-face role-playing, highfidelity simulation, etc.), to enhancing nursing teamwork is needed to fully evaluate the effectiveness of virtual simulation. Findings from the study showed that overall teamwork scores did improve pre- and post-intervention. In addition, scores on the subscales of trust, team orientation, backup, and team leadership significantly improved after participation in the virtual simulations. Despite improvement in teamwork behaviors, actual knowledge of teamwork did not improve significantly from pre- to post-intervention. In this study, nurses were not able to articulate team behaviors, but they were able to practice them, as noted by improved teamwork behavior scores post-implementation. Other notable findings included the lack of significance between changes in teamwork behaviors and knowledge and computer proficiency and participation in virtual worlds. All of the nurses were able to participate in the intervention with very limited training in virtual environments. This finding suggests that a staff member would be able to effectively participate in virtual, computer-based training, despite his or her level of computer experience and exposure to virtual training. There are limitations of this study that may have affected the results. Specifically, in a single-group design, this study took place on one unit with a small sample of nursing staff; thus generalizability is limited. The intervention should be retested with a larger sample of patient care unit nursing teams. Another limitation was the participation rate in the intervention. Not all of the staff participated, thus some did not have the opportunity to practice effective teamwork behaviors. Also, there was a large attrition rate from pre- to posttest response levels. Attrition, common in pre- and post-studies, was compounded in this study as it coincided with nursing staff vacations. This study described a new approach to team training using the virtual environment. This approach allows for greater access, as participants only need to be present “virtually.” In addition, the virtual environment provides a safe environment to practice non-technical skills such as teamwork. In the
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
177
Kalisch et al.
nursing environment, it is difficult to get staff away from work for training. Use of virtual simulations to train the teams may be a suitable alternative. Use of the virtual environment has the potential to disseminate information/ evidence rapidly to participants, thus breaking down barriers of the traditional classroom or training facility. Further work is needed to explore this new innovative approach to training nursing staff members in teamwork. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded in part by the Blue Cross Blue Shield Foundation of Michigan.
References Aebersold, M., Tschannen, D., & Bathish, M. (2012). Innovative simulation strategies in education. Nursing Research and Practice, 2012, Article 765212. Agarwal, R., Sands, D. Z., & Schneider, J. D. (2010). Quantifying the economic impact of communication inefficiencies in U.S. hospitals. Journal of Healthcare Management, 55, 265-281; discussion 281-262. Amos, M. A., Hu, J., & Herrick, C. A. (2005). The impact of team building on communication and job satisfaction of nursing staff. Journal for Nurses in Professional Development, 21, 10-16. Blegen, M. A., Vaughn, T., & Vojir, C. P. (2008). Nurse staffing levels: Impact of organizational characteristics and registered nurse supply. Health Services Research, 43(1, Pt. 1), 154-173. Brewer, B. B. (2006). Relationships among teams, culture, safety, and cost outcomes. Western Journal of Nursing Research, 28, 641-653. Burke, C. S., Sims, D. E., & Salas, E. (2005). Is there a “big five” in teamwork? Small Group Research, 36, 555-599. Catchpole, K. R., Giddings, A. E., Wilkinson, M., Hirst, G., Dale, T., & de Leval, M. R. (2007). Improving patient safety by identifying latent failures in successful operations. Surgery, 142, 102-110. doi:10.1016/j.surg.2007.01.033 Chang, W.-Y., Ma, J.-C., Chiu, H.-T., Lin, K.-C., & Lee, P.-H. (2009). Job satisfaction and perceptions of quality of patient care, collaboration and teamwork in acute care hospitals. Journal of Advanced Nursing, 65, 1946-1945. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Collette, J. E. (2004). Retention of nursing staff—A team-based approach. Australian Health Review, 28, 349-356.
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
178
Western Journal of Nursing Research 37(2)
Corrigan, J. M., Kohn, L. T., & Donaldson, M. S. (2000). To err is human. Washington, DC: National Academies Press. Day, T., Iles, N., & Griffiths, P. (2009). Effect of performance feedback on tracheal suctioning knowledge and skills: Randomized controlled trial. Journal of Advanced Nursing, 65, 1423-1431. Falcone, J. R. A., Daugherty, M., Schweer, L., Patterson, M., Brown, R. L., & Garcia, V. F. (2008). Multidisciplinary pediatric trauma team training using high-fidelity trauma simulation. Journal of Pediatric Surgery, 43, 1065-1071. Fransen, A. F., van de Ven, J., Merién, A. E. R., de Wit-Zuurendonk, L. D., Houterman, S., Mol, B. W., & Oei, S. G. (2012). Effect of obstetric team training on team performance and medical technical skills: A randomised controlled trial. BJOG, 119, 1387-1393. Holcomb, J. B., Dumire, R. D., Crommett, J. W., Stamateris, C. E., Fagert, M. A., Cleveland, J. A., . . . Mattox, K. L. (2002). Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. Journal of Trauma, 52, 1078-1085; discussion 1085-1076. Kalisch, B., & Schoville, R. (2012). It takes a team. American Journal of Nursing, 112(10), 50-54. doi:10.1097/01.NAJ.0000421024.08037.0d Kalisch, B. J., Curley, M., & Stefanov, S. (2007). An intervention to enhance nursing staff teamwork and engagement. Journal of Nursing Administration, 37, 77-84. Kalisch, B. J., Gosselin, K., & Choi, S. H. (2012). A comparison of patient care units with high versus low levels of missed nursing care. Health Care Management Review, 37, 320-328. Kalisch, B. J., & Lee, H. (2009). Nursing teamwork, staff characteristics, work schedules, and staffing. Health Care Management Review, 34, 323-333. Kalisch, B. J., & Lee, K. H. (2010). The impact of teamwork on missed nursing care. Nursing Outlook, 58, 233-241. doi:10.1016/j.outlook.2010.06.004 Kalisch, B. J., Lee, H., & Rochman, M. (2010). Nursing staff teamwork and job satisfaction. Journal of Nursing Management, 18, 938-947. Kalisch, B. J., Lee, H., & Salas, E. (2010). The development and testing of the nursing teamwork survey. Nursing Research, 59, 42-50. Kalisch, B. J., Weaver, S. J., & Salas, E. (2009). What does nursing teamwork look like? A qualitative study. Journal of Nursing Care Quality, 24, 298-307. Kidd, L. I., Knisley, S. J., & Morgan, K. I. (2012). Effectiveness of a Second Life(®) simulation as a teaching strategy for undergraduate mental health nursing students. Journal of Psychosocial Nursing and Mental Health Services, 50(7), 28-37. King, H., Battles, J., Baker, D., Alonso, A., Salas, E., Webster, J., . . . Salisbury, M. L. (2008). TeamSTEPPS™: Team strategies and tools to enhance performance and patient safety. In K. Henriksen, J. B. Battles, M. A. Keyes, & M. L. Grady (Eds.), Advances in patient safety: New directions and alternative approaches (Vol. 3, Performance and tools). Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from http://www.ncbi.nlm.nih.gov/books/NBK43686/ LeFlore, J. L., Anderson, M., Zielke, M. A., Nelson, K. A., Thomas, P. E., Hardee, G., & John, L. D. (2012). Can a virtual patient trainer teach student nurses how
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
179
Kalisch et al.
to save lives-teaching nursing students about pediatric respiratory diseases. Simulation in Healthcare, 7, 10-17. Maithel, S., Sierra, R., Korndorffer, J., Neumann, P., Dawson, S., Callery, M., . . . Scott, D. (2006). Construct and face validity of MIST-VR, Endotower, and CELTS. Surgical Endoscopy, 20, 104-112. Marnie, E. R., Mari, A., Vanessa, R., Mueck, J. T., Roslyn, F. S., & Paul, H. M. (2006). Achieving housestaff competence in emergency airway management using scenario based simulation training. Chest, 129, 1453-1458. Messmer, P. R. (2008). Enhancing nurse-physician collaboration using pediatric simulation. Journal of Continuing Education in Nursing, 39, 319-327. Pronovost, P. J., Thompson, D. A., Holzmueller, C. G., Lubomski, L. H., Dorman, T., Dickman, F., . . . Morlock, L. L. (2006). Toward learning from patient safety reporting systems. Journal of Critical Care, 21, 305-315. doi:10.1016/j. jcrc.2006.07.001 Rafferty, A. M., Ball, J., & Aiken, L. H. (2001). Are teamwork and professional autonomy compatible, and do they result in improved hospital care? Quality in Health Care, 10(Suppl. 2), ii32-ii37. Robertson, B., Kaplan, B., Atallah, H., Higgins, M., Lewitt, M. J., & Ander, D. S. (2010). The use of simulation and a modified TeamSTEPPS curriculum for medical and nursing student team training. Simulation in Healthcare, 5, 332-337. Rosen, M. A., Salas, E., Wilson, K. A., King, H. B., Salisbury, M., Augenstein, J. S., . . . Birnbach, D. J. (2008). Measuring team performance in simulation-based training: Adopting best practices for healthcare. Simulation in Healthcare, 3, 33-41. Safe practices for better healthcare–2010 update: A consensus report. (2010). Washington, DC: National Quality Forum. Shapiro, M. J., Morey, J. C., Small, S. D., Langford, V., Kaylor, C. J., Jagminas, L., . . . Jay, G. D. (2004). Simulation based teamwork training for emergency department staff: Does it improve clinical team performance when added to an existing didactic teamwork curriculum? Quality and Safety in Health Care, 13, 417-421. Tschannen, D., Aebersold, M., McLaughlin, E., Bowen, J., & Fairchild, J. (2012). Use of virtual simulations for improving knowledge transfer among baccalaureate nursing students. Journal of Nursing Education and Practice, 2(3), 15-24. Van Sickle, K. R., McClusky, R. D. A., Gallagher, A. G., & Smith, C. D. (2005). Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surgical Endoscopy, 19, 1227-1231. Wayne, D. B., Didwania, A., Feinglass, J., Fudala, M. J., Barsuk, J. H., & McGaghie, W. C. (2008). Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: A case-control study. Chest, 133, 56-61. Wheelan, S. A., Burchill, C. N., & Tilin, F. (2003). The link between teamwork and patients’ outcomes in intensive care units. American Journal of Critical Care, 12, 527-534. Wiegmann, D. A., ElBardissi, A. W., Dearani, J. A., Daly, R. C., & Sundt, T. M., III. (2007). Disruptions in surgical flow and their relationship to surgical errors: An exploratory investigation. Surgery, 142, 658-665. doi:10.1016/j.surg.2007.07.034
Downloaded from wjn.sagepub.com at University of Bath - The Library on June 13, 2015
