Clinical Nurse SpecialistA Copyright B 2015 Wolters Kluwer Health, Inc. All rights reserved.
Feature Article Creating a Sustainable, Interprofessional-Team Training Program: Initial Results Virginia K. Riggall, DNP, RN, CCRN, ACCNS-AG n Charlene M. Smith, DNS, MSEd, WHNP, RN-BC, CNE, ANEF
Purpose/Objectives: The purpose of this program evaluation was to explore whether incorporating deliberate learning concepts, through the use of simulated patient scenarios to teach interprofessional collaboration skills to a healthcare team on one acute-care hospital unit, would improve the resuscitation response in the first 5 minutes on that unit. Design/Setting: This was a pilot program evaluation utilizing a unit-based, clinical nurse specialist in the deployment of an interprofessional educational program involving simulation on an acute medical floor in a large tertiary-care hospital. Sample: Eighty-four staff members participated in 17 simulations. The sample included first-year internal-medicine residents, registered nurses, respiratory therapists, and patient care technicians. Methods: This was a program evaluation that used the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ) (Classroom slides: TeamSTEPPS essentials; http://www.ahrq.gov/professionals/ education/curriculum-tools/teamstepps/instructor/essentials/ slessentials.html#s3) during the presimulation/postsimulation sessions to assess the participants’ perceptions of teamwork. Expected intervention behaviors were collected through observations of participants in the simulations and compared with the American Heart Association guidelines (Circulation 2010;122:S685YS670, S235YS337). Common perceptions of participants regarding the experience were obtained through open-ended evaluation questions. Author Affiliations: Clinical Nurse Specialist, Department of Clinical Education and Nursing Research at Rochester General Health System (Dr Riggall); and Full-time Professor and Undergraduate Cochair at Wegmans School of Nursing, St John Fisher College; Research Coordinator, University of Rochester Medical Center, Highland Hospital; and Nursing Professional Development Specialist, Strong Memorial Hospital (Dr Smith), Rochester, New York. The authors report no conflicts of interest. Correspondence: Virginia K. Riggall, DNP, RN, CCRN, ACCNS-AG, 3178 Valley Dr, Walworth NY 14568 ([email protected]). DOI: 10.1097/NUR.0000000000000121
Clinical Nurse Specialist
Results: Fifty-three participants completed the preY and postYT-TPQ. Mean scores in the leadership category of T-TPQ decreased significantly (P = .003) from the pretest (median, 2.167) to the T-TPQ posttest (median, 2.566). Only 35% of the groups administered a defibrillation during the ventricular fibrillation simulation scenario, and only 1 group delivered this shock within the American Heart Association’s recommended time frame of 2 minutes (Circulation 2010;122:S235YS337). Conclusion: A single resuscitation simulation was not enough interventional dosage for staff to improve the resuscitation process. A longitudinal study should be conducted to determine the effectiveness of the program after staff members have repeated the program multiple times. Implications: A unit-based quality-improvement simulation training program could help improve the first-5-minute response and resuscitation skills of staff by increasing the frequency of unit-based training overseen by the unit’s clinical nurse specialist. KEY WORDS: clinical nurse specialist, collaboration, interprofessional, resuscitation, simulation training
A
cute-care medical units are composed of fluid teams, changing frequently, with floating healthcare staff members used to fill unit staffing needs or medical residents rotating units every 3 to 6 weeks. Developing and offering training programs for constantly changing teams are a challenge for centralized, hospital-based staff-development departments, although potentially more feasible for decentralized, unit-based clinical nurse specialists (CNSs). Unit-based qualityimprovement programs aimed at improving the first-5-minute response and cardiac resuscitation skills of interprofessional healthcare providers can be deployed with the assistance of an educational interprofessioal simulation program intervention developed by a system-based CNS for consistent
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
147
Feature Article deployment across the organization and supported by the unit-based CNS. Continuous efforts at team building to reduce delayed resuscitation efforts can be sustained through interprofessional training programs involving simulation conducted on the unit-by-unitYbased CNSs.
BACKGROUND In 2011, Health Grades1 reported that there were 13 preventable patient-safety events from 2007 through 2009, costing $7.3 billion and resulting in 79 670 potentially preventable deaths. The Institute of Medicine2 has suggested that one way to keep patients safe is to develop and support a highly trained, collaborative, interdisciplinary team. Studies have shown a relationship between adverse events, such as failure to rescue, and team collaboration.3 Failure to rescue is a measurement used by the Agency of Healthcare Research and Quality4 to determine the degree to which providers responded to a hospital-acquired condition, such as a cardiac arrest, where the event causes death or permanent disability. This measurement reflects the quality of care patients receive by looking at the type of patient monitoring that occurred and how effective the healthcare professionals’ response was to the change in the patient’s condition.4 LITERATURE REVIEW In 2006, the Institute for Healthcare Improvement’s initiative, the 100 000 Lives Campaign, was created to help reduce hospital failure to prevent deaths through the creation of rapidresponse teams in hospitals.5 Despite the creation of these teams, failure to rescue patients continues because the healthcare professionals caring for patients are unable to identify the symptoms of deterioration and do not seek timely help to correct the patient’s change in condition.5 In 2008, Chan et al6 reviewed 24 193 patient charts prior to the implementation of rapid-response team training and 24 978 patient charts after implementation in a large tertiarycare academic hospital to determine if there was a decrease in hospital mortality after implementation of the program. The retrospective chart review revealed that there was not a decrease in cardiac arrests, nor was there a decrease in hospital mortality for patients with a cardiac arrest after implementation of a rapid-response team.6 In 2009, in response to the study of Chan et al, the Joint Commission established the National Patient Safety Goal 16, which required hospitals to review their current early-recognition and response programs to improve survival for patients with a cardiac arrest.7 Interestingly, development of rapid-response teams in hospitals may lead to deterioration in cardiopulmonary resuscitation (CPR) skills for nurses working in acute-care units. Mondrup et al8 found a significant difference between the CPR skill level of first responders in hospitals and the CPR skill level of members of the rapid-response team. The authors determined that the first responders’ CPR skills deteriorated quickly because they did not perform CPR on a 148
regular basis, whereas members of the cardiac resuscitation team performed higher-quality chest compressions during CPR and had shorter times of no blood flow than did those nurses who responded first to a code.8 The authors found that patients with a cardiac arrest, who did not receive care until the rapid-response team arrived, were not defibrillated for 3.5 minutes.8 The European Resuscitation Council and the American Heart Association (AHA)8,9 currently recommend that a patient with a cardiac arrest should be defibrillated within 2 minutes for best survival outcomes. However, Hunt et al10 reported that staff members, usually nurses, who found patients unresponsive were found to be spending too much time preparing the room for the code team and did not start CPR before the code team arrived. In a witnessed cardiac arrest, immediate defibrillation for ventricular fibrillation or pulseless tachycardia is a class 1 recommendation according to the AHA.9 A class 1 recommendation has been defined by the AHA as a treatment or procedure that should be performed because the benefits outweigh the risks.11 In a study by Hunziker et al,12 100 teams, composed of 3 randomly chosen physicians and 1 nurse, were asked to participate in a simulated cardiac arrest. Half of the teams had undergone some team-building training prior to the simulated exercise. The researchers found that 36% of the teams without any team training delayed their first defibrillation beyond 2 minutes, compared with 12% of the teams that had completed team training in a simulated environment.12 In another study, data from the National Registry of Cardiopulmonary Resuscitation,13 related to ventricular fibrillation and pulseless ventricular tachycardia, revealed that 30% of the patients who experienced cardiac arrest due to ventricular arrhythmia were not defibrillated in 2 minutes or less.14 The researchers found a correlation between decreased survival rates and delay in defibrillation beyond 2 minutes.14 Delay in defibrillation may be attributable, in part, to failure to develop team skills.12,14 The National Registry of Cardiopulmonary Resuscitation is the largest registry in the United States that uses standardized definitions for measuring the care and outcomes during cardiac arrests that occur in hospitals.13 Assigning roles to each member of the team is an additional task that must be accomplished during a cardiac arrest; thus, delaying this essential step can lead to delaying the 2 evidence-based lifesaving measures involving early effective compressions and defibrillation.12,14 Hands-on time for compressions and time to defibrillation are considered 2 team-performance markers for CPR.12 It is suggested that nontechnical skills, such as communication and teamwork, can impact patient outcomes during cardiac arrests. Highly effective teams are associated with coordinated team leadership roles and clarity in responsibilities immediately following a cardiac arrest.12 Conversely, hands-on time for CPR and time to defibrillation are negatively affected by poorly developed teams.12 Lemieux-Charles and McGuire15 found that highly functioning teams were
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
characterized by positive communication patterns, low levels of conflict, and high levels of collaboration, coordination, cooperation, and participation. Poor outcomes during lifesaving measures were attributed to poor team building, a task that must be completed quickly by teams during patient emergencies.12 Team performance can improve through deliberate practice with the use of simulated scenarios performed in a safe environment with a human-patient simulator. The National League for Nursing defines simulation as ‘‘a technique that uses a situation or environment created to allow persons to experience a representation of a real event for the purpose of practice, learning, evaluation, or testing or to gain understanding of systems or human actions.’’16,17 Deliberate practice incorporates highly structured activities and monitoring of specific skills, such as CPR skills, by which the participants receive real-time feedback with the goal of improving skill levels.18 In addition to using simulation scenarios to improve resuscitation processes, programs such as Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) can be incorporated into training programs to develop the nontechnical skills such as communication and teamwork. TeamSTEPPS is an evidence-based program that was developed by the Agency for Healthcare Research and Quality and the Department of Defense to provide organizations with core competencies for trainable skills such as leadership, situation monitoring, mutual support, and communication.19 Incorporating TeamSTEPPS concepts into a simulation scenario provides staff with hands-on practice of resuscitation skills and an opportunity to reflect on how team collaboration, communication, performance, and behaviors impact patient outcomes. The pilot program evaluation, entitled Improving Interprofessional Collaboration Using Deliberate Practice, was developed to improve team collaboration and resuscitation skills in the first 5 minutes of a cardiac arrest. The purpose of the program evaluation was to incorporate deliberate learning concepts, through the use of a simulated patient scenario, to teach interprofessional collaboration skills to the healthcare team on 1 acute-care hospital unit in order to determine if the training program could improve the team response to cardiopulmonary arrest and the patient resuscitation process. The program was the first phase of a pilot project to mentor unit-based CNSs on how to use simulation as a strategy for teaching interprofessional collaboration to their unit team members so they may sustain collaborative, effective teams that provide safe patient care.
METHODS Design, Setting, Sample This was a pilot program evaluation that was developed to look at whether using simulation as an interprofessional teaching strategy would change the perceptions of healthcare providers about interprofessional collaboration on one Clinical Nurse Specialist
40-bed medical unit in a northeastern tertiary-care teaching hospital. Before initiating the program evaluation, resuscitation data for the prior year were reviewed for all acute-care units in the hospital; the unit with the largest number of resuscitation events was selected for the program. The resuscitation data were reviewed by the unit-based CNS and the principal investigator (PI) to identify the gaps in staff performance during resuscitation events. For example, the times to defibrillation were significantly longer than the AHA recommendation of 2 minutes. First-year internal-medicine residents, nurses, patient care technicians, and respiratory therapists who worked on the medical unit were invited to participate in the pilot program. The program was implemented at the end of the first residency year while the group was transitioning into their second year of residency. Second-year residents are required to take a leadership role in resuscitation events. The director of the residency program chose this group to prepare the residents with an educational opportunity to practice leadership skills required during resuscitation events. The simulation scenario was selected from The First 5 Minutes program to provide a standardized scenario for all participants on responding to a cardiopulmonary arrest. The First 5 Minutes program was developed by Laerdal Medical Corporation20 and the University of Pittsburgh Medical Center as an educational program for nurses, respiratory therapists, and other healthcare team members. Prior to beginning the simulation sessions with the staff, 4 CNSs, including the unit-based CNS from the pilot medical floor, participated in the simulation scenario. Each CNS completed the program evaluations, including an additional evaluation called the Pilot Simulation Evaluation Tool. This tool provided the PI with feedback and suggestions prior to the implementation of staff simulations. Including the unit-based CNSs in the pilot simulation was the beginning of a partnership between the centralized-CNS educator, the PI, and the unit-based CNSs. After the pilot simulation, the PI mentored the unit-based CNS on the presimulation planning by developing a simulation schedule that would take into consideration the unit’s staffing patterns while allowing enough staff to participate in each simulation to make the experience realistic. Once dates were identified, the unit-based CNS scheduled staff members for the simulations and coordinated time for the registered nurses and the patient care technicians to complete their presimulation questionnaires. The PI communicated the schedule to the residency program administrator and the respiratory care manager. Once the schedule was completed, the PI and the unit-based CNS customized The First 5 Minutes scenario to create a common patient scenario that could occur on the unit. The scenario was developed around the gaps in knowledge and skills that were identified from reviewing the resuscitation data for the unit. Prior to running the simulations, the PI trained the unit-based CNS on how to set up and run the
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
149
Feature Article human patient simulator and the video equipment. The training goal during this phase was to allow the unit-based CNS the opportunity to focus on the coordination and technical skills of running a simulation. While the PI did mentor the unit-based CNS on how to facilitate the simulation prebriefs and the postsimulation debriefing, it was decided that the PI would facilitate the simulation prebriefs and the postsimulation debriefings to maintain content consistency with each group. After each simulation, the unit-based CNS and the PI reviewed the simulated event for additional staff educational needs, such as how to set up and use a defibrillator. Many participants did not have the experience or skills to set up the defibrillator and deliver an electrical shock through either the manual defibrillation mode or by using automatic electronic defibrillator. Ninety staff members were invited to participate in the program evaluation, and 84 staff members participated in 17 simulations. The participants who completed the simulations included 15 first-year internal-medicine residents starting their second-year, 17 respiratory therapists, 15 patient care technicians, and 37 registered nurses. The sample group represented 75% of the hospital’s first-year internal medical residents (85% were invited to participate), 23% of the hospital’s respiratory therapy staff (100% invited participated), 90% of the unit’s patient care technicians (100% invited participated), and 82% of the unit’s staff nurses (100% invited participated). Approval was obtained from the institutional review board, and all study participants gave voluntary informed consent for the program evaluation data to be analyzed and published. Instruments and Measurements Prior to initiating the simulated scenarios, the PI developed a short questionnaire to assess staff members’ perceptions of teamwork/team functioning and communication on the unit. Content analysis of the narrative responses was conducted and recorded based on common themes. One-word responses were removed from the analysis. The purpose of the questions was to provide the PI with a baseline understanding of the team culture that existed on the unit prior to the project. Each participant was asked to complete a demographic survey, which included questions about the type of CPR training each participant had received prior to the simulation. Participants responded using a 5-point Likert scale (strongly agree = 1, agree = 2, neutral = 3, disagree = 4, strongly disagree = 5). The TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ), a valid and reliable tool created for hospitals by the Agency for Healthcare Research and Quality and the Department of Defense,21 was administered to the participants prior to the simulation, using the same 5-point Likert scale. The T-TPQ measures an individual’s perception of the level of teamwork that exists within a group.21 The T-TPQ reliability was determined for each of the 5 constructs: team structure, leadership, situation monitoring, mutual support, and communication. Cronbach’s ! coefficients for the 150
multi-item variables ranged from .88 (communication) to .95 (leadership). Content validity was obtained by a sample of 169 TeamSTEPPS participants who reviewed the questionnaire. Convergent validity was established with the Agency for Healthcare Research and Quality’s Hospital Survey on Patient Safety, a tool that measures patient-safety culture. The correlations with Hospital Survey on Patient Safety ranged from .60 (communication) to .81 for the complete TeamSTEPPS Teamwork Perception Questionnaire.21 Each simulation was videotaped to validate all observational data collected for the program evaluation. Expected behaviors, which were defined as essential resuscitation interventions recommended by AHA, were collected by the primary investigator as observational data. The timing of participants’ observed interventions was compared with the AHA recommendations,16 as noted in Figure 1. The observational data for the expected interventions were confirmed for accuracy by reviewing each simulation video. At the conclusion of the simulation, each participant was asked to complete an electronic evaluation regarding their perceptions of the learning experience. Six months after the completion of the simulations, each participant was sent an e-mail inviting them to complete a postsimulation T-TPQ questionnaire. Seventy-seven percent of the participants who completed the presimulation T-TPQ completed the postsimulation T-TPQ.
RESULTS Content analysis was used to categorize the responses from the following questions prior to simulations; ‘‘How would you describe the environment in which healthcare teams work on your unit?’’ and ‘‘How does the team communicate about patient-safety risks?’’ Responses included a contrast between recognition of the importance of working as a team for the benefit of the patient and the absence of interest in creating effective teams. One unit nurse stated that ‘‘the nurses work well as a team, but there is disconnection between the nurses and the patient care technicians.’’ On the other hand, the patient care technicians’ descriptions of the unit varied from the staff being ‘‘a well-oiled machine’’ to the teams being dependent on characteristics of the staff members who were working during a particular shift. Nurses stated that communication about patient-safety events occurred during staff huddles during the shift, but there was no mention of communication about patient-safety events during handoffs or shift reports. Interestingly, a respiratory therapist stated that communication was very weak on the floor in respect to patient safety, and another respiratory therapist commented that the physicians were often dismissive of the nurses’ input on patient care. Conversely, the internal-medicine residents stated that the nurses and patient care technicians worked well together to care for the patients. All quantitative data were analyzed with descriptive statistics usingIBMSPSSStatisticsversion22(THINKEDU,Carrollton,Texas).
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 1. Expected behaviors checklist.
Seventy-four participants completed the demographic survey. Thirty-nine percent of the participants responded that they had been involved in more than 5 resuscitation events in their career, yet 49% of the participants felt their resuscitation skills were at least at a competent level. Only 54% of the participants who completed the demographic survey stated they had attended formal basic life support training through the AHA or the American Red Cross, and 44.6% of the participants stated they had some critical-care experience. The complete sample demographics compiled from the demographic survey are listed in Table 1. The Expected Behaviors data, which were collected during the simulations by the PI and confirmed by viewing the simulation videos, revealed only 35% of the groups administered a defibrillation during the ventricular-fibrillationresuscitation scenario and that only 1 group delivered this shock within the AHA’s recommended time frame of 2 minutes.9 Twenty-nine percent of the groups assigned roles during the simulations, and 18% used closed-loop communication for medications during the scenario. The AHA recommendation for starting chest compressions is 10 seconds from the moment a person is found unresponsive and pulseless. In the 17 groups completing the scenarios, the average time to start compressions was 62.5 seconds. Postsimulation data from the participants were collected using both quantitative data and open-ended questions. The Simulation Evaluation Tool included questions using a 5-point Clinical Nurse Specialist
Likert scale and 2 narrative questions. Ninety-eight percent of the participants either strongly agreed or agreed that the simulated environment emphasized the importance of training as a team, and 97% of the participants felt the training reinforced the importance of effective communication. Mean scores from the simulation evaluations are summarized in Table 2. Narrative responses from the question, ‘‘List what you learned today that will be most useful to you while caring for patients,’’ were collected with common responses noted. Thirty-six percent of the participants stated that communicating more effectively with other team members during resuscitation events would be most useful while caring for patients. Figure 2 lists the major themes from this question. A second narrative question asked the participants for suggestions that would have improved the learning experience. Thirty-three percent of the participants suggested a more realistic environment would have improved the experience, and 23% suggested increasing the frequency of simulation experiences. Figure 3 summarizes all the major themes from this question. Data from the preY and postYT-TPQ were analyzed using a nonparametric test, the Wilcoxon signed rank test. The Wilcoxon signed rank test is used frequently to measure changes in attitudes in a single sample. The comparison of the T-TPQ results indicated that the participants who took part in the program evaluation felt there had been a change in their perception about leadership from the
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
151
Feature Article Table 1. Sample Demographic Characteristics
(n = 74) Category Discipline
Basic life support training
Participation in actual code
Perception of resuscitation skill level
Prior Experience with Simulation
Description
n (%)
First-year internal medicine residents
12 (16.2)
Registered nurses
32 (43.2)
Respiratory therapists
17 (23.0)
Patient care technicians
13 (17.6)
American Heart Association training
30 (40.5)
Red Cross training
10 (13.5)
Skills demonstrationVno class
23 (31.1)
Other
11 (14.9)
None
9 (12.2)
Once, but watched
16 (21.6)
1Y4 Codes
20 (27.0)
>5 Codes
29 (39.2)
Novice
11 (14.9)
Beginner
27 (36.5)
Competent
28 (37.7)
Advanced
5 (6.8)
Expert
3 (4.1)
None
41 (55.4)
Yes
33 (44.6)
preYT-TPQ (median, 2.167) to the postYT-TPQ (median, 2.566) (P = .003). No other category from the T-TPQ showed a significant change between the presimulation and postsimulation experience. See Table 3 for the full listing of results. Resuscitation data were reviewed for the year prior to the start of the program and for 10 months after the completion of the program evaluation. The participating medical unit was identified for the program because of the increasing number of resuscitation events that had occurred during this time period. Review of the presimulation code data revealed none of the resuscitation events requiring defibrillation had met the AHA guidelines. Review of the postsimulation resuscitation data for the 10 months after the program showed all patients who required defibrillation received it within the 2 minutes, therefore adhering to the AHA guideline for ventricular fibrillation.
DISCUSSION The words collaboration and teamwork, as defined by a sampling of the participants prior to the start of the project, had different meanings, depending on which professional group was asked. This finding is supported by the work of 152
Reeves and Lewin,22 who found that the word collaboration was understood differently by the members of a healthcare team on an acute-care unit.22 Physicians viewed collaboration as a way to manage and assign tasks to the other members of a healthcare team, rather than as cooperation among the members of a patient-centered team. Interprofessional collaboration is difficult for healthcare professionals because they do not understand or acknowledge the contributions that all healthcare providers bring to the care of a patient. This lack of respect and understanding of roles leads to a decrease in communication and a disregard for the knowledge that all the providers bring to each patient situation.22 One way to build effective healthcare teams is through experiential learning, such as the use of simulated interprofessional scenarios. Sawyer et al23 found significant improvements in communication about errors after implementing an educational program on neonatal resuscitation utilizing TeamSTEPPS, in addition to a simulation program for physicians, nurses, and respiratory therapists. Experiential learning can provide all members of the healthcare team with an understanding of the importance of each individual’s role.23 In addition to gaining an understanding of the various meanings of collaboration, this program evaluation also adds to the body of information on how resuscitation skills deteriorate when staff members do not perform or practice them on a regular basis. As shown in the data collection, 49% of the participants perceived that they were competent in resuscitation skills prior to participating in the simulations. This is in contrast to the participants’ actual skill competence, where only 35% of the groups defibrillated the patient and only 1 group accomplished this during the AHA-recommended time. Using Hunziker and colleagues’12 study, which defined the 2 team-performance markers of hands-on time for compressions and time to defibrillate, it can be reasonably assumed that the 17 teams were poorly developed and would therefore benefit from formal team training on a regular basis. In addition to team training, individual skills were assessed during the simulations, and it was found that most of the participants demonstrated poor-quality compressions Table 2. Mean Scores From Simulation
Evaluation (n = 69) The Learning Environment Is Conducive to:
Mean (SD)
Applying previous knowledge
1.67 (0.560)
Acquiring new knowledge
1.51 (0.699)
Training as a team
1.45 (0.530)
Reinforcing effective communication
1.35 (0.538)
Recognizing a deteriorating patient
1.75 (0.914)
Scale: 1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree.
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 2. Categories of responses from the simulation evaluation item: ‘‘List what you learned today that will be most useful to you while caring for patients.’’ (n = 69).
during the resuscitation. Prior to the start of the simulations, the PI investigated the training policies of the organization, and it was found that nurses working in nonYcritical care areas did not have to maintain AHA Basic Life Support Certification. The policy required every nurse to demonstrate the skill on an annual basis but did not require nonYcritical care nurses to complete the didactic theory behind the skill. The poor quality of chest compressions performed during the simulations during this program evaluations supported the need for a change in this policy. Currently, every registered nurse working in the organization, despite the setting in which they work, is required to maintain AHA Basic Life Support certification. The postYT-TPQ showed a change in the leadership category of the tool. One possible explanation for this change may be the leadership changes that occurred on the unit rather than the participation in the program evaluation. Because the participants did not complete the TeamSTEPPS program, they may have lacked the knowledge of the meaning of the terms used in the T-TPQ measurement. This is 1 limitation to using this T-TPQ tool in the program evaluation; however, the results are an assessment of the knowledge gap that exists with this team. When comparing the results from this program evaluation to a meta-analysis of studies that used simulation to train resuscitation teams, it is evident that more research is needed on the topic of simulation training. Mundell et al,24 in their meta-analysis of using simulation for resuscitation training, found 21 studies that showed simulation-based training is more effective when compared with providing no educational intervention. Furthermore, the studies suggested that simulation-based training was slightly more effective than other training methods for team training or process training, but not for increasing knowledge or to improve procedure times.24 The results of this program evaluation are based on 1 simulated experience for each team. Team training has been successfully implemented in the commercial airlines busiClinical Nurse Specialist
ness and the military, 2 high-risk areas. However, the Federal Aviation Administration has standardized the training that all cockpit crews must complete and requires 6 hours of training on leadership, crew resource management, and the culture of safety.25 To be competent to fly, each team member must not only have a required amount of team training hours each year, but must also demonstrate both technical and behavior-based skills.25 Salas et al26 reviewed studies on team training, concluding that there was limited evidence available that links team training to positive patient outcomes. The authors further stated that more robust program evaluations are needed to determine the full value of team training. Whereas there are studies that have shown a translation of skills from simulation-based task-training courses to improved patient outcomes, there are no longitudinal studies on how teamwork developed in simulation experiences has translated to improved patient care.27 Currently, there are several studies that support offering a series of additional simulation-based training sessions to enhance retention of skills, but the recommended timing and frequency of additional sessions have not been studied.28
FIGURE 3. Categories of responses from the simulation evaluation question,‘‘For future sessions, what would improve the simulation experience?’’ (n = 69).
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
153
Feature Article Table 3. Median Scores and Wilcoxon Signed Ranks Test for TeamSTEPPS Teamwork Perception Tool (n = 54) TeamSTEPPS Teamwork Perceptions Questionnaire Category
Presimulation Score Median
Postsimulation Score Median
Wilcoxon Signed Rank Test P
Team structure
2.13
2.11
.721
Leadership
2.167
2.566
.003a
Situational monitoring
2.188
2.21
.947
Communication
2.06
2.059
.887
Mutual support
2.15
2.05
.418
Scale: 1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree. a Rejects the null hypothesis.
This evaluation program supports using a unit-based CNS to review the resuscitation data prior to implementation of an educational program and to follow longitudinal data regarding the effectiveness of the simulation-training sessions, based on their timing and frequency. Implementation of a unit-based program, overseen by the unit CNS, can better define the educational needs of the staff and determine how often the simulations should take place as an intervention to improve staff performance. Given the fact that 25% of the participants stated that they would like to participate in a simulated resuscitation more often to help them improve their skills, multiple opportunities to participate in unit-based team simulations should be scheduled.
LIMITATIONS Because this program evaluation was a pilot project on 1 medical unit with a convenience sample size of 84 participants, generalizability to other units or hospitals is 1 limitation. Forty percent of the sample participants were registered nurses, whereas only 20% were internal-medicine residents. Although this representation of staff is close to the hospital environment, it does pose a limitation because the median results of the T-TPQ were influenced by 1 profession completing 43% of the surveys. The demographic survey and the Simulation Evaluation Tool were created by the author based on several other tools and were not tested for validity or reliability prior to the program evaluation. Another limitation of the study was the lack of interrater reliability, in that the primary investigator collected the observed data on the Expected Behaviors tool and validated the results by reviewing the videos of each simulation. APPLICATION TO CNS PRACTICE Within the 3 spheres of influence in which a CNS practices, which are the patient, the nurse, and the system, this program evaluation provides the unit-based CNS with the leadership and assessment data to implement an interprofessional educational program for a unit-based team. Sustaining this 154
program could be a measurable quality-improvement project for the unit-based CNS. Developing trusting relationships with staff and other members of the professional team by role modeling, coaching, and mentoring provides the unitbased CNS with opportunities to work with staff to improve patient outcomes for the patient population. The CNS understands the clinical needs of the patients and is able to assess the knowledge, skills, and attitudes required by staff members to address these clinical needs. When there are deficits in the knowledge, skills, and attitudes needed by staff members to safely care for their patients, the unit-based CNS can develop training programs to meet the ongoing needs of team members. Collaborating with physicians and other members of the healthcare team strengthens the role of the unit-based CNS in bridging the gap between evidence-based practice and the patient outcomes identified in quality measures. The unit-based CNS who participated in the program evaluation used this experience as a staff learning assessment to identify knowledge and skill gaps. This assessment data were used to create specific training around tasks that participants could not perform during the simulation program. In addition, this program evaluation developed a mentoring model by teaching the unit-based CNS how to run a simulated scenario and established the foundation for the next training phase for implementing the training model throughout the system to all unit-based CNSs. The other system CNSs will be taught how to incorporate simulation into their unit training programs. The second phase of the model is in the development stages and will focus on the skills needed to debrief a simulation. Although there was not an evaluation of the CNS as a simulation expert, this presents an interesting area to explore in the future.
CONCLUSIONS It is evident from national patient-safety data that there is a relationship between adverse events and poor communication among healthcare team members, especially during resuscitation events. In addition, based on nationally reported
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
patient-safety data, it is clear that healthcare professionals need training programs that will shape their attitudes, increase their knowledge about each other’s role, and promote behaviors that are needed to work effectively to deliver safe patient care as a team. Building on this model of utilizing unit-based CNSs to create sustainable, interprofessional training programs is 1 way of translating skills acquired in training to bedside practice. Developing innovative, experiential learning events is just 1 more example of the powerful change agent that exists in the unique role of the CNS.
13.
14. 15. 16.
References 1. Health Grades, Inc. Health Grades Patient Safety Study shows increase in hospital incidents. Wide gaps among states and hospitals. http://www.healthgrades.com/media/dms/2006DHPPSnational releaseFinal.pdf. Accessed July 10, 2012. 2. Institute of Medicine. Keeping Patients Safe: Transforming the Work Environment of Nurses. Washington, DC: National Academies Press; 2004. 3. Boyle S. Nursing unit characteristics and patient outcomes. Nurs Econ. 2004;22(3):111Y119. 4. Agency for Healthcare Research & Quality. Glossary: failure to rescue. Patient Safety Network. http://psnet.ahrq.gov/popup_glossary .aspx?name=failuretorescue. Updated October 2012. Accessed February 9, 2014. 5. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 Lives Campaign: setting a goal and a deadline for improving health care quality. JAMA. 2006;295(3):324Y327. 6. Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506Y2513. 7. The Joint Commission. 2009 Joint Commission Hospital National Patient Safety Goals. https://www.unchealthcare.org/site/Nursing/ servicelines/aircare/additionaldocuments/2009npsg. Published 2009. Accessed October 10, 2012. 8. Mondrup R, Brabrand M, Flokestad L, et al. In-hospital resuscitation evaluated by in situ simulation: a prospective simulation study. Scand J Trauma Resusc Emerg Med. 2011;19(55):1Y6. 9. Jacobs I, Sunde K, Deakin CD, et al. Part 6 Defibrillation: 2010 international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation. 2010;122:S235YS337. 10. Hunt EA, Walker AR, Shaffner DH, Miller MR, Pronovost P. Simulation of in-hospital pediatric medical emergencies and cardiopulmonary arrests: highlighting the importance of the first 5 minutes. Pediatrics. 2008;121:e34Ye43. 11. American College of Cardiology Foundation and American Heart Association, Inc. Methodology Manual and Policies From the ACCF/ AHA Task Force on Practice Guidelines. http://my.americanheart .org/idc/groups/ahamah-public/@wcm/@sop/documents/ downloadable/ucm:319826.pdf. Published 2010. Accessed February 19, 2014. 12. Hunziker S, Tschan F, Semmer NK, et al. Hands-on time during cardiopulmonary resuscitation is affected by the process of
Clinical Nurse Specialist
17.
18. 19.
20. 21.
22. 23.
24. 25.
26. 27. 28.
teambuilding: a prospective randomized simulator-based trial. BMC Emerg Med. 2009;9(3):1Y10. Cummins RO, Chamberlain D, Hazinski MF, et al. Recommended guidelines for reviewing, reporting, and conducting research on in-hospital resuscitation: the in-hospital ‘Utstein style.’ Circulation. 1997;95:2213Y2239. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in-hospital cardiac arrest. N Engl J Med. 2008; 358(1):9Y17. Lemieux-Charles L, McGuire WL. What do we know about health care team effectiveness? A review of the literature. Med Care Re Rev. 2006;63:263Y300. Berg RA, Hemphill R, Abella BS, et al. Part 5 Adult Basic Life Support: 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2010;122:S685YS700. National League for Nursing. SIRC Glossary. Simulation Innovation Resource Center. http://sirc.nln.org/mod/glossary/view.php?id= 183&mode=letter&hook=S&sortkey=&sortorder=. Updated 2013. Accessed February 17, 2014. Ericsson KA, Krampe RT, Tesch-Roemer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100(3):363Y406. King HB, Battles J, Baker DP, et al. TeamSTEPPSi: Team Strategies and Tools to Enhance Performance and Patient Safety (2006). www .ahrq.gov/downloads/pub/advances2/vol3/Advances-King_1.pdf. Accessed February 5, 2014. Laerdal. The first five minutesA curriculum. SimMedical. Published October 2008. American Institutes for Research. TeamSTEPPSA Teamwork Perceptions Questionnaire (T-TPQ) Manual. http://teamstepps.ahrq .gov/Teamwork_Perception_Questionnaire.pdf. Published June 28, 2010. Accessed July, 2012. Reeves S, Lewin S. Interprofessional collaboration in the hospital; strategies and meanings. J Health Serv Res Policy. 2004;9(4): 218Y225. Sawyer T, Laubach VA, Hudak J, Yamamura K, Pocrnich A. Improvements in teamwork during neonatal resuscitation after interprofessional TeamSTEPPS training. Neonatal Netw. 2013 JanYFeb; 32(1):26Y33. Mundell WC, Kennedy CC, Szostek JH, Cook DA. Simulation technology for resuscitation training: a systematic review and metaanalysis. Resuscitation. 2013;84:1174Y1183. Baker D, Gustafson S, Beaubien J, Salas E, Barach P. Medical teamwork and patient safety: the evidence-based relation. Agency for Healthcare Research & Quality. http://www.ahrq.gov/qual/ medteam/. Published July 2005. Accessed December 1, 2013. Salas E, Diaz Granados D, Weaver S, King H. Does team training work? Principles for health care. Acad Emerg Med. 2008;15: 1002Y1009. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc. 2011;6(suppl):S42YS47. Mosley C, Dewhurst C, Molloy S, Shaw BN. BEME guide: what is the impact of structured resuscitation training on healthcare practitioners, their clients and the wider service? A BEME system review: BEME Guide No. 20. Med Teach. 2012;34:e349Ye385.
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
155
Feature Article Creating a Sustainable, Interprofessional-Team Training Program: Initial Results Virginia K. Riggall, DNP, RN, CCRN, ACCNS-AG n Charlene M. Smith, DNS, MSEd, WHNP, RN-BC, CNE, ANEF
Purpose/Objectives: The purpose of this program evaluation was to explore whether incorporating deliberate learning concepts, through the use of simulated patient scenarios to teach interprofessional collaboration skills to a healthcare team on one acute-care hospital unit, would improve the resuscitation response in the first 5 minutes on that unit. Design/Setting: This was a pilot program evaluation utilizing a unit-based, clinical nurse specialist in the deployment of an interprofessional educational program involving simulation on an acute medical floor in a large tertiary-care hospital. Sample: Eighty-four staff members participated in 17 simulations. The sample included first-year internal-medicine residents, registered nurses, respiratory therapists, and patient care technicians. Methods: This was a program evaluation that used the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ) (Classroom slides: TeamSTEPPS essentials; http://www.ahrq.gov/professionals/ education/curriculum-tools/teamstepps/instructor/essentials/ slessentials.html#s3) during the presimulation/postsimulation sessions to assess the participants’ perceptions of teamwork. Expected intervention behaviors were collected through observations of participants in the simulations and compared with the American Heart Association guidelines (Circulation 2010;122:S685YS670, S235YS337). Common perceptions of participants regarding the experience were obtained through open-ended evaluation questions. Author Affiliations: Clinical Nurse Specialist, Department of Clinical Education and Nursing Research at Rochester General Health System (Dr Riggall); and Full-time Professor and Undergraduate Cochair at Wegmans School of Nursing, St John Fisher College; Research Coordinator, University of Rochester Medical Center, Highland Hospital; and Nursing Professional Development Specialist, Strong Memorial Hospital (Dr Smith), Rochester, New York. The authors report no conflicts of interest. Correspondence: Virginia K. Riggall, DNP, RN, CCRN, ACCNS-AG, 3178 Valley Dr, Walworth NY 14568 ([email protected]). DOI: 10.1097/NUR.0000000000000121
Clinical Nurse Specialist
Results: Fifty-three participants completed the preY and postYT-TPQ. Mean scores in the leadership category of T-TPQ decreased significantly (P = .003) from the pretest (median, 2.167) to the T-TPQ posttest (median, 2.566). Only 35% of the groups administered a defibrillation during the ventricular fibrillation simulation scenario, and only 1 group delivered this shock within the American Heart Association’s recommended time frame of 2 minutes (Circulation 2010;122:S235YS337). Conclusion: A single resuscitation simulation was not enough interventional dosage for staff to improve the resuscitation process. A longitudinal study should be conducted to determine the effectiveness of the program after staff members have repeated the program multiple times. Implications: A unit-based quality-improvement simulation training program could help improve the first-5-minute response and resuscitation skills of staff by increasing the frequency of unit-based training overseen by the unit’s clinical nurse specialist. KEY WORDS: clinical nurse specialist, collaboration, interprofessional, resuscitation, simulation training
A
cute-care medical units are composed of fluid teams, changing frequently, with floating healthcare staff members used to fill unit staffing needs or medical residents rotating units every 3 to 6 weeks. Developing and offering training programs for constantly changing teams are a challenge for centralized, hospital-based staff-development departments, although potentially more feasible for decentralized, unit-based clinical nurse specialists (CNSs). Unit-based qualityimprovement programs aimed at improving the first-5-minute response and cardiac resuscitation skills of interprofessional healthcare providers can be deployed with the assistance of an educational interprofessioal simulation program intervention developed by a system-based CNS for consistent
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
147
Feature Article deployment across the organization and supported by the unit-based CNS. Continuous efforts at team building to reduce delayed resuscitation efforts can be sustained through interprofessional training programs involving simulation conducted on the unit-by-unitYbased CNSs.
BACKGROUND In 2011, Health Grades1 reported that there were 13 preventable patient-safety events from 2007 through 2009, costing $7.3 billion and resulting in 79 670 potentially preventable deaths. The Institute of Medicine2 has suggested that one way to keep patients safe is to develop and support a highly trained, collaborative, interdisciplinary team. Studies have shown a relationship between adverse events, such as failure to rescue, and team collaboration.3 Failure to rescue is a measurement used by the Agency of Healthcare Research and Quality4 to determine the degree to which providers responded to a hospital-acquired condition, such as a cardiac arrest, where the event causes death or permanent disability. This measurement reflects the quality of care patients receive by looking at the type of patient monitoring that occurred and how effective the healthcare professionals’ response was to the change in the patient’s condition.4 LITERATURE REVIEW In 2006, the Institute for Healthcare Improvement’s initiative, the 100 000 Lives Campaign, was created to help reduce hospital failure to prevent deaths through the creation of rapidresponse teams in hospitals.5 Despite the creation of these teams, failure to rescue patients continues because the healthcare professionals caring for patients are unable to identify the symptoms of deterioration and do not seek timely help to correct the patient’s change in condition.5 In 2008, Chan et al6 reviewed 24 193 patient charts prior to the implementation of rapid-response team training and 24 978 patient charts after implementation in a large tertiarycare academic hospital to determine if there was a decrease in hospital mortality after implementation of the program. The retrospective chart review revealed that there was not a decrease in cardiac arrests, nor was there a decrease in hospital mortality for patients with a cardiac arrest after implementation of a rapid-response team.6 In 2009, in response to the study of Chan et al, the Joint Commission established the National Patient Safety Goal 16, which required hospitals to review their current early-recognition and response programs to improve survival for patients with a cardiac arrest.7 Interestingly, development of rapid-response teams in hospitals may lead to deterioration in cardiopulmonary resuscitation (CPR) skills for nurses working in acute-care units. Mondrup et al8 found a significant difference between the CPR skill level of first responders in hospitals and the CPR skill level of members of the rapid-response team. The authors determined that the first responders’ CPR skills deteriorated quickly because they did not perform CPR on a 148
regular basis, whereas members of the cardiac resuscitation team performed higher-quality chest compressions during CPR and had shorter times of no blood flow than did those nurses who responded first to a code.8 The authors found that patients with a cardiac arrest, who did not receive care until the rapid-response team arrived, were not defibrillated for 3.5 minutes.8 The European Resuscitation Council and the American Heart Association (AHA)8,9 currently recommend that a patient with a cardiac arrest should be defibrillated within 2 minutes for best survival outcomes. However, Hunt et al10 reported that staff members, usually nurses, who found patients unresponsive were found to be spending too much time preparing the room for the code team and did not start CPR before the code team arrived. In a witnessed cardiac arrest, immediate defibrillation for ventricular fibrillation or pulseless tachycardia is a class 1 recommendation according to the AHA.9 A class 1 recommendation has been defined by the AHA as a treatment or procedure that should be performed because the benefits outweigh the risks.11 In a study by Hunziker et al,12 100 teams, composed of 3 randomly chosen physicians and 1 nurse, were asked to participate in a simulated cardiac arrest. Half of the teams had undergone some team-building training prior to the simulated exercise. The researchers found that 36% of the teams without any team training delayed their first defibrillation beyond 2 minutes, compared with 12% of the teams that had completed team training in a simulated environment.12 In another study, data from the National Registry of Cardiopulmonary Resuscitation,13 related to ventricular fibrillation and pulseless ventricular tachycardia, revealed that 30% of the patients who experienced cardiac arrest due to ventricular arrhythmia were not defibrillated in 2 minutes or less.14 The researchers found a correlation between decreased survival rates and delay in defibrillation beyond 2 minutes.14 Delay in defibrillation may be attributable, in part, to failure to develop team skills.12,14 The National Registry of Cardiopulmonary Resuscitation is the largest registry in the United States that uses standardized definitions for measuring the care and outcomes during cardiac arrests that occur in hospitals.13 Assigning roles to each member of the team is an additional task that must be accomplished during a cardiac arrest; thus, delaying this essential step can lead to delaying the 2 evidence-based lifesaving measures involving early effective compressions and defibrillation.12,14 Hands-on time for compressions and time to defibrillation are considered 2 team-performance markers for CPR.12 It is suggested that nontechnical skills, such as communication and teamwork, can impact patient outcomes during cardiac arrests. Highly effective teams are associated with coordinated team leadership roles and clarity in responsibilities immediately following a cardiac arrest.12 Conversely, hands-on time for CPR and time to defibrillation are negatively affected by poorly developed teams.12 Lemieux-Charles and McGuire15 found that highly functioning teams were
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
characterized by positive communication patterns, low levels of conflict, and high levels of collaboration, coordination, cooperation, and participation. Poor outcomes during lifesaving measures were attributed to poor team building, a task that must be completed quickly by teams during patient emergencies.12 Team performance can improve through deliberate practice with the use of simulated scenarios performed in a safe environment with a human-patient simulator. The National League for Nursing defines simulation as ‘‘a technique that uses a situation or environment created to allow persons to experience a representation of a real event for the purpose of practice, learning, evaluation, or testing or to gain understanding of systems or human actions.’’16,17 Deliberate practice incorporates highly structured activities and monitoring of specific skills, such as CPR skills, by which the participants receive real-time feedback with the goal of improving skill levels.18 In addition to using simulation scenarios to improve resuscitation processes, programs such as Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) can be incorporated into training programs to develop the nontechnical skills such as communication and teamwork. TeamSTEPPS is an evidence-based program that was developed by the Agency for Healthcare Research and Quality and the Department of Defense to provide organizations with core competencies for trainable skills such as leadership, situation monitoring, mutual support, and communication.19 Incorporating TeamSTEPPS concepts into a simulation scenario provides staff with hands-on practice of resuscitation skills and an opportunity to reflect on how team collaboration, communication, performance, and behaviors impact patient outcomes. The pilot program evaluation, entitled Improving Interprofessional Collaboration Using Deliberate Practice, was developed to improve team collaboration and resuscitation skills in the first 5 minutes of a cardiac arrest. The purpose of the program evaluation was to incorporate deliberate learning concepts, through the use of a simulated patient scenario, to teach interprofessional collaboration skills to the healthcare team on 1 acute-care hospital unit in order to determine if the training program could improve the team response to cardiopulmonary arrest and the patient resuscitation process. The program was the first phase of a pilot project to mentor unit-based CNSs on how to use simulation as a strategy for teaching interprofessional collaboration to their unit team members so they may sustain collaborative, effective teams that provide safe patient care.
METHODS Design, Setting, Sample This was a pilot program evaluation that was developed to look at whether using simulation as an interprofessional teaching strategy would change the perceptions of healthcare providers about interprofessional collaboration on one Clinical Nurse Specialist
40-bed medical unit in a northeastern tertiary-care teaching hospital. Before initiating the program evaluation, resuscitation data for the prior year were reviewed for all acute-care units in the hospital; the unit with the largest number of resuscitation events was selected for the program. The resuscitation data were reviewed by the unit-based CNS and the principal investigator (PI) to identify the gaps in staff performance during resuscitation events. For example, the times to defibrillation were significantly longer than the AHA recommendation of 2 minutes. First-year internal-medicine residents, nurses, patient care technicians, and respiratory therapists who worked on the medical unit were invited to participate in the pilot program. The program was implemented at the end of the first residency year while the group was transitioning into their second year of residency. Second-year residents are required to take a leadership role in resuscitation events. The director of the residency program chose this group to prepare the residents with an educational opportunity to practice leadership skills required during resuscitation events. The simulation scenario was selected from The First 5 Minutes program to provide a standardized scenario for all participants on responding to a cardiopulmonary arrest. The First 5 Minutes program was developed by Laerdal Medical Corporation20 and the University of Pittsburgh Medical Center as an educational program for nurses, respiratory therapists, and other healthcare team members. Prior to beginning the simulation sessions with the staff, 4 CNSs, including the unit-based CNS from the pilot medical floor, participated in the simulation scenario. Each CNS completed the program evaluations, including an additional evaluation called the Pilot Simulation Evaluation Tool. This tool provided the PI with feedback and suggestions prior to the implementation of staff simulations. Including the unit-based CNSs in the pilot simulation was the beginning of a partnership between the centralized-CNS educator, the PI, and the unit-based CNSs. After the pilot simulation, the PI mentored the unit-based CNS on the presimulation planning by developing a simulation schedule that would take into consideration the unit’s staffing patterns while allowing enough staff to participate in each simulation to make the experience realistic. Once dates were identified, the unit-based CNS scheduled staff members for the simulations and coordinated time for the registered nurses and the patient care technicians to complete their presimulation questionnaires. The PI communicated the schedule to the residency program administrator and the respiratory care manager. Once the schedule was completed, the PI and the unit-based CNS customized The First 5 Minutes scenario to create a common patient scenario that could occur on the unit. The scenario was developed around the gaps in knowledge and skills that were identified from reviewing the resuscitation data for the unit. Prior to running the simulations, the PI trained the unit-based CNS on how to set up and run the
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
149
Feature Article human patient simulator and the video equipment. The training goal during this phase was to allow the unit-based CNS the opportunity to focus on the coordination and technical skills of running a simulation. While the PI did mentor the unit-based CNS on how to facilitate the simulation prebriefs and the postsimulation debriefing, it was decided that the PI would facilitate the simulation prebriefs and the postsimulation debriefings to maintain content consistency with each group. After each simulation, the unit-based CNS and the PI reviewed the simulated event for additional staff educational needs, such as how to set up and use a defibrillator. Many participants did not have the experience or skills to set up the defibrillator and deliver an electrical shock through either the manual defibrillation mode or by using automatic electronic defibrillator. Ninety staff members were invited to participate in the program evaluation, and 84 staff members participated in 17 simulations. The participants who completed the simulations included 15 first-year internal-medicine residents starting their second-year, 17 respiratory therapists, 15 patient care technicians, and 37 registered nurses. The sample group represented 75% of the hospital’s first-year internal medical residents (85% were invited to participate), 23% of the hospital’s respiratory therapy staff (100% invited participated), 90% of the unit’s patient care technicians (100% invited participated), and 82% of the unit’s staff nurses (100% invited participated). Approval was obtained from the institutional review board, and all study participants gave voluntary informed consent for the program evaluation data to be analyzed and published. Instruments and Measurements Prior to initiating the simulated scenarios, the PI developed a short questionnaire to assess staff members’ perceptions of teamwork/team functioning and communication on the unit. Content analysis of the narrative responses was conducted and recorded based on common themes. One-word responses were removed from the analysis. The purpose of the questions was to provide the PI with a baseline understanding of the team culture that existed on the unit prior to the project. Each participant was asked to complete a demographic survey, which included questions about the type of CPR training each participant had received prior to the simulation. Participants responded using a 5-point Likert scale (strongly agree = 1, agree = 2, neutral = 3, disagree = 4, strongly disagree = 5). The TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ), a valid and reliable tool created for hospitals by the Agency for Healthcare Research and Quality and the Department of Defense,21 was administered to the participants prior to the simulation, using the same 5-point Likert scale. The T-TPQ measures an individual’s perception of the level of teamwork that exists within a group.21 The T-TPQ reliability was determined for each of the 5 constructs: team structure, leadership, situation monitoring, mutual support, and communication. Cronbach’s ! coefficients for the 150
multi-item variables ranged from .88 (communication) to .95 (leadership). Content validity was obtained by a sample of 169 TeamSTEPPS participants who reviewed the questionnaire. Convergent validity was established with the Agency for Healthcare Research and Quality’s Hospital Survey on Patient Safety, a tool that measures patient-safety culture. The correlations with Hospital Survey on Patient Safety ranged from .60 (communication) to .81 for the complete TeamSTEPPS Teamwork Perception Questionnaire.21 Each simulation was videotaped to validate all observational data collected for the program evaluation. Expected behaviors, which were defined as essential resuscitation interventions recommended by AHA, were collected by the primary investigator as observational data. The timing of participants’ observed interventions was compared with the AHA recommendations,16 as noted in Figure 1. The observational data for the expected interventions were confirmed for accuracy by reviewing each simulation video. At the conclusion of the simulation, each participant was asked to complete an electronic evaluation regarding their perceptions of the learning experience. Six months after the completion of the simulations, each participant was sent an e-mail inviting them to complete a postsimulation T-TPQ questionnaire. Seventy-seven percent of the participants who completed the presimulation T-TPQ completed the postsimulation T-TPQ.
RESULTS Content analysis was used to categorize the responses from the following questions prior to simulations; ‘‘How would you describe the environment in which healthcare teams work on your unit?’’ and ‘‘How does the team communicate about patient-safety risks?’’ Responses included a contrast between recognition of the importance of working as a team for the benefit of the patient and the absence of interest in creating effective teams. One unit nurse stated that ‘‘the nurses work well as a team, but there is disconnection between the nurses and the patient care technicians.’’ On the other hand, the patient care technicians’ descriptions of the unit varied from the staff being ‘‘a well-oiled machine’’ to the teams being dependent on characteristics of the staff members who were working during a particular shift. Nurses stated that communication about patient-safety events occurred during staff huddles during the shift, but there was no mention of communication about patient-safety events during handoffs or shift reports. Interestingly, a respiratory therapist stated that communication was very weak on the floor in respect to patient safety, and another respiratory therapist commented that the physicians were often dismissive of the nurses’ input on patient care. Conversely, the internal-medicine residents stated that the nurses and patient care technicians worked well together to care for the patients. All quantitative data were analyzed with descriptive statistics usingIBMSPSSStatisticsversion22(THINKEDU,Carrollton,Texas).
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 1. Expected behaviors checklist.
Seventy-four participants completed the demographic survey. Thirty-nine percent of the participants responded that they had been involved in more than 5 resuscitation events in their career, yet 49% of the participants felt their resuscitation skills were at least at a competent level. Only 54% of the participants who completed the demographic survey stated they had attended formal basic life support training through the AHA or the American Red Cross, and 44.6% of the participants stated they had some critical-care experience. The complete sample demographics compiled from the demographic survey are listed in Table 1. The Expected Behaviors data, which were collected during the simulations by the PI and confirmed by viewing the simulation videos, revealed only 35% of the groups administered a defibrillation during the ventricular-fibrillationresuscitation scenario and that only 1 group delivered this shock within the AHA’s recommended time frame of 2 minutes.9 Twenty-nine percent of the groups assigned roles during the simulations, and 18% used closed-loop communication for medications during the scenario. The AHA recommendation for starting chest compressions is 10 seconds from the moment a person is found unresponsive and pulseless. In the 17 groups completing the scenarios, the average time to start compressions was 62.5 seconds. Postsimulation data from the participants were collected using both quantitative data and open-ended questions. The Simulation Evaluation Tool included questions using a 5-point Clinical Nurse Specialist
Likert scale and 2 narrative questions. Ninety-eight percent of the participants either strongly agreed or agreed that the simulated environment emphasized the importance of training as a team, and 97% of the participants felt the training reinforced the importance of effective communication. Mean scores from the simulation evaluations are summarized in Table 2. Narrative responses from the question, ‘‘List what you learned today that will be most useful to you while caring for patients,’’ were collected with common responses noted. Thirty-six percent of the participants stated that communicating more effectively with other team members during resuscitation events would be most useful while caring for patients. Figure 2 lists the major themes from this question. A second narrative question asked the participants for suggestions that would have improved the learning experience. Thirty-three percent of the participants suggested a more realistic environment would have improved the experience, and 23% suggested increasing the frequency of simulation experiences. Figure 3 summarizes all the major themes from this question. Data from the preY and postYT-TPQ were analyzed using a nonparametric test, the Wilcoxon signed rank test. The Wilcoxon signed rank test is used frequently to measure changes in attitudes in a single sample. The comparison of the T-TPQ results indicated that the participants who took part in the program evaluation felt there had been a change in their perception about leadership from the
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
151
Feature Article Table 1. Sample Demographic Characteristics
(n = 74) Category Discipline
Basic life support training
Participation in actual code
Perception of resuscitation skill level
Prior Experience with Simulation
Description
n (%)
First-year internal medicine residents
12 (16.2)
Registered nurses
32 (43.2)
Respiratory therapists
17 (23.0)
Patient care technicians
13 (17.6)
American Heart Association training
30 (40.5)
Red Cross training
10 (13.5)
Skills demonstrationVno class
23 (31.1)
Other
11 (14.9)
None
9 (12.2)
Once, but watched
16 (21.6)
1Y4 Codes
20 (27.0)
>5 Codes
29 (39.2)
Novice
11 (14.9)
Beginner
27 (36.5)
Competent
28 (37.7)
Advanced
5 (6.8)
Expert
3 (4.1)
None
41 (55.4)
Yes
33 (44.6)
preYT-TPQ (median, 2.167) to the postYT-TPQ (median, 2.566) (P = .003). No other category from the T-TPQ showed a significant change between the presimulation and postsimulation experience. See Table 3 for the full listing of results. Resuscitation data were reviewed for the year prior to the start of the program and for 10 months after the completion of the program evaluation. The participating medical unit was identified for the program because of the increasing number of resuscitation events that had occurred during this time period. Review of the presimulation code data revealed none of the resuscitation events requiring defibrillation had met the AHA guidelines. Review of the postsimulation resuscitation data for the 10 months after the program showed all patients who required defibrillation received it within the 2 minutes, therefore adhering to the AHA guideline for ventricular fibrillation.
DISCUSSION The words collaboration and teamwork, as defined by a sampling of the participants prior to the start of the project, had different meanings, depending on which professional group was asked. This finding is supported by the work of 152
Reeves and Lewin,22 who found that the word collaboration was understood differently by the members of a healthcare team on an acute-care unit.22 Physicians viewed collaboration as a way to manage and assign tasks to the other members of a healthcare team, rather than as cooperation among the members of a patient-centered team. Interprofessional collaboration is difficult for healthcare professionals because they do not understand or acknowledge the contributions that all healthcare providers bring to the care of a patient. This lack of respect and understanding of roles leads to a decrease in communication and a disregard for the knowledge that all the providers bring to each patient situation.22 One way to build effective healthcare teams is through experiential learning, such as the use of simulated interprofessional scenarios. Sawyer et al23 found significant improvements in communication about errors after implementing an educational program on neonatal resuscitation utilizing TeamSTEPPS, in addition to a simulation program for physicians, nurses, and respiratory therapists. Experiential learning can provide all members of the healthcare team with an understanding of the importance of each individual’s role.23 In addition to gaining an understanding of the various meanings of collaboration, this program evaluation also adds to the body of information on how resuscitation skills deteriorate when staff members do not perform or practice them on a regular basis. As shown in the data collection, 49% of the participants perceived that they were competent in resuscitation skills prior to participating in the simulations. This is in contrast to the participants’ actual skill competence, where only 35% of the groups defibrillated the patient and only 1 group accomplished this during the AHA-recommended time. Using Hunziker and colleagues’12 study, which defined the 2 team-performance markers of hands-on time for compressions and time to defibrillate, it can be reasonably assumed that the 17 teams were poorly developed and would therefore benefit from formal team training on a regular basis. In addition to team training, individual skills were assessed during the simulations, and it was found that most of the participants demonstrated poor-quality compressions Table 2. Mean Scores From Simulation
Evaluation (n = 69) The Learning Environment Is Conducive to:
Mean (SD)
Applying previous knowledge
1.67 (0.560)
Acquiring new knowledge
1.51 (0.699)
Training as a team
1.45 (0.530)
Reinforcing effective communication
1.35 (0.538)
Recognizing a deteriorating patient
1.75 (0.914)
Scale: 1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree.
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 2. Categories of responses from the simulation evaluation item: ‘‘List what you learned today that will be most useful to you while caring for patients.’’ (n = 69).
during the resuscitation. Prior to the start of the simulations, the PI investigated the training policies of the organization, and it was found that nurses working in nonYcritical care areas did not have to maintain AHA Basic Life Support Certification. The policy required every nurse to demonstrate the skill on an annual basis but did not require nonYcritical care nurses to complete the didactic theory behind the skill. The poor quality of chest compressions performed during the simulations during this program evaluations supported the need for a change in this policy. Currently, every registered nurse working in the organization, despite the setting in which they work, is required to maintain AHA Basic Life Support certification. The postYT-TPQ showed a change in the leadership category of the tool. One possible explanation for this change may be the leadership changes that occurred on the unit rather than the participation in the program evaluation. Because the participants did not complete the TeamSTEPPS program, they may have lacked the knowledge of the meaning of the terms used in the T-TPQ measurement. This is 1 limitation to using this T-TPQ tool in the program evaluation; however, the results are an assessment of the knowledge gap that exists with this team. When comparing the results from this program evaluation to a meta-analysis of studies that used simulation to train resuscitation teams, it is evident that more research is needed on the topic of simulation training. Mundell et al,24 in their meta-analysis of using simulation for resuscitation training, found 21 studies that showed simulation-based training is more effective when compared with providing no educational intervention. Furthermore, the studies suggested that simulation-based training was slightly more effective than other training methods for team training or process training, but not for increasing knowledge or to improve procedure times.24 The results of this program evaluation are based on 1 simulated experience for each team. Team training has been successfully implemented in the commercial airlines busiClinical Nurse Specialist
ness and the military, 2 high-risk areas. However, the Federal Aviation Administration has standardized the training that all cockpit crews must complete and requires 6 hours of training on leadership, crew resource management, and the culture of safety.25 To be competent to fly, each team member must not only have a required amount of team training hours each year, but must also demonstrate both technical and behavior-based skills.25 Salas et al26 reviewed studies on team training, concluding that there was limited evidence available that links team training to positive patient outcomes. The authors further stated that more robust program evaluations are needed to determine the full value of team training. Whereas there are studies that have shown a translation of skills from simulation-based task-training courses to improved patient outcomes, there are no longitudinal studies on how teamwork developed in simulation experiences has translated to improved patient care.27 Currently, there are several studies that support offering a series of additional simulation-based training sessions to enhance retention of skills, but the recommended timing and frequency of additional sessions have not been studied.28
FIGURE 3. Categories of responses from the simulation evaluation question,‘‘For future sessions, what would improve the simulation experience?’’ (n = 69).
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
153
Feature Article Table 3. Median Scores and Wilcoxon Signed Ranks Test for TeamSTEPPS Teamwork Perception Tool (n = 54) TeamSTEPPS Teamwork Perceptions Questionnaire Category
Presimulation Score Median
Postsimulation Score Median
Wilcoxon Signed Rank Test P
Team structure
2.13
2.11
.721
Leadership
2.167
2.566
.003a
Situational monitoring
2.188
2.21
.947
Communication
2.06
2.059
.887
Mutual support
2.15
2.05
.418
Scale: 1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree. a Rejects the null hypothesis.
This evaluation program supports using a unit-based CNS to review the resuscitation data prior to implementation of an educational program and to follow longitudinal data regarding the effectiveness of the simulation-training sessions, based on their timing and frequency. Implementation of a unit-based program, overseen by the unit CNS, can better define the educational needs of the staff and determine how often the simulations should take place as an intervention to improve staff performance. Given the fact that 25% of the participants stated that they would like to participate in a simulated resuscitation more often to help them improve their skills, multiple opportunities to participate in unit-based team simulations should be scheduled.
LIMITATIONS Because this program evaluation was a pilot project on 1 medical unit with a convenience sample size of 84 participants, generalizability to other units or hospitals is 1 limitation. Forty percent of the sample participants were registered nurses, whereas only 20% were internal-medicine residents. Although this representation of staff is close to the hospital environment, it does pose a limitation because the median results of the T-TPQ were influenced by 1 profession completing 43% of the surveys. The demographic survey and the Simulation Evaluation Tool were created by the author based on several other tools and were not tested for validity or reliability prior to the program evaluation. Another limitation of the study was the lack of interrater reliability, in that the primary investigator collected the observed data on the Expected Behaviors tool and validated the results by reviewing the videos of each simulation. APPLICATION TO CNS PRACTICE Within the 3 spheres of influence in which a CNS practices, which are the patient, the nurse, and the system, this program evaluation provides the unit-based CNS with the leadership and assessment data to implement an interprofessional educational program for a unit-based team. Sustaining this 154
program could be a measurable quality-improvement project for the unit-based CNS. Developing trusting relationships with staff and other members of the professional team by role modeling, coaching, and mentoring provides the unitbased CNS with opportunities to work with staff to improve patient outcomes for the patient population. The CNS understands the clinical needs of the patients and is able to assess the knowledge, skills, and attitudes required by staff members to address these clinical needs. When there are deficits in the knowledge, skills, and attitudes needed by staff members to safely care for their patients, the unit-based CNS can develop training programs to meet the ongoing needs of team members. Collaborating with physicians and other members of the healthcare team strengthens the role of the unit-based CNS in bridging the gap between evidence-based practice and the patient outcomes identified in quality measures. The unit-based CNS who participated in the program evaluation used this experience as a staff learning assessment to identify knowledge and skill gaps. This assessment data were used to create specific training around tasks that participants could not perform during the simulation program. In addition, this program evaluation developed a mentoring model by teaching the unit-based CNS how to run a simulated scenario and established the foundation for the next training phase for implementing the training model throughout the system to all unit-based CNSs. The other system CNSs will be taught how to incorporate simulation into their unit training programs. The second phase of the model is in the development stages and will focus on the skills needed to debrief a simulation. Although there was not an evaluation of the CNS as a simulation expert, this presents an interesting area to explore in the future.
CONCLUSIONS It is evident from national patient-safety data that there is a relationship between adverse events and poor communication among healthcare team members, especially during resuscitation events. In addition, based on nationally reported
www.cns-journal.com
May/June 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
patient-safety data, it is clear that healthcare professionals need training programs that will shape their attitudes, increase their knowledge about each other’s role, and promote behaviors that are needed to work effectively to deliver safe patient care as a team. Building on this model of utilizing unit-based CNSs to create sustainable, interprofessional training programs is 1 way of translating skills acquired in training to bedside practice. Developing innovative, experiential learning events is just 1 more example of the powerful change agent that exists in the unique role of the CNS.
13.
14. 15. 16.
References 1. Health Grades, Inc. Health Grades Patient Safety Study shows increase in hospital incidents. Wide gaps among states and hospitals. http://www.healthgrades.com/media/dms/2006DHPPSnational releaseFinal.pdf. Accessed July 10, 2012. 2. Institute of Medicine. Keeping Patients Safe: Transforming the Work Environment of Nurses. Washington, DC: National Academies Press; 2004. 3. Boyle S. Nursing unit characteristics and patient outcomes. Nurs Econ. 2004;22(3):111Y119. 4. Agency for Healthcare Research & Quality. Glossary: failure to rescue. Patient Safety Network. http://psnet.ahrq.gov/popup_glossary .aspx?name=failuretorescue. Updated October 2012. Accessed February 9, 2014. 5. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 Lives Campaign: setting a goal and a deadline for improving health care quality. JAMA. 2006;295(3):324Y327. 6. Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506Y2513. 7. The Joint Commission. 2009 Joint Commission Hospital National Patient Safety Goals. https://www.unchealthcare.org/site/Nursing/ servicelines/aircare/additionaldocuments/2009npsg. Published 2009. Accessed October 10, 2012. 8. Mondrup R, Brabrand M, Flokestad L, et al. In-hospital resuscitation evaluated by in situ simulation: a prospective simulation study. Scand J Trauma Resusc Emerg Med. 2011;19(55):1Y6. 9. Jacobs I, Sunde K, Deakin CD, et al. Part 6 Defibrillation: 2010 international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation. 2010;122:S235YS337. 10. Hunt EA, Walker AR, Shaffner DH, Miller MR, Pronovost P. Simulation of in-hospital pediatric medical emergencies and cardiopulmonary arrests: highlighting the importance of the first 5 minutes. Pediatrics. 2008;121:e34Ye43. 11. American College of Cardiology Foundation and American Heart Association, Inc. Methodology Manual and Policies From the ACCF/ AHA Task Force on Practice Guidelines. http://my.americanheart .org/idc/groups/ahamah-public/@wcm/@sop/documents/ downloadable/ucm:319826.pdf. Published 2010. Accessed February 19, 2014. 12. Hunziker S, Tschan F, Semmer NK, et al. Hands-on time during cardiopulmonary resuscitation is affected by the process of
Clinical Nurse Specialist
17.
18. 19.
20. 21.
22. 23.
24. 25.
26. 27. 28.
teambuilding: a prospective randomized simulator-based trial. BMC Emerg Med. 2009;9(3):1Y10. Cummins RO, Chamberlain D, Hazinski MF, et al. Recommended guidelines for reviewing, reporting, and conducting research on in-hospital resuscitation: the in-hospital ‘Utstein style.’ Circulation. 1997;95:2213Y2239. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in-hospital cardiac arrest. N Engl J Med. 2008; 358(1):9Y17. Lemieux-Charles L, McGuire WL. What do we know about health care team effectiveness? A review of the literature. Med Care Re Rev. 2006;63:263Y300. Berg RA, Hemphill R, Abella BS, et al. Part 5 Adult Basic Life Support: 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2010;122:S685YS700. National League for Nursing. SIRC Glossary. Simulation Innovation Resource Center. http://sirc.nln.org/mod/glossary/view.php?id= 183&mode=letter&hook=S&sortkey=&sortorder=. Updated 2013. Accessed February 17, 2014. Ericsson KA, Krampe RT, Tesch-Roemer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100(3):363Y406. King HB, Battles J, Baker DP, et al. TeamSTEPPSi: Team Strategies and Tools to Enhance Performance and Patient Safety (2006). www .ahrq.gov/downloads/pub/advances2/vol3/Advances-King_1.pdf. Accessed February 5, 2014. Laerdal. The first five minutesA curriculum. SimMedical. Published October 2008. American Institutes for Research. TeamSTEPPSA Teamwork Perceptions Questionnaire (T-TPQ) Manual. http://teamstepps.ahrq .gov/Teamwork_Perception_Questionnaire.pdf. Published June 28, 2010. Accessed July, 2012. Reeves S, Lewin S. Interprofessional collaboration in the hospital; strategies and meanings. J Health Serv Res Policy. 2004;9(4): 218Y225. Sawyer T, Laubach VA, Hudak J, Yamamura K, Pocrnich A. Improvements in teamwork during neonatal resuscitation after interprofessional TeamSTEPPS training. Neonatal Netw. 2013 JanYFeb; 32(1):26Y33. Mundell WC, Kennedy CC, Szostek JH, Cook DA. Simulation technology for resuscitation training: a systematic review and metaanalysis. Resuscitation. 2013;84:1174Y1183. Baker D, Gustafson S, Beaubien J, Salas E, Barach P. Medical teamwork and patient safety: the evidence-based relation. Agency for Healthcare Research & Quality. http://www.ahrq.gov/qual/ medteam/. Published July 2005. Accessed December 1, 2013. Salas E, Diaz Granados D, Weaver S, King H. Does team training work? Principles for health care. Acad Emerg Med. 2008;15: 1002Y1009. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc. 2011;6(suppl):S42YS47. Mosley C, Dewhurst C, Molloy S, Shaw BN. BEME guide: what is the impact of structured resuscitation training on healthcare practitioners, their clients and the wider service? A BEME system review: BEME Guide No. 20. Med Teach. 2012;34:e349Ye385.
A
www.cns-journal.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
155
