Innovations

On a Wing and a Prayer: An Assessment of Modularized Crew Resource Management Training for Health Care Professionals

ROBYN CLAY-WILLIAMS, PHD; DAVID GREENFIELD, PHD; JUDY STONE, MSC; JEFFREY BRAITHWAITE, PHD Introduction: Evidence suggests that Crew Resource Management (CRM), a form of team training, is beneficial. In CRM training, participants learn individual portable team skills such as communication and decision making through group discussion and activities. However, the usual 1-day course format is not always compatible with health care organizational routines. A modular training format, while theoretically sound, is untested for interprofessional team training. The aim of this study was to explore the potential for modularized CRM training to be delivered to a group of interprofessional learners. Method: Modularized CRM training, consisting of two 2-hour workshops, was delivered to health care workers in an Australian tertiary hospital. Kirkpatrick’s evaluation model provided a framework for the study. Baseline attitude surveys were conducted prior to each workshop. Participants completed a written questionnaire at the end of each workshop that examined their motivations, reactions to the training, and learner demographics. An additional survey, administered 6 weeks post training, captured self-assessed behavior data. Results: Twenty-three individuals from a range of professions and clinical streams participated. One in 5 participants (22%) reported that they translated teamwork skills to the workplace. While positive about the workshop format and content, many respondents identified personal, team, and organizational barriers to the application of the workshop techniques. Discussion: CRM training when delivered in a modular format has positive outcomes. Following the training, some respondents overcame workplace barriers to attempt to change negative workplace behavior. This progress provides cautious optimism for the potential for modular CRM training to benefit groups of interprofessional health staff. Key Words: Innovative educational interventions, interprofessional education, small group/team learning, Crew Resource Management, modular training

Disclosures: The authors report this research was supported under Australian Research Council’s Linkage Projects funding scheme (project number LP0775514) and National Health and Medical Research Council Program Grant 568612. Ethics committee approval for this project was given by the University of New South Wales Social/Health Research Human Research Ethics Advisory panel, approval number 09-10-006. Dr. Clay-Williams: Postdoctoral Research Fellow in Human Factors, Centre for Clinical Governance Research, Australian Institute of Health Innovation; Faculty of Medicine, University of New South Wales; Dr. Greenfield: Senior Research Fellow, Centre for Clinical Governance Research, Australian Institute of Health Innovation; Faculty of Medicine, University of New South Wales; Ms. Stone: Rehabilitation Physiotherapy Manager, Rehabilitation, Aged and Community Care, Canberra Hospital & Health Services; Dr. Braithwaite: Director, Centre for Clinical Governance Research, Foundation Director, Australian Institute of Health Innovation; Professor, Faculty of Medicine, University of New South Wales. Correspondence: Robyn Clay-Williams, PhD, Faculty of Medicine, University of New South Wales, Sydney NSW 2052, Australia; e-mail: [email protected]. © 2014 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council

Introduction Crew Resource Management in Aviation and Health Care Crew Resource Management (CRM) is a type of team skills training originating 3 decades ago in the field of aviation.1 Initially, CRM programs were implemented to improve communication between pilots and to encourage assertiveness on the part of junior aircrew.2 Subsequently, CRM has evolved into mandatory training in identification and minimization of errors and other threats to safe flight.1,3 Although CRM training is conducted in small to medium groups, the team skills taught are individual and portable. This is important, as in a large airline, pilots may not know each other and are required to form a functional team quickly in order to conduct a safe flight. CRM training is based on the theory that teamwork can create a safer environment and, to manage risk, on Continuing Medical Education, Association for Hospital Medical Education. • Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.21218

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 34(1):56–67, 2014

Assessing Modular CRM Training

it is necessary to both minimize errors and enhance human performance.4,5 Well-functioning teams can assist with both goals. CRM training has been shown to reduce mishaps by 50% to 81% in military aviation.6,7 CRM training has migrated from aviation to health care. Studies have shown improved teamwork attitudes and behaviors of learners, and modest improvements in levels of patient safety to be associated with the training.8,9 Health care CRM training was encouraged in the United States by the 1999 Institute of Medicine (IOM) report To Err Is Human,10 and in the United Kingdom by the 2000 seminal report on patient safety, An Organisation With a Memory.11 Both reports identified the need to focus on a systems approach to error management, and recommended that one element of a systems approach would be the implementation of team training. Inquiries have also recommended team training to improve patient safety.12,13 Health care is similar in some respects yet different in others from aviation. The problems are comparable, in that communication failures contribute to error, and communication problems are associated with poor teamwork.14,15 Poor communication has been identified as one of the contributing factors to an adverse event or close call in 82% of rootcause analyses16 ; in other studies, communication failures have resulted in 91% of reported errors at a 600-bed teaching hospital17 and have been identified as a leading source of adverse events in surgery, being present in 30% to 43% of cases.18 There are similarities between aviation and health care in workplace relationships: staff may know each other but have limited experience working together as a team; they have probably not undertaken team training together; junior staff find it difficult to raise concerns with more senior colleagues;19 and there is limited psychological safety in the workplace.20 Mechanisms for Delivery of Training There are, however, important differences. In health care, for many services and individual professionals, the multiday training format used in aviation is problematic and impractical due to 24-hour shift work and continuous workflows. As a result, elements of training are often attached to the start or end of normal workdays, thereby enabling members of the outgoing and incoming shifts to participate with only a short extension of their day. Suitable training rooms may be multiuse facilities that can be booked for only short periods of time. These constraints have resulted, in some cases, in compression of CRM training, where a full-day course is delivered in as little as 4 hours. In this situation, there is insufficient time for the normal interactive format, and material is generally presented as a didactic lecture.21 While the outcomes of classroom training in general have been shown to

be positive,22–24 the outcomes of compressed versions of the training have proven difficult to quantify.25 Health care team training is usually conducted either in a classroom or patient simulation facility.22–24,26,27 Classroom-based CRM training is typically delivered as a combination of skills training and experiential learning. Teamwork knowledge, skills, and attitudes are explored using case studies, discussion, role play, and video vignettes. Topics include communication, situation awareness, decision making, leadership, managing stress, and coping with fatigue.

Modular Delivery As an alternative to the full-day seminar, classroom-based CRM training in health care may be conducted as a series of individual lessons, or “modules,” delivered over a period of days or weeks. This approach would allow the experiential format, found to be successful in aviation and used in other health care training courses, to be retained, rather than compressing the full-day seminar into a didactic lecture to shorten delivery time. Breaking the CRM curriculum into manageable components may give health professionals more flexibility to attend training. Much CRM training being delivered has already been broken down into a series of lessons; however, each lesson is constructed on previous learning. The more challenging lessons, where concepts are synthesized, are given toward the end of the program after psychological safety has been built. This mode of construction is not essential, and the content of CRM training can be grouped into independent self-contained modules, each addressing specific competencies that do not rely on previous learning. There are advantages to breaking the course into modules. Modular training would allow health professionals to choose those topics that are relevant to their workplace and current level of experience and to add other topics as and when needed. Modules can be selected to bridge knowledge or skill gaps, as refresher or remedial training in individual teamwork skill competencies. While there is a risk that clinicians “do not know what they don’t know” and may therefore choose poorly, selection of training could be guided by supervisors. In addition, phrasing the training options in terms that are relevant to the learners, such as “communicating in chaos: how to get your message across in a complex time critical environment,” for example, will help in molding appropriate choice. Potential problems included determining the optimal duration of each module, creating sufficient psychological safety in each module to encourage full participation and deciding which teamwork competencies to include in each module.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

57

Clay-Williams et al.

Aims While theoretically sound, we do not know the impact and outcomes of modular CRM training. In studies included in systematic reviews of classroom-based team training interventions9,22–24,28,29 and other analyses of the literature,26,30 training duration varied from 4 hours to 3 days. No comprehensive meta-analysis has directly assessed training duration as a mediator of training effectiveness.27 There has been some development of modular training for individual professional colleges31 and for surgical skills simulator training,32 and shortened curricula have been shown to be effective in task-specific team training in a neonatal simulator setting.33 However, this concept does not appear to have been extended to comprehensive interprofessional or generic team training for health care clinicians. Therefore, we conducted an exploratory investigation, using Kirkpatrick’s training evaluation model,34 as this has become the “de facto” method for evaluating CRM training in health care.8 The purpose of this study was to assess the potential of modularized health care CRM training delivered to an interprofessional group of learners to improve participants’ team skills.

and Situational Awareness modules were each delivered in 2-hour workshops to clinicians in a tertiary hospital. The workshops were run on consecutive days. Communication competency was determined to be the ability to communicate efficiently and effectively. Situational awareness competency was defined as the ability to recognize and attribute meaning to workplace occurrences, and to use that understanding to predict the future trajectory of those occurrences. Training techniques were selected to facilitate learning, and encourage practical application of the learning outcomes (TABLE 1). Activities included facilitated discussion of health care teamwork behaviors presented on video vignettes, case studies of adverse events, and role-plays and practice of assertiveness and conflict resolution techniques. Open-source course material, in particular TeamSTEPPS,43 provided valuable access to practical and validated assertiveness and conflict resolution techniques. It was anticipated that the experiential workshop format and evidence-based content would provide appropriate tools and an environment to facilitate “effective discourse”45 to enable learning of communication and situational awareness team skills. Study Approach

Method Study Setting and CRM Training Program This study formed part of a large-scale research project investigating interprofessional learning and interprofessional practice within a bounded autonomous health service in Australia. The research protocol35 and findings36–40 from other aspects of the project have been reported elsewhere. Aviation CRM knowledge, skills, and attitudes were translated to learning outcomes (TABLE 1) for health care. The outcomes were based on a combination of a pretraining needs analysis, a review of the current evidence base for team training, and an expert panel review,41 and incorporated into a modular Healthcare Team Training Program (HTTP).42 The HTTP program consisted of 4 modules, each covering 1 or more competencies: Communication, Situational Awareness, Task Management and Decision Making, and Leadership. The modular program was developed to allow each module to be delivered as a stand-alone session addressing a specific topic. This meant that introductory activities to build psychological safety, and concluding activities to encourage reflection and translation to the workplace, were included with each module. Modules were designed to be delivered in any order, and did not rely on prior participant knowledge. This study evaluated 2 of the 4 HTTP modules: the Communication and Situational Awareness lessons. Case studies have been used in other analyses of medical team training where insufficient resources were available to conduct experimental studies.21 The Communication 58

Kirkpatrick46 identified 4 levels in evaluating a training program: reaction, learning, behavior, and results. This study conducted the first 3 steps; results were not assessed, as the resources required to evaluate clinical and organizational performance were not available. Training program constraints influenced selection of the evaluation tools. Selection criteria were validated instruments covering the broad themes of the program’s knowledge, skills, and attitudes47 ; instruments enabling comparison with other studies; public availability of tools; surveys that were easy for participants to complete; and tools that were simple for researchers to administer and analyze data in line with an exploratory approach. Participants were recruited via open invitation through the health service’s e-mail system. Potential participants were given a description of each workshop that included details of the competencies that each module addressed. Learners elected to attend 1 or both workshops, depending on their perceived need for training. To evaluate learning, a baseline attitude survey consisting of the 6 Teamwork Climate domain questions from the Sexton and colleagues’ Safety Attitudes Questionnaire (SAQ),47 was selected (TABLE 2). The survey, which used a 5-point Likert scale from “strongly disagree” to “strongly agree,” was conducted prior to each workshop and took participants less than 5 minutes to complete. In addition, a questionnaire was developed to examine participants’ reactions to the training and assess behaviors in the workplace. The HTTP (CRM) Training and Behavior Evaluation Questionnaire consisted of 10 questions

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

Assessing Modular CRM Training

TABLE 1. Health Team Training Program Learning Outcomes

Module

Learning outcomes (competencies)

Knowledge

Communication—communicates

●

Discusses the importance of open communication in medicine

●

●

Describes call-out and check-back procedures43

●

efficiently and effectively

●

●

●

●

Situational

Skills

●

Treats others with respect

●

Values the importance of listening

●

Values the importance of resolving conflict in the workplace

Describes a method for asserting a corrective action

Implements strategies to deal with conflict

●

Describes a method for articulating concern about a course of action

Asserts a corrective action in a firm and respectful manner

●

Articulates when concerned about a course of action

●

Communicates critical information in a succinct manner

●

Ensures that team members have a clear picture of the objective

●

Appreciates the importance of maintaining situational awareness

●

Ensures that team members have role clarity and relevant information to achieve goals

●

Appreciates the importance of a team “shared mental model”

●

●

Uses support process to facilitate maintenance of team situational awareness

Displays a positive attitude towards asking and answering questions

●

Displays a positive attitude towards briefing

●

Appreciates the importance of briefing in maintaining a team “shared mental model”

Lists the steps in Situation, Background, Assessment, Recommendation (SBAR)44

●

Describes the concept of ‘situational awareness’, and how to identify when it has been lost

●

Describes the concept of a team “shared mental model”

●

Lists the steps in identifying work environmental or operational threats that could affect safety of the patient

●

Describes a support process to facilitate maintenance of team situational awareness

●

Describes strategies that may be used to maintain individual situational awareness

●

Speaks up when concerned

●

Lists the elements of an effective handover brief

situational awareness

Listens critically and provides feedback to clarify information

●

Describes strategies to deal with conflict

●

Awareness—maintains

Establishes an atmosphere to encourage open communication

Attitudes

●

Asks questions to facilitate understanding of the situation

●

Uses strategies to maintain individual situational awareness

●

Maintains shared mental models via briefing

Lists the elements of a brief

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

59

Clay-Williams et al. TABLE 2. Learning Evaluation Tool: Teamwork Climate Domain Questions*

TABLE 3. Training and Behaviors Evaluation Tool: Healthcare Team Training Program (CRM) Training and Behavior Evaluation Questionnaire

Questions

Aspect evaluated

1. Nurse/allied health input is well received in my workplace setting.

Utility reactions

2. In my workplace setting, it is difficult to speak up if I perceive a problem

1. Have you previously been involved in team training or interprofessional education, as a

with patient/client care.

participant or instructor?

3. Disagreements in my workplace setting are appropriately resolved (ie,

2. Why were you interested in participating in this

not who is right but what is best for the patient/client).

session?

4. I have the support I need from other staff to care for patients/clients.

7. What aspects of the session do you anticipate

5. It is easy for staff in my workplace setting to ask questions when there is something that they do not understand.

Questions

would be useful in your work? Affective reactions

6. The doctors, nurses, and allied health professionals in my workplace

3. What did you find most interesting about the session?

setting work together as a well-coordinated team.

4. What did you enjoy most about the session? 5. What did you find least interesting about the

*Team Climate domain questions from the Safety Attitudes Questionnaire.47

session? 6. What did you find least enjoyable about the session?

examining participants’ utility and affective reactions to the training, and their self-reported workplace behaviors in relation to CRM tools and techniques (TABLE 3). The survey was developed by 2 experienced organizational trainers, based on elements of the Kirkpatrick framework, and was designed to be completed in less than 15 minutes. The instrument was discussed and further refined with input from the broader interprofessional project research team.35 The questionnaire was administered following the completion of the training program, and readministered 6 weeks later. Based on previous studies,48 6 weeks was assessed as adequate to allow enough time for participants to reflect on and begin to implement the skills they had learned. Data Analysis Data were analyzed via descriptive statistics, automated textual analysis, and thematic analysis. Responses to the Team Climate domain questions from the SAQ were analyzed in accordance with published guidelines.47 HTTP (CRM) Training and Behavior Evaluation Questionnaire responses, gathered immediately post training, were transcribed into 1 document and an automated textual analysis of the responses to each question was conducted using Wordle.30 Wordle is an online tool for generating “word clouds” that gives prominence to words that appear more frequently in the source document. It can be used as a research tool to enable analysis of exploratory data,49 and allows general patterns to be identified. Thematic analysis was conducted of the responses to 60

Utility reactions

8. Will you/did you try to apply any of the

(Workplace

techniques taught in the session in your

behavior = Q9)

workplace? 9. What barriers do you expect to/did you experience when applying the techniques/ideas from the session in your workplace? 10. Did you learn anything new, interesting, useful, or unexpected as a result of training as a mixed group?

HTTP (CRM) Training and Behavior Evaluation Questionnaire gathered 6 weeks post training. Two researchers independently read and coded the original text, and searched the data set for patterns of meaning. The researchers discussed their findings, common themes were identified, and differences were resolved by discussion.50

Results Results are presented as follows: characteristics of participants; Teamwork Climate domain attitude mean scores; and the HTTP (CRM) Training and Behavior Evaluation Questionnaire analyzed using automated textual analysis, immediately post training, and also thematically analyzed, at 6 weeks after the workshops.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

Assessing Modular CRM Training

FIGURE 1. Workshop Participants

Characteristics of Participants Twenty-three female health professionals, from medicine (4%), nursing (44%), midwifery (17%), and allied health (35%) participated. Clinical streams represented included medicine (39%), obstetrics (17%), community health (9%), medical education (13%), and other medical (22%). Participants had an average experience of 17.5 (range 1 to 39) years in their clinical stream, and 9.5 (range < 1 to 30) years in their current organization. The Communication workshop attracted 16 learners, and 13 learners completed the Situational Awareness workshop (FIGURE 1). Six participants completed both workshops. No participants withdrew from training, and all learners completed the posttraining questionnaire at the conclusion of each workshop. Of the 13 learners (57%) who responded to the 6-week follow-up questionnaire, 4 participated in the Communication workshop, 4 participated in the Situational Awareness workshop, and 5 attended both.

Pretraining Teamwork Climate Domain Attitude Questionnaire The baseline attitude survey was conducted prior to commencement of each workshop. The Communication and Situational Awareness workshops’ mean and standard deviation (SD) Teamwork Climate domain scores were 3.97 (0.82) and 4.29 (0.56) respectively. The Teamwork Climate domain scores are typical of those encountered in health care workplaces in Australia,51 the United Kingdom,52 and the United States.47

HTTP (CRM) Training and Behavior Evaluation Questionnaire—Immediate Post Training The resultant posttraining questionnaire Wordle53 is presented in FIGURE 2 to illustrate responses to specific survey questions covering utility reactions (Q2 and Q7), affective reactions (Q3 and Q4, combined), and perceived barriers to training transfer (Q9). We can see from FIGURE 2 that participants primarily attended training to learn tools and strategies to improve general communication and team skills. Training provided an opportunity to practice strategies and obtain different perspectives. The most useful lessons were in communication strategies and techniques. Learners reported that workplace culture and the need for further practice would be impediments to implementing the strategies in the workplace. The topics that learners anticipated would be useful in the workplace, the topics that were most enjoyable, the topics that learners found most interesting, and the reasons why learners attended the training are shown in TABLE 4. Only 1 participant in the communication session, and 2 learners in the situational awareness session did not expect to encounter any barriers to applying the techniques in the workplace, with 2 learners “unsure.” The remainder nominated expected barriers, such as insufficient time to prepare, assemble the team, or brief; the need for regular practice; workplace or team culture; personal lack of confidence; reluctance to challenge seniors; unanticipated or non-textbook responses; and the difficulty of obtaining help when overloaded. Despite identification of barriers, all participants except 1 said that they would try to apply 1 or more of the workshop techniques in their workplace.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

61

Clay-Williams et al.

FIGURE 2. Wordle Clouds: HTTP (CRM) Training and Behavior Questionnaire Responses.

HTTP (CRM) Training and Behavior Evaluation Questionnaire—6-Week Follow-up Analysis of the 6-week follow-up responses identified 4 themes (TABLE 5). They are: value of the training, technique application, perceived effectiveness, and enablers and barriers to their use. 62

The 5 respondents who applied the techniques indicated that they were moderately to very useful. While respondents reported valuing the format and multidisciplinary mix of the training, participant application of the techniques taught in the workshops varied considerably. Techniques were perceived to be more effective and were therefore more likely to

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

Assessing Modular CRM Training TABLE 4. Results—Immediate Post Training

Session

Communication

Issue

Reasons for attending, to improve:

Response (number of responses)

Specific communication skills (1) General communication skills (7) Team communication (2) Interprofessional communications (2) Be more effective in their work team (1) Quality in health care (1)

Topics most interesting:

Sharing experiences with others (3) Discovering tools or strategies to use in resolving difficult workplace communication situations (8)

Topics that provided the greatest enjoyment:

Practice scenarios and exercises in the session (7) Video vignettes (3) Mix of theory and practice (2) “Real” situations presented as examples (2) Variety of participants (2) Professional and inclusive facilitation (1)

Topics most useful for workplace:

Graded assertiveness (8) Conflict resolution (6) “ISBAR” handover tool (4) “Most” or “all of it” (2)

Situational Awareness

Reasons for attending:

To learn something new or interesting (4) To develop or improve specific or general team skills (6) To improve organization or time management (4)

Topics most interesting:

Briefing strategies (5) Situational awareness (7) Shared mental models (1) Method of presentation (1)

Topics that provided the greatest enjoyment:

Interactive group activities (8) Video vignettes (3) Briefing exercise (2)

Topics most useful for workplace:

Situational awareness (6) Shared mental model (4) Briefing strategies (4) “Most” of the material (4)

be adopted when they could be used to better accomplish normal day-to-day processes. Participants reported using their new knowledge to reflect on past behavior. I realized how I may have exacerbated a situation by inappropriate words in comb[ination] with tone and body language [in] both professional and personal situations. Respondent 2

Discussion We set out to investigate the impact and outcomes of modular CRM training to groups of interprofessional health staff in order to determine whether modular training is a feasible method for delivery of teamwork training. The workshop topics and presentation format attracted staff from a diversity

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

63

Clay-Williams et al. TABLE 5. Results—6-Week Follow-up

Theme

Valuing the format and

Issues reported

●

Benefits from the diversity of input from multiple professions and disciplines

●

The opportunity to network with colleagues across clinical streams

●

The normalization of workplace and team difficulties

●

Interest in and surprise at the different perspectives within the health professions

●

Five participants stated they applied the techniques with moderate success, drawing upon:

●

Communication tools, including graded assertiveness

●

Conflict resolution and briefing or information sharing tools

●

Situational awareness techniques, in particular trying to balance workload among the team to improve situational awareness and avoid some of the potential for error associated with individuals being over- or underworked

●

One participant stated the intention to apply the techniques in the future

●

The remaining seven respondents had not applied the techniques

●

Communication techniques that constituted more efficient methods of accomplishing normal healthcare processes (such as the handover tool, “ISBAR”) were more readily adopted than tools that were perceived to be new and more time consuming, such as (de)briefing

●

No enablers that might assist in implementing the techniques were identified by respondents

●

Barriers were readily identified, including:

●

Finding time to review and apply the ideas

●

The need to practisc the techniques so they became integrated skills

●

The need to adapt the techniques for specific work groups

●

Difficulty in using the techniques with colleagues who had not learned the concepts

●

The inflexibility of organizational processes when trying to improve workload distribution within a team

multidisciplinary mix of the workshops

Application of the techniques taught in the workshop

Perceived effectiveness of techniques

Enablers and barriers to the use of the techniques

64

Exemplar quote

[I’m now] aware that different professionals have different mental models. Respondent 6

I realized how I may have exacerbated a situation by inappropriate words in comb[ination] with tone and body language [in] both professional and personal situations. Respondent 2

When I tried to use the techniques to raise my concerns, they were brushed off or “forgotten” in the rush of the ward round. I also tried to use techniques to deescalate a confrontational event (with the same person) and again it was not particularly successful. As I tried to reflect back on my personal experience and to acknowledge the other person’s issues it all got a bit lost on the other person. Still I . . . will try to continue to use the techniques I learnt that day. Respondent 11

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

Assessing Modular CRM Training

of professions and clinical streams in the organization. The reported impact of the workshops is that they met the needs of participants, providing them with interprofessional learning experiences and practical skills to use in their workplaces. Trainees stated they left the courses enthused and ready to try the techniques. The communication module was rated highly by learners. A significant proportion of learners was interested in improving communication skills, particularly in learning about tools or strategies—such as graded assertiveness or conflict resolution—to resolve workplace communication problems. It was therefore not surprising that these were the techniques that participants attempted to apply when they returned to work. Perceived communication problems, such as workplace conflict, can be specific in time and place. For a health care professional, attending a 2-hour training module to learn how to address a current workplace conflict is likely to be more useful for the time-poor clinician than sitting through a full day of training that is only partially applicable to the problem. Short modules on specific topics may also make it easier to repeat training if more practise is required, or if sufficient time has elapsed since initial training that techniques have been forgotten. All but 2 participants anticipated facing workplace barriers to applying the tools, and these beliefs were borne out 6 weeks later, with many of these barriers being identified as the main reason why the strategies learned in training were not implemented. Barriers, both perceived and encountered by participants, were their individual skills and confidence, workplace culture, and team issues. Despite these difficulties, 22% of participants reported overcoming significant workplace barriers to attempt to change their own and colleagues’ negative individual or team behaviors. Culture is notoriously difficult to change.54,55 Although 1 in 5 may seem a small proportion of learners, these new behaviors resulted from only 2 hours of training and in an environment without coaching or workplace support. This outcome gives reason for cautious optimism for the potential for modular CRM training to benefit groups of interprofessional health staff. Using Kirkpatrick’s framework, we found positive results for reactions (Level 1) and self-reported workplace behaviors (Level 3). These findings agree with similar results for classroom-based CRM training that is delivered in more traditional day-long formats.22–24 The finding that reactions and behaviors for the modular training are similar to those

encountered for full-day courses is particularly useful for health care workers where professional development normally occurs in 1- to 3-hour blocks. It would be possible to include CRM training as part of regular professional development activities, thereby enabling team training to be delivered to a wider audience. This strategy may also be used to provide in-service education where the entire care team learns together, which is not normally feasible with a course of 1 day’s duration. Limitations and Next Steps The study findings are limited by the number of participants and the fact that the changes in behavior were self-reported. In addition, a selection bias resulting from the voluntary nature of the training limits the generalizability of the findings. Often, learners who are motivated to take part in team training already have some knowledge or interest in teamwork and their base level of knowledge can be higher than that of a typical clinician. This can sometimes reduce the degree of attitude and behavior change following the training, as there is “less room” for improvement. The next step is to investigate modular training on larger scale, preferably where clinicians are supported, following training, in overcoming workplace barriers to implementing the techniques. Mutual peer support could be accomplished by training a whole patient care team, or support could be provided on an individual basis via workplace mentoring or coaching. This has been found to be successful in changing clinician team behaviors following other CRM programs, such as TeamSTEPPS.56 An experimental design, where clinicians are randomized to participate in either modular or full-day training, would assist in determining the strengths and weaknesses of each. Studies examining impact on clinical and organizational performance (Kirkpatrick’s Results level) would also be valuable. Conclusion Continuing to provide education and training where clinicians spend long periods away from the workplace is no longer an option for the future. Modular training may open the door to further innovation in team training, whereby learners can engage in virtual online sessions unrestricted by time and space constraints.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

65

Clay-Williams et al.

Lessons for Practice ●

●

●

The study findings provide an indication of the potential of CRM training to be delivered in modular format to groups of interprofessional learners. Delivering CRM in modules may provide greater flexibility for health care professionals to select training that meets their individual knowledge, experience, and scheduling needs. Modular training of teamwork skills may pave the way for regular refresher, update, or remedial training in teamwork competencies, in line with continuing medical education practices for clinical skills.

References 1. Helmreich RL, Merritt AC, Wilhelm JA. The evolution of Crew Resource Management training in commercial aviation. Int J Aviat Psychol. 1999;9(1):19–32. 2. Helmreich RL, Wiener EL, Kanki BG. Cockpit Resource Management. San Diego: Academic Press; 1993. 3. Helmreich RL, Merritt AC. Safety and Error Management: The Role of Crew Resource Management. Aldershot, England: Ashgate; 2000. 4. Reason JT. Understanding adverse events: the human factor. In: Vincent C, ed. Clinical Risk Management: Enhancing Patient Safety. 2nd ed. London, England: BMJ Books; 2001:9–30. 5. Hollnagel E, Pari`es J, Woods DD, Wreathall J. Resilience Engineering in Practice: A Guidebook. Surrey, England: Ashgate Publishing Limited; 2011. 6. Grubb G, Morey JC, Simon R. Sustaining and advancing performance improvements achieved by crew resource management training. 11th International Symposium on Aviation Psychology. 2001;1:1–4. 7. Diehl A. The effectiveness of training programs for preventing aircrew “error”. Sixth International Symposium on Aviation Psychology. 1991;1:640–655. 8. Salas E, Wilson KA, Burke CS, Wightman DC. Does Crew Resource Management training work? An update, an extension, and some critical needs. Hum Factors. 2006;48(2):392–412. 9. O’Connor P, Campbell J, Newon J, Melton J, Salas E, Wilson KA. Crew Resource Management training effectiveness: a meta-analysis and some critical needs. Int J Aviat Psychol. 2008;18(4):353–368. 10. Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academies Press; 2000. 11. Donaldson L. An Organisation With a Memory: Report of an Expert Group on Learning From Adverse Events in the NHS. London, England: UK Department of Health; 2000. 12. Hindle D, Braithwaite J, Travaglia J, Iedema R. Patient Safety: A Comparative Analysis of Eight Inquiries in Six Countries. Sydney, Australia: Centre for Clinical Governance Research, University of New South Wales; 2008. 66

13. Garling P. Final Report of the Special Commission of Inquiry: Acute Care Services in NSW Public Hospitals. Vol. 1–3. Sydney, Australia: NSW Government; 2008. 14. Sexton JB, Holzmueller CG, Pronovost PJ, et al. Variation in caregiver perceptions of teamwork climate in labor and delivery units. J Perinatol. 2006;26(8):463–470. 15. Sexton JB, Makary MA, Tersigni AR, et al. Teamwork in the operating room: frontline perspectives among hospitals and operating room personnel. Anesthesiology. 2006;105(5):877–884. 16. Dunn EJ, Mills PD, Neily J, Crittenden MD, Carmack AL, Bagian JP. Medical team training: applying Crew Resource Management in the Veterans Health Administration. Jt Comm J Qua Patient Saf. 2007;33(6):317–325. 17. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004;13(5):330–334. 18. Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133(6):614–621. 19. Sexton JB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: cross sectional surveys. BMJ. 2000(320):745– 749. 20. Nembhard IM, Edmondson AC. Making it safe: the effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. J Organ Behav. 2006;27(7):941– 966. 21. Baker DP, Beaubien JM, Holtzman AK. DoD medical training programs: an independent case study analysis: AHRQ Publication No 060001, Rockville, MD; 2006. 22. Buljac-Samardzic M, Dekker-van Doorn C, van Wijngaarden J, van Wijk K. Interventions to improve team effectiveness: a systematic review. Health Policy. 2009. 23. Rabøl LI, McPhail M, Bjørn B, et al. Outcomes of a classroom-based team training intervention for multi-professional hospital staff. Med Teach. 2012;34(10):868–869. 24. Salas E, DiazGranados D, Klein C, et al. Does team training improve team performance? A meta-analysis. Hum Factors. 2008;50(6):903– 933. 25. Baker DP, Gustafson S, Beaubien JM, Salas E, Barach P. Medical Teamwork and Patient Safety: The Evidence-Based Relation. Washington, DC: American Institute for Research; 2003. 26. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2008. 27. Weaver SJ, Rosen MA. Team-training in healthcare: brief update review. In: Shekelle PG, Wachter RM, Pronovost PJ, eds. Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices. Rockville, MD: Agency for Healthcare Research and Quality; 2013. 28. McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care. 2009;18(2):109–115. 29. Meri´en A, Van de Ven J, Mol B, Houterman S, Oei S. Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet Gynecol. 2010;115(5):1021. 30. Weaver SJ, Lyons R, DiazGranados D, et al. The anatomy of health care team training and the state of practice: a critical review. Acad Med. 2010;85(11):1746–1760. 31. Skinner CA. Re-inventing medical work and training: a view from generation X. Med J Australia. 2006;185(1):35. 32. Cox M, Irby DM, Reznick RK, MacRae H. Teaching surgical skills— changes in the wind. N Engl J Med. 2006;355(25):2664–2669.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

Assessing Modular CRM Training 33. Thomas EJ, Williams AL, Reichman EF, Lasky RE, Crandell S, Taggart WR. Team training in the neonatal resuscitation program for interns: teamwork and quality of resuscitations. Pediatrics. 2010;125(3):539– 546. 34. Kirkpatrick DL. Evaluation of training. In: Craig RL, ed. Training and Development Handbook. 2nd ed. New York: McGraw-Hill; 1976:1811–18-27. 35. Braithwaite J, Westbrook J, Foxwell AR, et al. An action research protocol to strengthen system-wide inter-professional learning and practice. BMC Health Serv Res. 2007;7(1):144. 36. Greenfield D, Nugus P, Fairbrother G, Milne J, Debono D. Applying and developing health service theory: an empirical study into clinical governance. Clin Govern. 2011;16(1):8–19. 37. Greenfield D, Nugus P, Travaglia J, Braithwaite J. Auditing an organization’s interprofessional learning and interprofessional practice: The interprofessional praxis audit framework (IPAF). J Interprof Care. 2010;24(4):436–449. 38. Greenfield D, Nugus P, Travaglia J, Braithwaite J. Factors that shape the development of interprofessional improvement initiatives in health organisations. BMJ Qual Saf. 2011;20(4):332– 337. 39. Nugus P, Greenfield D, Travaglia J, Westbrook J, Braithwaite J. How and where clinicians exercise power: interprofessional relations in health care. Soc Sci Med. 2010;71(5):898–909. 40. Nugus P, Greenfield D, Travaglia J, Braithwaite J. The politics of action research: “If you don’t like the way things are going, get off the bus”. Soc Sci Med. 2012. 41. Clay-Williams R, Braithwaite J. Determination of health-care teamwork training competencies: a Delphi study. Int J Qual Health Care. 2009;21(6):433–440. 42. Clay-Williams R, McIntosh C, Kerridge R, Braithwaite J. Classroom and simulation team training: a randomised controlled trial. Int J Qual Health Care. 2013;25(3):doi: 10.1093/intqhc/mzt1027. 43. TeamSTEPPS Multimedia Curriculum Kit. In: Department of Defense, ed: Agency for Healthcare Research and Quality. 2006; http://teamstepps.ahrq.gov/abouttoolsmaterials.htm. Accessed Feb. 18, 2014.

44. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004;13(Suppl 1):i85–i90. 45. Mezirow J. Transformative learning: theory to practice. New Dir Adult Cont Educ. 1997;1997(74):5–12. 46. Kirkpatrick D. Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler; 1998. 47. Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44. 48. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):1553–1581. 49. McNaught C, Lam P. Using Wordle as a supplementary research tool. Qual Rep. 2010;15(3):630–643. 50. Liamputtong P, Ezzy D. Qualitative Research Methods. Melbourne, Australia: Wiley Online Library; 2006. 51. Braithwaite J, Westbrook M, Pirone C, Robinson P, Robinson M, Michael S. Staff Survey on Patient Safety. South Australian Council for Safety and Quality in Health Care and South Australian Department of Health. Sydney, Australia: Communio; 2009. 52. Hutchinson A, Cooper K, Dean J, et al. Use of a safety climate questionnaire in UK health care: factor structure, reliability and usability. Qual Saf Health Care. 2006;15(5):347–353. 53. Wordle [computer program]. Medford, MA. 2009; www.wordle.net. Accessed Dec. 13, 2012. 54. Braithwaite J, Westbrook M, Nugus P, et al. A four-year, systems-wide intervention promoting interprofessional collaboration. BMC Health Serv Res. 2012;12(1):99. 55. Braithwaite J, Hyde P, Pope C. Culture and Climate in Health Care Organisations. London, England: Palgrave Macmillan; 2009. 56. Henriksen K, Battles JB, Keyes MA, et al. TeamSTEPPSTM : Team strategies and tools to enhance performance and patient safety. Advances in Patient Safety: New Directions and Alternative Approaches (Vol 3. Performance and tools). Rockville, MD: Agency for Heathcare Research and Quality; 2008.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—34(1), 2014 DOI: 10.1002/chp

67