American Journal of Pharmaceutical Education 2016; 80 (5) Article 85.

INSTRUCTIONAL DESIGN AND ASSESSMENT Designing and Evaluating an Interprofessional Experiential Course Series Involving Medical and Pharmacy Students Melissa E. Rotz, PharmD,a,b Gladys G. Duen˜as, PharmD,a Aileen Zanoni,c Anisha B. Grover, PharmDa a

University of the Sciences, Philadelphia College of Pharmacy, Philadelphia, Pennsylvania Temple University School of Pharmacy, Philadelphia, Pennsylvania c Cooper Medical School of Rowan University, Camden, New Jersey b

Submitted May 30, 2015; accepted August 5, 2015; published June 25, 2016.

Objective. To prepare first-year and second-year pharmacy and medical students to build effective collaborative health care teams by participating in an interprofessional experiential 6-semester course series. Design. An interprofessional experiential course series was designed using a variety of teaching methods to achieve both interprofessional and experiential learning outcomes. A standardized objective behavioral assessment was developed to measure team performance of interprofessional communication and teamwork. In addition, student perceptions were measured using a validated instrument. Assessment. A majority of teams demonstrated appropriate competence with respect to interprofessional communication and teamwork. Additionally, a majority of students expressed positive perceptions of interprofessional collaboration with respect to teamwork, roles and responsibilities, and patient outcomes. Conclusion. An interprofessional experiential course series can be successfully implemented to achieve both interprofessional and experiential learning outcomes. Highly collaborative teams and positive student perceptions provide evidence of achievement of interprofessional education learning outcomes. Keywords: interprofessional education, behaviors, perceptions, SPICE, assessment

should competently “participate in experiential educational activities with prescribers/student prescribers and other student/professional health care team members, including face-to-face interactions that are designed to advance interprofessional team effectiveness.”4 Several instructional design formats for the delivery of IPE are reported in the literature. A review conducted by Abu-Rish and colleagues reviewed 83 eligible studies and found small group discussion and problem-based learning to be the most commonly used strategies, followed by experiential-based clinical teaching, simulation-based learning, and various other educational interventions.5 Because of the variety of instructional design formats employed and the varying frequency of these activities throughout the curriculum (ie, one-time activities vs multiple activities throughout the semester), it is not possible to make generalizations regarding the impact of IPE at this time. A systematic review conducted by Reeves and colleagues found that of the 15 eligible studies examined: seven reported an assortment of positive outcomes indicating IPE was well received and assisted with developing knowledge and skills for collaboration; four studies

INTRODUCTION Experiential education is a methodology involving the incorporation of learners into direct practice experiences and guided reflective observations with the goal of increasing student knowledge and professional abilities such as skills, attitudes, and behaviors.1 Interprofessional education (IPE) is a pedagogical approach that engages students of various health professions, dedicated to the introduction, reinforcement, and mastery of core competencies for provision of patient care in a collaborative team environment.2 Both experiential education and IPE are integral components of the doctor of pharmacy (PharmD) curriculum supported by the American Association of Colleges of Pharmacy’s Center for the Advancement of Pharmacy Education (CAPE).3 Additionally, the Accreditation Council for Pharmacy Education (ACPE) outlines these required standards and specifically states that all students Corresponding Author: Dr. Melissa E. Rotz, 3307 N. Broad St., Philadelphia, PA 19140. Tel: 215-707-4943. Fax: 215707-8326. E-mail: [email protected]

1

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. and the core competencies developed by IPEC.2 At the beginning of the fall semester, students were divided as evenly as possible into teams so that each team contained a combination of first-year medical students (M1s), second-year medical students (M2s), first professional year pharmacy students (P1s), and/or second professional year pharmacy students (P2s). For the practice experiences within the course series, teams alternated weekly between a student-run clinic and other practice sites. Each cohort of students returned to the student-run clinic on a biweekly basis, in between their visits to other sites on the alternate weeks. Throughout the academic year, students also convened during designated class time for several didactic sessions involving disease state topics commonly encountered in the clinic. During these didactic sessions, students collaborated to work through patient cases. Students provided direct patient care within the student-run clinic under the supervision of internal medicine residents and physicians and pharmacy faculty members. For pharmacy students, the experiential course series is focused on achieving both interprofessional and experiential learning outcomes specifically within the ambulatory care, community, and institutional pharmacy practice settings. As pharmacy students progress through semesters 1 through 6, incremental goals are set to foster introduction to pharmacy practice experiences (IPPE) core abilities: (1) employ knowledge of the medicationuse process; (2) use information related to drugs and human diseases currently being taught in class; (3) practice basic patient assessment skills; and (4) apply drug information analytical skills. Students are also expected to participate in team-based care, and exercise interprofessional communication skills. Moreover, ongoing-targeted reflection sessions are scheduled to discuss IPE key concepts such as values and ethics and roles and responsibilities. For the medical students, the experiential course series is focused on achieving their own set of goals in a similar incremental manner. A combination of teaching methods were used in this experiential course series, including scheduled orientation, midpoint and final reflective sessions, and a hybrid of didactic and active-learning activities that introduced IPE core competencies, profession-specific course content, and practice management workflow expectations and tasks. Each course involved a pass/fail format in which formative assessments conducted by faculty members and residents, using rubrics available to the students, assisted in evaluating student learning. For the most part, students participated in self-directed learning at the student-run clinic, where upper-classmen or peers served as mentors to guide all “hands-on” activities including

reported mixed outcomes; and another four reported IPE interventions had no impact on practice or patient care.6 The following themes are notable regarding studies reporting positive outcomes: IPE interventions result in changes of learners’ perceptions and values,7-9 and IPE interventions enhance teamwork and collaborative behavior.10,11 Shrader and colleagues described multiple IPE activities delivered longitudinally within a required clinical assessment course that significantly improved students’ perceptions of interprofessional collaboration.9 MacDonnell and colleagues demonstrated that after a 1-day IPE workshop, students from three different health professions not only recognized the value IPE in their training, but also demonstrated effective teamwork in taking care of patients.10 The purpose of this report is to describe the design and evaluation of an innovative interprofessional experiential 6-semester course series involving first-year and second-year pharmacy and medical students. The curricular goal of creating this interprofessional experiential course series was to better prepare students to build effective collaborative health care teams.12 To evaluate this goal, we measured student perceptions and team performance of roles and responsibilities, interprofessional communication, and teams and teamwork as these are three of the four Interprofessional Education Collaborative (IPEC) core competency domains.2 Our decision to evaluate student perceptions and team behaviors was based on the assumption that perceptions and behaviors impact the effectiveness of a team and are important aspects of collaboration. We hypothesized that by implementing this interprofessional experiential course series, students would express positive perceptions of interprofessional collaboration with respect to teamwork, roles and responsibilities, and patient outcomes, and student teams would demonstrate competence in interprofessional communication and teamwork.

DESIGN Our longitudinal interprofessional practice experience combines a 6-semester experiential course series created by Cooper Medical School of Rowan University (CMSRU) and University of the Sciences (USciences) Philadelphia College of Pharmacy. Semesters 1 and 2 for first-year students run concurrently with semesters 3 and 4 for second-year students. At the time of this evaluation, semesters 5 and 6 within the course series were under development with no students enrolled as the inaugural medical class had only reached the second year at this point. At the beginning of the course series, students were introduced to definitions and concepts related to IPE 2

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. patient care, pharmacy, care coordination, and practice management roles and responsibilities. To evaluate the curricular goal of better preparing students to build more effective collaborative health care teams, we designed a standardized objective behavioral assessment to measure team performance of interprofessional communication and teamwork. To standardize the assessment process for all student teams, we designed a standardized patient case that mimicked students’ patient care responsibilities within the student-run clinic. It was conducted similarly to an objective structured clinical examination; however, the focus of the assessment was to measure team behaviors rather than clinical knowledge and skills. In addition to the behavioral assessment, student perceptions were measured using a validated instrument. Both the behavioral and perception measurements served as formative assessments for students enrolled in the course series at the end of the spring 2014 semester. Faculty members from CMSRU and USciences worked collaboratively to create objectives for and design the assessment activity. Specifically, we created a patient with diabetes who recently lost medical and prescription insurance and who did not have a way to pay for medications. A patient presenting with these problems is common at the student-run clinic, and faculty members determined it would be applicable to all students. Objective criteria specific to the IPE standardized objective behavioral assessment were developed to measure team performance of interprofessional communication and teamwork with regard to: team structure, leadership, mutual support, communication, and situation monitoring. The objectives and design of the standardized patient case were reviewed and modified by additional faculty members involved in the interprofessional experiential course series. The budget to design and conduct this assessment did not allow for each team to interact with a live standardized patient. To mimic a clinical practice experience as closely as possible within the constraints of our limited budget, the standardized patient program coordinator at CMSRU created a 5-minute video of a health care team consisting of medical and pharmacy students conducting a history and physical examination on a standardized patient. Students were divided into 48 teams, consisting of three to four students. The teams were subsets of teams that students were assigned to for the duration of the course series at the beginning of the fall 2013 semester. Care was given to ensure that at least one pharmacy student was on each team; however, the number of medical students significantly exceeded the number of pharmacy students because of the prespecified 2:1 enrollment ratio of medical students to pharmacy students within the

respective courses. As a result, six teams were composed of only medical students. Student teams were assigned to participate in a specific standardized objective behavioral assessment session at the CMSRU Simulation and Clinical Skills Center on two dates in April 2014. Students were not required to prepare outside of class except to complete the validated survey instrument to collect their perceptions of IPE. Three sessions of eight teams participated each day. At the beginning of each session, students were oriented to the objectives and logistics of the standardized objective behavioral assessment. After the orientation, student teams proceeded to the patient examination rooms to watch the 5-minute video. After watching the video, teams were given information about the standardized patient that included physical examination findings, laboratory results, and self-monitored blood glucose readings. Because some of the students had not yet completed didactic courses on diabetes at the time of this activity, teams were also given a blood glucose management protocol that included glycemic goals of therapy and a titration schedule for basal and preprandial insulin.13,14 Teams were given 20 minutes to review information about the standardized patient, identify the issues that impact the patient’s health care, and develop a care plan for the patient. Teams were permitted to rewind the video and to use the computer to access Lexi-Comp, a drug information resource. They were not permitted to use other resources or reference materials because the focus of the assessment was to measure communication and teamwork, rather than clinical knowledge and skills. There was a concern that if teams were permitted to use any resource, the team’s focus may shift to searching for the correct information rather than collaborating to develop a solution. Additionally, other activities within the course series, specifically the didactic sessions, were designed to assess clinical knowledge and application. Teams were instructed to write the patient’s care plan on a postencounter note, which was submitted at the end of the encounter. Each examination room was equipped with a computer monitor, ceiling mounted microphones, and cameras to record all encounters. Teams were assessed by evaluators via a live video feed in a nearby monitoring room, or via a recorded video after the session. Teams could not all be evaluated in real time because there were not enough evaluators present for all encounters at all sessions. After all of the sessions were complete, faculty members compiled written feedback for the optimal care plan into one document and e-mailed this to students to provide global feedback. The study protocol, along with all associated study materials, assessments, and procedures, 3

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. was approved as exempt research by both Rowan University and USciences institutional review boards.

using descriptive statistics to produce general frequency distributions. Open-ended questions from the student evaluations were analyzed qualitatively to produce thematic content. Out of the 163 students enrolled in the course series for the 2013-2014 academic year, 156 students divided into 48 teams completed the standardized objective behavioral assessment. Faculty and preceptor evaluators scored each team’s performance during the standardized objective behavioral assessment using the PACT-novice tool. Team performance of each behavioral domain are presented in Figures 1A (all teams) and 1B (interprofessionalonly teams). For all of the domains, a majority of teams performed average or excellent, and few teams performed poorly for any domain. For each domain, it was

EVALUATION AND ASSESSMENT Total enrollment in the interprofessional experiential course series for the 2013-2014 academic year was 163 students: 49 pharmacy students (25 P1s and 24 P2s) and 114 medical students (64 M1s and 50 M2s). Two validated instruments were used for assessment purposes: the Performance Assessment of Communication and Teamwork (PACT) novice tool and the Student Perceptions of Physician-Pharmacist Interprofessional Clinical Education (SPICE) Instrument (see Appendix 1 & 2).15,16 The PACT-novice tool was developed for assessing performance of communication and teamwork within interprofessional teams. The PACT-novice tool was utilized by evaluators, specifically medical and pharmacy faculty members and internal medicine resident preceptors, to assess team performance during the standardized objective behavioral assessment. It contained five domains with a 3-point scale (15poor, 25average, and 35excellent performance). The five domains were team structure, leadership, mutual support, communication, and situation monitoring. Each domain was comprised of multiple critical behaviors that help define the domain. The final assessment was based on the presence of these critical behaviors within each domain, as well as the quality of performance. Another key aspect of this tool was that it assessed team behaviors, not individual behaviors.15 The SPICE instrument is a relatively new tool for measuring student perceptions of IPE and aligns with the IPEC core competencies of roles and responsibilities and teams and teamwork.16 The instrument contains 10 items categorized into three factors, specifically, (1) interprofessional teamwork and team-based practice, (2) roles/ responsibilities for collaborative practice, and (3) patient outcomes from collaborative practice. It uses a 5-point Likert scale for responses with 5 representing strongly agree and 1 representing strongly disagree. Teams received either a passing or failing grade for the standardized objective behavioral assessment. Passing was based solely on attendance and active participation as the focus of the formative assessment was to measure how learning was going and identify areas for improvement. After completion, students were asked to evaluate the assessment activity. The SPICE instrument was administered prior to the standardized objective behavioral assessment to collect student perceptions regarding the value of IPE. The PACT-novice assessments, SPICE responses, and student evaluations of the assessment were collected using Qualtrics (Qualtrics, Provo, UT) and were analyzed

Figure 1a. Team performance of interprofessional communication and teamwork. Includes assessment data for all 48 teams that participated in teh standardized objective behavioral assessment. Situation monitoring involves assessment of conflict management, which was not observed for all teams. Figure 1b. Team performance of interprofessional communication and teamwork. Excludes assessment data for 7 noninterprofessional teams that participated in the standardized objective behavioral assessment. Situation monitoring involves assessment of conflict management, which was not observed for all teams.

4

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. evenly split between teams performing excellent and average. For the situation monitoring domain, 40% (19/48) of teams were not assessed because this domain involves conflict management, which was not observed for all teams. Overall, a majority of team behaviors met and/or exceeded performance expectations at this point in the curricula for the standardized objective behavioral assessment. All 48 student teams submitted a care plan at the end of the standardized objective behavioral assessment. Care plans were reviewed by faculty members to provide global feedback to the entire class rather than individualized feedback because the focus of the activity was on team behaviors during the activity, rather than the clinical content of the care plans. The SPICE survey was a required assignment to be completed outside of the course and prior to the standardized objective behavioral assessment. Out of the 163 students enrolled in the course series for the 2013-2014 academic year, 155 students completed the SPICE survey for a response rate of 95%. Student responses to the SPICE survey are presented in Figure 2. For a majority of the items, more than 80% of students responded in agreement (ie, either strongly agree or agree), and for all of the items, more than 50% of students responded in agreement. To highlight a few outliers, students responded neutrally or in disagreement (ie, either strongly disagree or disagree) with items 2, 4, and 7 more so compared to the other items, though these responses were considerably less common compared to the students who responded in agreement. Specifically, for item 2 (“My role within the interdisciplinary team is clearly defined”), 27% (42/155) of students responded neutrally, and 18% (27/155) responded in disagreement. For item 7 (“I understand the roles of other professionals within the interdisciplinary

team”), 21% (33/155) of students responded neutrally, 12% (18/155) responded in disagreement. Item 4 (“Patient satisfaction is improved when patients are treated by a team of professionals from different disciplines”) had the highest percentage of neutral responses at 34% (52/155) of students. Overall, student perceptions of IPE were predominantly positive, which is similar to previously published literature.8,11,17-23 Out of the 156 students who completed the standardized objective behavioral assessment, 45 students subsequently completed the voluntary student evaluation survey for a response rate of 29%. Overall, the responses were positive (Table 1). Most students agreed that the assessment allowed them to practice working within their teams, contributed to their learning, and that they would like to participate in similar assessments in the future. Students also provided their opinions about the strengths of the assessment activity and suggestions for improvement, which are qualitatively summarized in Table 2. Six faculty members from USciences and CMSRU worked collaboratively to create the standardized patient case scenario and videos which included two standardized patients (one man and one woman for the intent to reduce sharing of information between sessions) and three actors portraying health care profession students (two medical students and one pharmacy student). To ensure assessment with the PACT-novice tool was consistent across evaluators, a training program was created by the USciences faculty members. All eight evaluators participated in a 60-minute training program to become familiar with the PACT-novice tool and learn how to assess specific behaviors. The training involved watching three videos depicting team behaviors that were indicative of poor, average, and excellent performance based on the PACT-novice tool

Figure 2. Student perceptions of interprofessional education and collaboration.

5

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. Table 1. Student Evaluations of the Standardized Objective Behavioral Assessment Statement

Strongly Disagree

The assessment provided an opportunity for me to practice working within my team. The assessment contributed positively to my learning within the Ambulatory Clerkship course. I would like to participate in similar assessments in the future.

Responses (N=45), n (%) Disagree Neutral Agree

Strongly Agree

1 (2)

0

4 (9)

22 (49)

18 (40)

1 (2)

2 (5)

6 (13)

22 (49)

14 (31)

1 (2)

5 (11)

9 (20)

12 (27)

18 (40)

criteria. The training videos were created with the help of seven USciences pharmacy students not involved with the IPE program. Consideration should be given when creating a training program and conducting behavioral assessments as these require considerable time and resources.

Healthcare Research and Quality in collaboration with the Department of Defense.24 The PACT-novice tool served as a relatively concise method of evaluating the desired team behavioral outcomes. The SPICE instrument was chosen to evaluate student perceptions related to IPE because it has demonstrated validity and reliability in a student population similar to ours, it aligns with the IPEC core competencies, it incorporates student perceptions on the impact of IPE on patient outcomes, and is relatively easy to administer as it includes only 10 items.2,16 Since the time of this assessment, the SPICE instrument was revised (SPICE-R) and profession-specific language was removed to broaden use among more diverse groups of health professions students.25 We observed that a high percentage of medical and pharmacy students agreed that working with students from another profession enhances student education and future abilities to work on an interprofessional team, and that all health profession students should be educated to establish collaborative relationships with members from other professions, which is similar to the findings of many other studies.7,18-20,23 The majority of students agreed that physicians and pharmacists should collaborate in teams and that medical and pharmacy students should be involved in teamwork during their education in order to understand their respective roles. These perceptions were measured towards the end of the spring semester, just before the standardized objective behavioral assessment. Although maintenance of perceived value over the

DISCUSSION We measured student perceptions and team performance of roles and responsibilities, interprofessional communication, and teamwork within our experiential course series to evaluate our curricular goal of better preparing students to build more effective collaborative health care teams. We hypothesized that by implementing this interprofessional experiential course series, students would express positive perceptions of interprofessional collaboration with respect to teamwork, roles and responsibilities, and patient outcomes, and that student teams would demonstrate competence in interprofessional communication and teamwork. The PACT-novice tool was chosen for its capacity to evaluate behaviors of teams across five domains. It is designed for new evaluators without previous coding experience and provides a method for conducting a realtime assessment, which matched our evaluator population and our behavioral assessment design. It is also based on the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) framework, which is an evidence-based teamwork system designed specifically for health care professionals by the Agency for

Table 2. Open-ended Student Evaluations of the Standardized Objective Behavioral Assessment Emergent Themes

Responses n (%)

Strengths of the assessment (n533) Teamwork Opportunity to practice skills Design of activity (eg, appropriate length of time, real life scenario) Request for similar activities Suggestions for improvement (n526) Design of activity (eg, more time, more activities, more supplementary materials) Instant debrief and feedback Technology issues Uneven balance of knowledge among students

6

15 10 6 2

(46) (30) (18) (6)

13 8 3 2

(50) (31) (12) (7)

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. span of several semesters cannot be demonstrated without a pretest of perceptions, the timing of this evaluation captured student perceptions following a period of time when the student respondents had been working together in the student-run clinic environment for either one or two academic years, depending on their time of enrollment within the professional programs. Although more than half of students responded in agreement, 27% of students responded neutrally about the clear definition of their own role within the interdisciplinary team. A larger percentage of students agreed that they understood the roles of other professionals within the interprofessional team. Based on their stage in the program, this was not unexpected, nor a negative finding. This experiential program is designed to allow students to step in and out of every role within the studentrun clinic to establish an awareness of the roles of various members of the health care team. During their first year in the interprofessional clinic, both pharmacy and medical students have intentionally similar roles so they can build a foundational appreciation for their teammates and foster collaboration. As students progress through their respective programs, roles differentiate and become more clearly defined as they gain more knowledge and experience. Therefore, it is not surprising that first-year and second-year students are still defining their roles within the clinic environment. If students were unclear about their roles within the team, this may have impacted their confidence in clearly articulating this information to the patient. This may account for the 34% of students who responded neutrally to the statement involving improved patient satisfaction when patients are treated by an interprofessional team. Although the positive perceptions of students were predictable based on the results of other studies, student behaviors are less represented in the literature. 7-11,17,20,23 Shrader and colleagues demonstrated that student perceptions, collected with an attitudinal survey instrument, were not a significant predictor of clinical outcome scores, whereas teamwork scores, rated by trained observers using a checklist, were a significant predictor of clinical outcome scores.11 Our standardized objective behavior assessment followed a similar model and involved trained observers evaluating student team behaviors using a checklist. A majority of teams performed average to excellent for all five domains. One interesting finding was that the communication domain had the highest performances across teams. No teams performed poorly within this domain, which may be reflective of the level of comfort that teams had developed throughout the year. Student familiarity with their teammates, along with the type of patient represented in the case may have

contributed to the high level of team performance across domains. The standardized patient was intentionally designed to represent a patient that the students commonly encounter in their interprofessional studentrun clinic, which serves an uninsured, underserved, urban population. The patient problem list included both clinical challenges and access-related issues, providing an opportunity for input from first-year students, along with the more clinically advanced second-year students. Strengths of the course series include the incorporation of IPEC core competencies into required experiential courses, which are fully integrated into the curricula of both schools.2 Additionally, this course series provides opportunities for direct application of knowledge that students learn in didactic coursework of each respective curriculum. A student-run clinic is an ideal educational environment to practice the skills required for future practice within an interprofessional team, because it provides real clinical experiences for students to apply the knowledge and skills they are learning throughout the didactic portion of their curricula. Challenges of the course series include the coordination of logistics, scheduling, and satisfaction of accreditation requirements for two distinct academic institutions. Additionally, limited space and resources, including financial and personnel, are expected barriers in any free, student-run clinic environment. These limited resources prevented the collection of pre/ post-data, which may have provided evidence of a change in perceptions and/or behaviors. Performance assessment is extremely resource- and time-intensive; therefore, student abilities were evaluated only once, towards the end of the course series, to demonstrate competency. Specific strengths of the standardized objective behavioral assessment include an opportunity for students of varying years and professions to collaborate on developing a therapeutic and nontherapeutic plan for a commonly encountered patient case. Additional strengths include the collection of three different types of research data in order to capture a well-rounded set of information regarding individual student perceptions, team behaviors, and observational data related to the development of care plans for the patient case. Many resources were used for both development and evaluation of the standardized objective behavioral assessment. We originally planned to use standardized patients for each team of students to make the activity as realistic as possible. However, after discussing the financial implications of this approach, we developed a cost-sparing alternative by creating the video-recorded patient encounter. This video was deemed to be more lifelike rather than just providing teams with a paper patient case. Additionally, this approach provided students with 7

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. the opportunity to practice listening for and identification of critical information during a commonly encountered interview framework. Implementation of the PACT-novice and SPICE instruments reinforced the importance of several course design elements, including the opportunity for early, frequent, and longitudinal exposure to and immersion within IPE in an experiential setting. Students submitted feedback in the form of an activity evaluation following the standardized objective behavioral assessment and a majority of respondents requested increased exposure to similar assessment opportunities that allowed open dialogue and collaboration with team members. Additionally, students expressed the desire for an increased time allotment for in-depth faculty feedback, which emphasizes the need for increased personnel resources. Since that offering, course improvements have included an increased effort to incorporate standardized patients and opportunities for students to practice skills in a team-based environment, starting at the beginning of each academic year with an orientation session and a series of introductory workshops to provide students with foundational information about IPE and the basic skills needed in clinic. These skills include introduction to the patient, measurement of vital signs, obtainment of a complete medication history, obtainment of accurate drug information using evidence-based resources, and provision of medication counseling. To ensure students are achieving both IPEC core competencies and experiential objectives mandated by accreditation standards, faculty members have developed, implemented, revised, and continually improved several workbooks, checklists, and assessment tools to monitor student progress.2,4 There are limitations to the design and analysis of our standardized objective behavioral assessment. Specifically, there was potential for low interrater reliability when evaluating the standardized objective behavioral assessment. To reduce this potential limitation, evaluators were trained using prerecorded videos and were given an opportunity to become familiar with the rubric. However, interrater reliability was not formally established. Additionally, some evaluators were faculty members or preceptors for the experiential course and had worked closely with the students, while others had not. Because team performance could not be assessed in a blinded manner, some evaluators may have been more familiar with the students they were assessing. Although the instrument used by faculty members was designed to only assess team behaviors, several faculty members and students also felt that it was important to provide feedback regarding the clinical decisions of the teams. In similarly structured future activities, it would be preferable to incorporate more faculty and preceptor evaluators, so that

each team can benefit from more in-depth feedback related to both clinical and behavior-based assessments.

SUMMARY Medical and pharmacy students from two academic institutions were grouped into teams within the first two years of their professional programs to complete an interprofessional experiential 6-semester course series within a student-run clinic serving an underserved, uninsured patient population. Student perceptions regarding the value of IPE were generally positive, and a majority of team behaviors met and/or exceeded performance expectations. These exploratory findings provide supportive evidence that IPE can inculcate effective collaboration among students. The course series provided an opportunity for didactic instruction, standardized patient interviews, and the ability to apply knowledge and practice skills within an experiential framework. Additionally, the course series fulfilled required credit hours for students from both institutions. High student perception scores, and, perhaps more importantly, high team performance of collaborative behaviors, demonstrated the positive impact of these learning experiences.

ACKNOWLEDGMENTS The authors wish to acknowledge Amanda Burden, MD, for her contributions to the development of the protocol and the coordination of logistics; Anna Headly, MD, for her contributions to the development of a patient case commonly encountered by students in clinic; and Michael Goldberg, MD, for his leadership support. The authors also wish to acknowledge the PACT evaluators who scored team behaviors, in addition to several of the listed authors, including Michelle Murphy, PharmD, Phung Phan-Geneblazo, PharmD, Loren Martinez, PharmD, Namrata Baxi, MD, and Michael Taylor, MD. Finally, the authors wish to acknowledge Justine Campo, PharmD, Stephanie Yenner, PharmD, Laura Walter, PharmD, Priya Patel, PharmD, Susan Youssef, PharmD, Brian Duong, PharmD, and Anthony Lee, PharmD for creating the videos that were used to train the PACT evaluators.

REFERENCES 1. Haase KK, Smythe MA, Orlando PL, et al. Quality experiential education. Pharmacotherapy. Dec 2008; (12):1547. 2. Interprofessional Education Collaborative Expert Panel. Core competencies for interprofessional collaborative practice: report of an expert panel. Washington, DC: Interprofessional Education Collaborative; 2011. 3. Medina MS, Plaza CM, Stowe CD, et al. Center for the Advancement of Pharmacy Education 2013 educational outcomes. Am J Pharm Educ. Oct 2013;77(8):162. 4. Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in

8

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. 16. Fike DS, Zorek JA, MacLaughlin AA, Samiuddin M, Young RB, MacLaughlin EJ. Development and validation of the student perceptions of physician-pharmacist interprofessional clinical education (SPICE) instrument. Am J Pharm Educ. Nov 2013;77(9):190. 17. Zorek JA, MacLaughlin EJ, Fike DS, MacLaughlin AA, Samiuddin M, Young RB. Measuring changes in perception using the Student Perceptions of Physician-Pharmacist Interprofessional Clinical Education (SPICE) instrument. BMC Med Educ. 2014;14:101. 18. Horsburgh M, Lamdin R, Williamson E. Multiprofessional learning: the attitudes of medical, nursing and pharmacy students to shared learning. Med Educ. Sep 2001;35(9):876-883. 19. Curran VR, Sharpe D, Flynn K, Button P. A longitudinal study of the effect of an interprofessional education curriculum on student satisfaction and attitudes towards interprofessional teamwork and education. J Interprof Care. Jan 2010;24(1):41-52. 20. Seif G, Coker-Bolt P, Kraft S, Gonsalves W, Simpson K, Johnson E. The development of clinical reasoning and interprofessional behaviors: service-learning at a student-run free clinic. J Interprof Care. Nov 2014;28(6):559-564. 21. Hawk C, Buckwalter K, Byrd L, Cigelman S, Dorfman L, Ferguson K. Health professions students’ perceptions of interprofessional relationships. Acad Med. Apr 2002;77(4):354-357. 22. Pittenger AL, Westberg S, Rowan M, Schweiss S. An interprofessional diabetes experience to improve pharmacy and nursing students’ competency in collaborative practice. Am J Pharm Educ. Nov 2013;77(9):197. 23. Rotz ME, Duen˜as GG, Grover AB, Headly A, Parvanta CF. Exploring first-year pharmacy and medical students’ experiences during a longitudinal interprofessional education program. Currents in Pharmacy Teaching and Learning. May-June 2015;7 (3):302-311. 24. King HB, Battles J, Baker DP, et al. TeamSTEPPS: Team Strategies and Tools to Enhance Performance and Patient Safety. Agency for Healthcare Quality and Research. 2006. Available from: http://www.ahrq.gov/professionals/quality-patient-safety/patientsafety-resources/resources/advances-in-patient-safety-2/vol3/ Advances-King_1.pdf. 25. Dominguez DG, Fike DS, MacLaughlin EJ, Zorek JA. A comparison of the validity of two instruments assessing health professional student perceptions of interprofessional education and practice. J Interprof Care. 2015;29(2):144-9.

pharmacy leading to the Doctor of Pharmacy degree (“Standards 2016”). Chicago, Illinois: Accreditation Council for Pharmacy Education; Approved January 25, 2015. Released February 2, 2015. 5. Abu-Rish E, Kim S, Choe L, Varpio L, Malik E, White AA, Craddick K, Blondon K, Robins L, Nagasawa P, Thigpen A, Chen LL, Rich J, Zierler B. Current trends in interprofessional education of health sciences students: a literature review. J Interprof Care. 2012 Nov;26(6):444-51. doi: 10.3109/13561820.2012.715604. 6. Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database Syst Rev. 2013 Mar 28;3:CD002213. doi: 10.1002/14651858.CD002213.pub3. 7. Vaughn LM, Cross B, Bossaer L, Flores EK, Moore J, Click I. Analysis of an interprofessional home visit assignment: student perceptions of team-based care, home visits, and medication-related problems. Fam Med. 2014 Jul-Aug 2014;46(7):522-526. 8. Gunaldo TP, Andrieu SC, Garbee D, et al. Student perceptions about interprofessional education after an elective course. J Interprof Care. Oct 2014:1-2. 9. Shrader S, Griggs C. Multiple interprofessional education activities delivered longitudinally within a required clinical assessment course. Am J Pharm Educ. Feb 2014;78(1):14. 10. MacDonnell CP, Rege SV, Misto K, Dollase R, George P. An introductory interprofessional exercise for healthcare students. Am J Pharm Educ. Oct 2012;76(8):154. 11. Shrader S, Kern D, Zoller J, Blue A. Interprofessional teamwork skills as predictors of clinical outcomes in a simulated healthcare setting. J Allied Health. 2013;42(1):e1-6. 12. World Health Organization. Framework for action on interprofessional education & collaborative practice. Æhttp://whqlibdoc. who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdfæ; 2010. Accessed July 13, 2015. 13. American Diabetes Association. Standards of medical care in diabetes–2014. Diabetes Care. Jan 2014;37 Suppl 1:S14-80. 14. Mooradian AD, Bernbaum M, Albert SG. Narrative review: a rational approach to starting insulin therapy. Ann Intern Med. Jul 2006;145(2):125-134. 15. Chiu C-J, Brock D, Abu-Rish E, University of Washington Macy Assessment Team. Performance Assessment of Communication and Teamwork (PACT) Tool Set. 2014; http://collaborate.uw.edu/ educators-toolkit/tools-for-evaluation/performance-assessment-ofcommunication-and-teamwork-pact-too. Accessed May 30, 2015.

9

Recognize leader s One team member assumed a leadership role u Understand role s Coder can recognize the role/ responsibility of each team member s Team members provide suggestions/ feedback based on their professional training s Team members take up tasks appropriate to their role u Understand team goals s Coder can recognize at least one of the team goals s Team leader elicits team goals s Team members offer professional suggestions which help achieve goals s Team members follow orders which lead to achieving team goal u Refer to protocols/checklists s Coder can recognize protocols/checklists which team refers to s Team members name a protocol/checklist to follow s Team members offer suggestions with the source s Team members take out handbook or pocket cards for reference u Respond to potential errors s Coder can recognize a team member who stops the flow s Team members raise their concerns s Team members stop the intervention for potential error u Share information s Coder can recognize the essential information is shared

u

Team Structure

Quality

Comments

Not performed well u Adequately performed u Performed well

u

Mutual Support Acknowledge statement s Coder can recognize team leader and members receive questions/concerns raised by others well s Coder can recognize the team is responsive to concerns and requests s Team leader and/or members acknowledge team members’ concern VERBALLY u All participate s Coder can recognize the team leader assigns tasks to each team leader s Leader delegates tasks to ensure participate of all team members s Coder can recognize team members working on individual tasks s All team members work on assigned tasks in a timely manner u Call attention s Coder can recognize concerns raised by the team s Member expresses concerns verbally s Member ensures team is aware of the possibility of errors/complications u Ask for help s Coder hears request for help from team members (when appropriate)

u

Appendix 1. Performance Assessment of Communication and Teamwork (PACT)-Novice Tool15 Quality Not performed well

(Continued)

Comments

Adequately performed u Performed well

u

u

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85.

10

Team members share lab results Team member read patient history out loud

Delegate tasks s Coder can recognize that team members are comfortable with their responsibilities s Team leader assigns team members appropriate tasks for professional roles s Coder can recognize if any team member is overwhelmed s Team leader asks different team members to perform different tasks s Team leader does not overload any one of the team members u Authority vs participation s Coder can recognize the inputs from team members s Team members gather together to offer professional suggestions s Leader is receptive about team members suggestions u Collective input s Coder can recognize the input from team members s Team members provide professional suggestions s Coder can recognize who the leader is s Leader asks for professional suggestions u Speak up s Coder can recognize the culture of speaking freely and asking questions s Team leader asks questions s Team members ask questions s Team members raise their concerns and offer professional opinions

u

Leadership

s

s

Team Structure

(Continued )

Comments

u

Not performed well Adequately performed u Performed well

u

Quality

Quality

Situation Monitoring u Maintain situation awareness s Coder can recognize disagreements or conflicts between team members s Coder can recognize the team is aware of the situation during conflict resolution s Team members have the big picture in mind when they are in disagreement s Team members resolve conflicts

Verbalize s Coder can hear what team members are doing for the patient s Team members speak with a volume that other team members can hear u Ask for clarification s Coder can hear request to explain or repeat s Team member asks for explanation

u

Communication

Mutual Support

11

Comments

Quality u Not performed well u Adequately performed u Performed well

Comments

u

Not performed well Adequately performed u Performed well

u

Quality

Quality

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85.

Session Time: 3-3:45pm 4-4:45pm Simulation Room: 1 2 3 4 5 6 7 8

5-5:45pm

Evaluator:_________________

Instructions for use: d While observing each team during the simulated activity, check off the team behaviors (in italics) that were observed for each domain (i.e. team structure, leadership, mutual support, communication, and situation monitoring) s Underneath each behavior are examples of these behaviors—all examples do NOT have to be observed in order to check off the behavior d While observing, write additional comments in the appropriate boxes for each domain to help guide evaluations at the end of the activity d After the activity is complete, check off the quality of performance for each domain (ie, not performed well, adequately performed, or performed well) d Use the following scoring system to evaluate each domain as poor, average, or excellent s POOR: multiple critical behaviors absent or not performed well s AVERAGE: most behaviors present and adequately performed s EXCELLENT: all critical behaviors present and performed well d Reminders: s Assessing TEAM performance, NOT individual performance s Assessing STUDENT team, NOT expert team Tool adapted from: http://collaborate.uw.edu/sites/default/files/files/PACT_RealTime_ShortForm_Generic_110611_copyright.pdf

Session Day: Tuesday Thursday Simulation Group:_________________

(Continued)

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85.

12

American Journal of Pharmaceutical Education 2016; 80 (5) Article 85. Appendix 2. Student Perceptions of Physician Pharmacist Interprofessional Clinical Education (SPICE) Instrument.15 The purpose of this instrument is to assess perceptions of medical and pharmacy students in interprofessional clinical education. These perceptions will be compared with student behaviors to explore the relationships between the two. Please be completely honest as you rate the extent of your agreement with each of the following statements: Statement

1 2 3 4 5

1. Working with another discipline of students enhances my education. 2. My role within the interdisciplinary team is clearly defined. 3. Health outcomes are improved when patients are treated by a team of professionals from different disciplines. 4. Patient satisfaction is improved when patients are treated by a team of professionals from different disciplines. 5. Participating in educational experiences with another discipline of students enhances my future ability to work on an interdisciplinary team. 6. All health professions students should be educated to establish collaborative relationships with members from other disciplines. 7. I understand the roles of other professionals within the interdisciplinary team. 8. Clinical practice experiences are the ideal place within their respective curricula for medical and pharmacy students to interact. 9. Physicians and pharmacists should collaborate in teams. 10. During their education, medical and pharmacy students should be involved in teamwork in order to understand their respective roles. 15strongly disagree, 25disagree, 35neutral, 45agree, 55strongly agree

13