Patient Education and Counseling 98 (2015) 168–173
Contents lists available at ScienceDirect
Patient Education and Counseling journal homepage: www.elsevier.com/locate/pateducou
Medical education
Comparison of standardized patients and real patients as an experiential teaching strategy in a nutrition counseling course for dietetic students Vicki S. Schwartz a,b,*, Pamela Rothpletz-Puglia b, Robert Denmark b, Laura Byham-Gray b a b
Drexel University, Department of Nutrition Sciences, Philadelphia, USA Rutgers University, New Jersey School of Health Related Professions, Department of Nutritional Sciences, Newark, USA
A R T I C L E I N F O
A B S T R A C T
Article history: Received 15 May 2014 Received in revised form 27 October 2014 Accepted 11 November 2014
Objectives: To compare the quality of communication and behavioral change skills among dietetic students having two nutrition encounters with either a real patient or a standardized patient in the simulation laboratory at Drexel University, Philadelphia, PA, United States. Methods: A retrospective analysis of video recordings (n = 138) containing nutrition encounters of dietetic students (n = 75) meeting with a standardized patient (SP) or a real patient (RP). Trained raters evaluated communication skills with the 28 item Calgary Cambridge Observation Guide (CCOG) and skills promoting behavior change using the 11 item Behavior Change Counseling Index (BECCI) tool. Results: Using the CCOG, there was a significantly greater mean score in the SP group for the category of ‘‘Gathering Information’’ in encounter one (p = 0.020). There were good to excellent ratings in all categories of the CCOG and the BECCI scores for the SP and the RP groups at both encounters. There was no significant differences in change scores from encounter one to encounter two between groups. Conclusions: Encounters with SPs and RPs are both effective strategies for dietetic students to demonstrate their communication and behavior change skills. Practice implications: Utilizing SPs is an effective experiential strategy for nutrition counseling curricula. ß 2014 Elsevier Ireland Ltd. All rights reserved.
Keywords: Standardized patients Nutrition counseling Experiential Communication skills, Dietetic students Behavior change counseling index Calgary Cambridge tool Simulation
1. Introduction Spahn and coworkers [1] define nutrition counseling as a ‘‘supportive process to set priorities, establish goals and create individualized action plans that acknowledge and foster responsibility for self-care’’ [1]. This involves learning communication skills which apply behavior change counseling (BCC) strategies, such as motivational interviewing, problem solving, goal setting, and selfmonitoring [1]. There is strong evidence (Grade I or good) that using these strategies in an outpatient counseling setting can promote positive eating behavior changes conducive to decreasing risk factors of cardiovascular disease [2] and delaying the onset of type 2 diabetes mellitus [3]. Furthermore, in patients having diabetes, there is strong evidence that behavioral therapy can improve weight control [4], fasting blood glucose concentrations,
* Corresponding author at: Drexel University, Nutrition Sciences, 1505 Race Street, Philadelphia, PA 19102, USA. Tel.: +1 610 864 0272. E-mail address: [email protected] (V.S. Schwartz). http://dx.doi.org/10.1016/j.pec.2014.11.009 0738-3991/ß 2014 Elsevier Ireland Ltd. All rights reserved.
glycosylated hemoglobin levels, as well as reduce risk factors of cardiovascular disease [5]. Furthermore, nutrition counseling skills improve when educational sessions are experiential, which allow dietetic students/ interns to practice their counseling and communication skills [6]. Experiential learning is facilitated through applied participation and reflection of a simulated or actual clinical experience [7]. Research has demonstrated that the use of experiential learning strategies, such as role playing, and patient simulated experiences with standardized patients (SPs) (i.e., trained persons acting as a real patients in a medical scenario) [8] may be more effective strategies for teaching communication skills essential to nutrition counseling, when compared to the traditional didactic methods of teaching communication skills [9]. Although a didactic lecture can be made more interactive with case examples, and discussion among the students [10], it does not provide the ‘‘hands on’’ approach of experiential learning [11]. Providing a laboratory for the nutrition counseling course utilizing either SPs or real patients (RPs) recruited from the community will allow dietetic students to combine their didactic knowledge with the application of clinical skills.
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173
For over 25 years, Drexel University has provided a nutrition counseling course for senior (4th year) undergraduate and second year graduate dietetic students, which has prepared students to apply their nutrition counseling skills for counseling real patients. After six weeks of classroom lectures, role playing counseling scenarios with classmates, student observation of prior student video counseling scenarios, and written examinations, the dietetic students completed two video recordings of their nutrition counseling encounters with RPs. The RPs were recruited from Drexel University’s staff and faculty by advertising in the online Drexel University newspaper. Although many of the students had excellent experiences with the RP, this method was problematic, due to the difficulty in recruiting RPs and scheduling conflicts. Although all RPs were screened by a Registered Dietitian (RD) prior to the student counseling experience, there were some clients that presented with more complex diet and medical histories, leading to dissimilar clinical experiences among students. Finally, it can be a negative experience when a student makes errors with an RP. SPs provide an experiential encounter for medical and other health care students for applying and evaluating communication skills [12–15]. Studies reveal that SPs are effective in improving communication skills among medical students [9] and students in other health professions [12–15]. When compared to role-playing, medical students rated SPs with significantly higher scores for being a ‘‘worthwhile’’ and ‘‘useful’’ strategy [16]. Similarly, SPs have been reported to be beneficial for providing practice and evaluation of dietetic students [17]. Hence, this study explored and compared the quality and improvement in communication and behavior change skills among dietetic students in two SP or RP practice counseling experiences. The goal was to explore the feasibility of using SPs for the experiential component of the nutrition counseling course. The difference in the quality of communication and behavior change skills in the two encounters compared between the SP group (SPG) and the RP group (RPG). It was hypothesized that there would not be a difference in communication skill scores and behavior change counseling scores between the two groups at the first and second experiential nutrition counseling encounter. Finally, it was hypothesized that there will be a difference between groups in the change in communication skill scores and behavior change counseling skills from encounter one to encounter two.
169
replacement RP could not be found, the dietetic student was switched to the SPG. Due to five ‘‘no shows’’ and/or cancellations among RPs during the winter quarter of 2013, one of the students was rescheduled with another RP and four of the students were switched from the RP to the SPG, due to inability to recruit RPs and the need to provide the two nutrition counseling encounters. Hence, 75 dietetic students were assigned to the SPG (n = 42) and RPG (n = 33), respectively. A comparison of demographics between the years of 2011 and 2013 for the RPG and the years 2012 and 2013 for the SPG revealed no significant difference. Finally, there was no significant differences in communication and behavior change skills when comparing the RPG group from 2011 and 2013, as well as comparing the SPG from 2012 and 2013. This will be reported later in the results. 2.2. Nutrition counseling curriculum from 2011 to 2013 Nutrition counseling is an advanced level required course for undergraduate (NFS 431) and graduate students (NFS 630) enrolled within the Didactic Program in Dietetics at Drexel University. The courses had similar objectives and learning activities during the years 2011–2013. Regardless of group assignment, students in the SPG and the RPG attended identical lectures for six weeks which emphasized communication and behavior change counseling skills. Learning assessments included role playing with classmates, review of video recorded counseling sessions of previous students and trained counselors and written examinations. At the conclusion of the didactic component of the course, the students were required to participate in two experiential nutrition counseling encounters. The first encounter consisted of probing a three day patient diet record, taking a medical and diet history, while the second encounter provided feedback on the three day record, educating on diet and setting food related behavioral goals. After the first encounter, all students reviewed their videorecording and completed a self-evaluation. They would then meet with the instructor to discuss the selfevaluation and the instructor’s evaluation. If they were assigned an SP, they would also receive immediate feedback from the SP assigned as their client. 2.3. Real patient group: Recruitment of real patients in 2011 and 2013
2. Methods 2.1. Design and allocation of subjects into groups This was an ex post facto design conducted at Drexel University, Philadelphia, Pennsylvania, United States in June of 2013. This study was approved by the Drexel University and Rutgers University Institutional Review Boards prior to the initiation of the study. As part of the course requirements, online video recordings were used to capture the dietetic student’s nutrition counseling encounters during the winter quarters (ten weeks between Jan and March) of 2011, 2012 and 2013. The dietetic student’s (n = 19) from 2011 had their experiential nutrition counseling encounters with real patients (RPs) and were placed in the real patient group (RPG), while the dietetic students from 2012 (n = 20), worked with standardized patients (SPs) and were placed in the standardized patient group (SPG). The dietetic students from 2013 (n = 35) were randomly assigned to be in either the SPG (n = 18) or the RPG (n = 17). As part of the requirement for the course, all students were required two experiential encounters with one patient for the nutrition counseling course. If an RP cancelled their scheduled appointment and was unable to reschedule at another time, the Principal Investigator (PI) recruited another RP who may or may not be of equivalent health status. If a
During week one of the nutrition counseling class in January of 2011 and 2013, RPs were recruited from the Drexel University community. An advertisement was placed in the university electronic newsletter. After the interested participants sent emails to the PI, they were contacted by telephone and screened for eligibility. Faculty and staff were accepted into the RPG, if they agreed to the video recordings, to keep a three-day food record and had counseling needs that were related to either weight management or a desire to improve their diet for the prevention of chronic diseases. Exclusion of participants included pre-existing conditions, such as diabetes requiring insulin or oral medications, cancer, end stage renal or hepatic disease or any other disease that required advanced medical nutrition therapy training by the dietetic student. 2.4. Standardized patient group: Standardized patient training in 2012 and 2013 Actors, who fit the physical criteria of a patient having abdominal obesity were selected from a photo list of actors from the simulation department during winter quarter of 2012 and 2013. All of the actors had previous experience working in patient simulations and had received general training by the simulation
170
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173
department at Drexel University, prior to training for the two nutrition counseling encounters. The training included review of role expectations, and guidelines for providing feedback after each encounter. The PI was the major instructor of the SP training in January of 2012 and 2013. 2.5. Experiential nutrition counseling encounters and training of raters All student encounters between 2011 and 2013 took place in the Drexel University Simulation Laboratory located at the Center City Campus in the debriefing room. This room was equipped with hidden cameras and audio equipment that transmitted and recorded the encounters to the computer lab room. The laboratory technician was responsible for taping each encounter and placing it online at a password protected site, so it was visible only to the individual student and the instructor. After the 2013 nutrition course was completed, trained raters were given access to the videotapes. The two trained raters were previous graduates of Drexel University, who took the nutrition counseling course and met or exceeded programmatic standards. They were given 4 h of instruction by the PI on use of the shortened Calgary-Cambridge Observation Guide (CCOG) [18,19] and the Behavior Change Counseling Index (BECCI) [20], while observing the video recordings. Each rater required a personal user name and password for access to the secure website containing the student recordings. 2.6. Inter-rater reliability Inter-rater reliability of the shortened 28 item CCOG and the 11 item BECCI was assessed by trained raters who independently rated the same videos (n = 23). Using the Kappa Measure of Agreement [21], results of the CCOG tool revealed 75% agreement or good agreement (p < 0.007) for encounter 1 and 100% agreement or very good agreement (p < 0.001) for encounter 2. When assessing the BECCI tool, the ratings were 100% agreement (p < 0.001) for encounter one and 84.3% or very good agreement with a significance of (p < 0.001) for encounter two. [21]. 2.7. Tools used to measure communication and behavior change skills The 28 item Calgary Cambridge Observation Guide Rating (CCOG) tool, used with permission [18,19], included categories of skills such as: ‘‘initiating the session,’’ ‘‘gathering information,’’ ‘‘providing structure to the consultation,’’ ‘‘building a relationship,’’ and ‘‘closing the session.’’ For individual skills, the dietetic student received a score of zero to two: ‘‘0’’ if the skill was not achieved, ‘‘1’’ if the skill needed improvement and ‘‘2’’ if the skill was carried out effectively. When evaluating the final means and medians for each category encompassing multiple skills, scores were carried out to two decimal points. To provide an interpretation for ratings containing two decimal points between 0 and 2 was divided into thirds, and 0 to 0.66 was considered poor, 0.67 to 1.33 was considered fair and 1.34 to 2 was good to excellent. Mean scores for each category and total communication score were calculated and recorded in SPSS 20 [22]. The 11 item Behavior Change Counseling Index (BECCI tool) [20] is specifically for measuring counseling skills for promoting behavioral change [20]. The 11 skills emphasized patient centered counseling, which encouraged the patient to talk about behavior change, elicited patient feedback on the plan, demonstrated empathy to patient concerns, respected patient choices and exchanged ideas on how to change behavior [23]. Scoring of the BECCI tool ranged from 0 = poor, 1 = barely passing, 2 = fair,
3 = good and 4 = excellent. Mean total behavior change scores were tallied and recorded in SPSS 20 [22]. 2.8. Power analysis and sample size Given our sample size of 75, G Power 3.1.5 [24] using the nonparametric Mann Whitney U between groups [25], we have an alpha error of 0.05, 1-beta error of 0.80 and a small effect size of 0.1. Due to no shows of real patients, more dietetic students were provided with standardized patients. Hence, the final sample size was 42 in the SPG and 33 in the RPG. 2.9. Statistical analysis 2.9.1. Baseline demographics Baseline demographics, such as age, gender, race and educational level (e.g., graduate versus undergraduate) of students were collected from accessing Drexel University student records. Age was recorded as the age when they were enrolled in the course. Due to an abnormal distribution for age, Mann Whitney U was used to analyze significant differences of age between groups [25]. Since the categorical variable ‘‘gender’’ contained two males and the assumption of a minimum of 5 in a cell was violated, Fisher Exact test [25] was used to analyze gender differences between SPG and RPG. When analyzing ‘‘Race’’, there were less than five in 50% of the cells (due to very small numbers in the categories of race). Hence, the group ‘‘Race’’ was divided into the categories of ‘‘Caucasian’’ and ‘‘Other’’. Pearson Chi-Square test for independence using Yates Continuity Correction [25] was then conducted for the categories of ‘‘race’’ and ‘‘education’’. 2.9.2. Statistical analysis of skills All skills within each category and total scores were averaged and placed in SPSS 20 statistical software [22]. Significance was set at a priori alpha level of p = 0.05. Descriptive statistics, independent sample t-tests and Mann Whitney U was used to analyze differences in encounter one and two for scores in the categories and total score of the CCOG tool and the total score for the BECCI tool between the SPG and the RPG. Differences in total score change between encounter one and two were analyzed with the independent sample t-test. Significance was set at a priori alpha level of p = 0.05. 3. Results 3.1. Baseline demographics The Mann–Whitney U test revealed no significant difference in the age levels in the SPG {(Md = 24.5, n = 42) and the RPG (Md = 24, n = 33), U = 584, p = 0.241}. There were no significant differences in gender, educational level or race between the SPG and RPG (Table 1). 3.2. Quality of communication skills and behavior change skills Raters evaluated 68 videos out of 75 potential first encounters (missing = 7) in the first encounter and 70 out of 75 potential videos for the second encounter (missing = 5). A reason for the missing tapes (n 13) included an inability to identify dietetic students on the recordings saved online at the website. The mean and median scores for both the SPG and RPG were good to excellent for all categories of CCOG as well as the total mean scores for CCOG and BECCI (Table 2). When scores for each encounter were inspected, only in the category of ‘‘Gathering Information’’, did the SPG have significantly greater scores (p = 0.020) than the RPG in encounter one.
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173 Table 1 Baseline demographic characteristics of participants (n = 75) in the standardized patient group (SPG = 42), compared to the real patient group (RPG n = 33). Characteristic
SPG (n = 42) Total no. (%)
RPG (n = 33) a
Total no. (%)
Gender Female Male
42 (100%) 0 (0%)
31 (94.1%) 2 (5.9%)
Education Graduate Undergraduate
27 (64.3%) 15 (35.7%)
18 (55.9%) 15 (44.1%)
Raced Caucasian Other
30 (71.4%) 12 (28.6)
28 (84.8%) 5 (15.2%)
x2
p-value
0.80
0.371b
Table 3 Difference between RPG 2011 and 2013 from RPG and differences between 2012 and 2013 SPG from SPG. Groups
a
RPG 2011 RPG 2013 0.381c
0.537c SPG 2012 SPG 2013
1.21c
0.271c a b
a
Total number and % within group assignment. b Fisher’s exact test was reported as cells had expected counts
Contents lists available at ScienceDirect
Patient Education and Counseling journal homepage: www.elsevier.com/locate/pateducou
Medical education
Comparison of standardized patients and real patients as an experiential teaching strategy in a nutrition counseling course for dietetic students Vicki S. Schwartz a,b,*, Pamela Rothpletz-Puglia b, Robert Denmark b, Laura Byham-Gray b a b
Drexel University, Department of Nutrition Sciences, Philadelphia, USA Rutgers University, New Jersey School of Health Related Professions, Department of Nutritional Sciences, Newark, USA
A R T I C L E I N F O
A B S T R A C T
Article history: Received 15 May 2014 Received in revised form 27 October 2014 Accepted 11 November 2014
Objectives: To compare the quality of communication and behavioral change skills among dietetic students having two nutrition encounters with either a real patient or a standardized patient in the simulation laboratory at Drexel University, Philadelphia, PA, United States. Methods: A retrospective analysis of video recordings (n = 138) containing nutrition encounters of dietetic students (n = 75) meeting with a standardized patient (SP) or a real patient (RP). Trained raters evaluated communication skills with the 28 item Calgary Cambridge Observation Guide (CCOG) and skills promoting behavior change using the 11 item Behavior Change Counseling Index (BECCI) tool. Results: Using the CCOG, there was a significantly greater mean score in the SP group for the category of ‘‘Gathering Information’’ in encounter one (p = 0.020). There were good to excellent ratings in all categories of the CCOG and the BECCI scores for the SP and the RP groups at both encounters. There was no significant differences in change scores from encounter one to encounter two between groups. Conclusions: Encounters with SPs and RPs are both effective strategies for dietetic students to demonstrate their communication and behavior change skills. Practice implications: Utilizing SPs is an effective experiential strategy for nutrition counseling curricula. ß 2014 Elsevier Ireland Ltd. All rights reserved.
Keywords: Standardized patients Nutrition counseling Experiential Communication skills, Dietetic students Behavior change counseling index Calgary Cambridge tool Simulation
1. Introduction Spahn and coworkers [1] define nutrition counseling as a ‘‘supportive process to set priorities, establish goals and create individualized action plans that acknowledge and foster responsibility for self-care’’ [1]. This involves learning communication skills which apply behavior change counseling (BCC) strategies, such as motivational interviewing, problem solving, goal setting, and selfmonitoring [1]. There is strong evidence (Grade I or good) that using these strategies in an outpatient counseling setting can promote positive eating behavior changes conducive to decreasing risk factors of cardiovascular disease [2] and delaying the onset of type 2 diabetes mellitus [3]. Furthermore, in patients having diabetes, there is strong evidence that behavioral therapy can improve weight control [4], fasting blood glucose concentrations,
* Corresponding author at: Drexel University, Nutrition Sciences, 1505 Race Street, Philadelphia, PA 19102, USA. Tel.: +1 610 864 0272. E-mail address: [email protected] (V.S. Schwartz). http://dx.doi.org/10.1016/j.pec.2014.11.009 0738-3991/ß 2014 Elsevier Ireland Ltd. All rights reserved.
glycosylated hemoglobin levels, as well as reduce risk factors of cardiovascular disease [5]. Furthermore, nutrition counseling skills improve when educational sessions are experiential, which allow dietetic students/ interns to practice their counseling and communication skills [6]. Experiential learning is facilitated through applied participation and reflection of a simulated or actual clinical experience [7]. Research has demonstrated that the use of experiential learning strategies, such as role playing, and patient simulated experiences with standardized patients (SPs) (i.e., trained persons acting as a real patients in a medical scenario) [8] may be more effective strategies for teaching communication skills essential to nutrition counseling, when compared to the traditional didactic methods of teaching communication skills [9]. Although a didactic lecture can be made more interactive with case examples, and discussion among the students [10], it does not provide the ‘‘hands on’’ approach of experiential learning [11]. Providing a laboratory for the nutrition counseling course utilizing either SPs or real patients (RPs) recruited from the community will allow dietetic students to combine their didactic knowledge with the application of clinical skills.
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173
For over 25 years, Drexel University has provided a nutrition counseling course for senior (4th year) undergraduate and second year graduate dietetic students, which has prepared students to apply their nutrition counseling skills for counseling real patients. After six weeks of classroom lectures, role playing counseling scenarios with classmates, student observation of prior student video counseling scenarios, and written examinations, the dietetic students completed two video recordings of their nutrition counseling encounters with RPs. The RPs were recruited from Drexel University’s staff and faculty by advertising in the online Drexel University newspaper. Although many of the students had excellent experiences with the RP, this method was problematic, due to the difficulty in recruiting RPs and scheduling conflicts. Although all RPs were screened by a Registered Dietitian (RD) prior to the student counseling experience, there were some clients that presented with more complex diet and medical histories, leading to dissimilar clinical experiences among students. Finally, it can be a negative experience when a student makes errors with an RP. SPs provide an experiential encounter for medical and other health care students for applying and evaluating communication skills [12–15]. Studies reveal that SPs are effective in improving communication skills among medical students [9] and students in other health professions [12–15]. When compared to role-playing, medical students rated SPs with significantly higher scores for being a ‘‘worthwhile’’ and ‘‘useful’’ strategy [16]. Similarly, SPs have been reported to be beneficial for providing practice and evaluation of dietetic students [17]. Hence, this study explored and compared the quality and improvement in communication and behavior change skills among dietetic students in two SP or RP practice counseling experiences. The goal was to explore the feasibility of using SPs for the experiential component of the nutrition counseling course. The difference in the quality of communication and behavior change skills in the two encounters compared between the SP group (SPG) and the RP group (RPG). It was hypothesized that there would not be a difference in communication skill scores and behavior change counseling scores between the two groups at the first and second experiential nutrition counseling encounter. Finally, it was hypothesized that there will be a difference between groups in the change in communication skill scores and behavior change counseling skills from encounter one to encounter two.
169
replacement RP could not be found, the dietetic student was switched to the SPG. Due to five ‘‘no shows’’ and/or cancellations among RPs during the winter quarter of 2013, one of the students was rescheduled with another RP and four of the students were switched from the RP to the SPG, due to inability to recruit RPs and the need to provide the two nutrition counseling encounters. Hence, 75 dietetic students were assigned to the SPG (n = 42) and RPG (n = 33), respectively. A comparison of demographics between the years of 2011 and 2013 for the RPG and the years 2012 and 2013 for the SPG revealed no significant difference. Finally, there was no significant differences in communication and behavior change skills when comparing the RPG group from 2011 and 2013, as well as comparing the SPG from 2012 and 2013. This will be reported later in the results. 2.2. Nutrition counseling curriculum from 2011 to 2013 Nutrition counseling is an advanced level required course for undergraduate (NFS 431) and graduate students (NFS 630) enrolled within the Didactic Program in Dietetics at Drexel University. The courses had similar objectives and learning activities during the years 2011–2013. Regardless of group assignment, students in the SPG and the RPG attended identical lectures for six weeks which emphasized communication and behavior change counseling skills. Learning assessments included role playing with classmates, review of video recorded counseling sessions of previous students and trained counselors and written examinations. At the conclusion of the didactic component of the course, the students were required to participate in two experiential nutrition counseling encounters. The first encounter consisted of probing a three day patient diet record, taking a medical and diet history, while the second encounter provided feedback on the three day record, educating on diet and setting food related behavioral goals. After the first encounter, all students reviewed their videorecording and completed a self-evaluation. They would then meet with the instructor to discuss the selfevaluation and the instructor’s evaluation. If they were assigned an SP, they would also receive immediate feedback from the SP assigned as their client. 2.3. Real patient group: Recruitment of real patients in 2011 and 2013
2. Methods 2.1. Design and allocation of subjects into groups This was an ex post facto design conducted at Drexel University, Philadelphia, Pennsylvania, United States in June of 2013. This study was approved by the Drexel University and Rutgers University Institutional Review Boards prior to the initiation of the study. As part of the course requirements, online video recordings were used to capture the dietetic student’s nutrition counseling encounters during the winter quarters (ten weeks between Jan and March) of 2011, 2012 and 2013. The dietetic student’s (n = 19) from 2011 had their experiential nutrition counseling encounters with real patients (RPs) and were placed in the real patient group (RPG), while the dietetic students from 2012 (n = 20), worked with standardized patients (SPs) and were placed in the standardized patient group (SPG). The dietetic students from 2013 (n = 35) were randomly assigned to be in either the SPG (n = 18) or the RPG (n = 17). As part of the requirement for the course, all students were required two experiential encounters with one patient for the nutrition counseling course. If an RP cancelled their scheduled appointment and was unable to reschedule at another time, the Principal Investigator (PI) recruited another RP who may or may not be of equivalent health status. If a
During week one of the nutrition counseling class in January of 2011 and 2013, RPs were recruited from the Drexel University community. An advertisement was placed in the university electronic newsletter. After the interested participants sent emails to the PI, they were contacted by telephone and screened for eligibility. Faculty and staff were accepted into the RPG, if they agreed to the video recordings, to keep a three-day food record and had counseling needs that were related to either weight management or a desire to improve their diet for the prevention of chronic diseases. Exclusion of participants included pre-existing conditions, such as diabetes requiring insulin or oral medications, cancer, end stage renal or hepatic disease or any other disease that required advanced medical nutrition therapy training by the dietetic student. 2.4. Standardized patient group: Standardized patient training in 2012 and 2013 Actors, who fit the physical criteria of a patient having abdominal obesity were selected from a photo list of actors from the simulation department during winter quarter of 2012 and 2013. All of the actors had previous experience working in patient simulations and had received general training by the simulation
170
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173
department at Drexel University, prior to training for the two nutrition counseling encounters. The training included review of role expectations, and guidelines for providing feedback after each encounter. The PI was the major instructor of the SP training in January of 2012 and 2013. 2.5. Experiential nutrition counseling encounters and training of raters All student encounters between 2011 and 2013 took place in the Drexel University Simulation Laboratory located at the Center City Campus in the debriefing room. This room was equipped with hidden cameras and audio equipment that transmitted and recorded the encounters to the computer lab room. The laboratory technician was responsible for taping each encounter and placing it online at a password protected site, so it was visible only to the individual student and the instructor. After the 2013 nutrition course was completed, trained raters were given access to the videotapes. The two trained raters were previous graduates of Drexel University, who took the nutrition counseling course and met or exceeded programmatic standards. They were given 4 h of instruction by the PI on use of the shortened Calgary-Cambridge Observation Guide (CCOG) [18,19] and the Behavior Change Counseling Index (BECCI) [20], while observing the video recordings. Each rater required a personal user name and password for access to the secure website containing the student recordings. 2.6. Inter-rater reliability Inter-rater reliability of the shortened 28 item CCOG and the 11 item BECCI was assessed by trained raters who independently rated the same videos (n = 23). Using the Kappa Measure of Agreement [21], results of the CCOG tool revealed 75% agreement or good agreement (p < 0.007) for encounter 1 and 100% agreement or very good agreement (p < 0.001) for encounter 2. When assessing the BECCI tool, the ratings were 100% agreement (p < 0.001) for encounter one and 84.3% or very good agreement with a significance of (p < 0.001) for encounter two. [21]. 2.7. Tools used to measure communication and behavior change skills The 28 item Calgary Cambridge Observation Guide Rating (CCOG) tool, used with permission [18,19], included categories of skills such as: ‘‘initiating the session,’’ ‘‘gathering information,’’ ‘‘providing structure to the consultation,’’ ‘‘building a relationship,’’ and ‘‘closing the session.’’ For individual skills, the dietetic student received a score of zero to two: ‘‘0’’ if the skill was not achieved, ‘‘1’’ if the skill needed improvement and ‘‘2’’ if the skill was carried out effectively. When evaluating the final means and medians for each category encompassing multiple skills, scores were carried out to two decimal points. To provide an interpretation for ratings containing two decimal points between 0 and 2 was divided into thirds, and 0 to 0.66 was considered poor, 0.67 to 1.33 was considered fair and 1.34 to 2 was good to excellent. Mean scores for each category and total communication score were calculated and recorded in SPSS 20 [22]. The 11 item Behavior Change Counseling Index (BECCI tool) [20] is specifically for measuring counseling skills for promoting behavioral change [20]. The 11 skills emphasized patient centered counseling, which encouraged the patient to talk about behavior change, elicited patient feedback on the plan, demonstrated empathy to patient concerns, respected patient choices and exchanged ideas on how to change behavior [23]. Scoring of the BECCI tool ranged from 0 = poor, 1 = barely passing, 2 = fair,
3 = good and 4 = excellent. Mean total behavior change scores were tallied and recorded in SPSS 20 [22]. 2.8. Power analysis and sample size Given our sample size of 75, G Power 3.1.5 [24] using the nonparametric Mann Whitney U between groups [25], we have an alpha error of 0.05, 1-beta error of 0.80 and a small effect size of 0.1. Due to no shows of real patients, more dietetic students were provided with standardized patients. Hence, the final sample size was 42 in the SPG and 33 in the RPG. 2.9. Statistical analysis 2.9.1. Baseline demographics Baseline demographics, such as age, gender, race and educational level (e.g., graduate versus undergraduate) of students were collected from accessing Drexel University student records. Age was recorded as the age when they were enrolled in the course. Due to an abnormal distribution for age, Mann Whitney U was used to analyze significant differences of age between groups [25]. Since the categorical variable ‘‘gender’’ contained two males and the assumption of a minimum of 5 in a cell was violated, Fisher Exact test [25] was used to analyze gender differences between SPG and RPG. When analyzing ‘‘Race’’, there were less than five in 50% of the cells (due to very small numbers in the categories of race). Hence, the group ‘‘Race’’ was divided into the categories of ‘‘Caucasian’’ and ‘‘Other’’. Pearson Chi-Square test for independence using Yates Continuity Correction [25] was then conducted for the categories of ‘‘race’’ and ‘‘education’’. 2.9.2. Statistical analysis of skills All skills within each category and total scores were averaged and placed in SPSS 20 statistical software [22]. Significance was set at a priori alpha level of p = 0.05. Descriptive statistics, independent sample t-tests and Mann Whitney U was used to analyze differences in encounter one and two for scores in the categories and total score of the CCOG tool and the total score for the BECCI tool between the SPG and the RPG. Differences in total score change between encounter one and two were analyzed with the independent sample t-test. Significance was set at a priori alpha level of p = 0.05. 3. Results 3.1. Baseline demographics The Mann–Whitney U test revealed no significant difference in the age levels in the SPG {(Md = 24.5, n = 42) and the RPG (Md = 24, n = 33), U = 584, p = 0.241}. There were no significant differences in gender, educational level or race between the SPG and RPG (Table 1). 3.2. Quality of communication skills and behavior change skills Raters evaluated 68 videos out of 75 potential first encounters (missing = 7) in the first encounter and 70 out of 75 potential videos for the second encounter (missing = 5). A reason for the missing tapes (n 13) included an inability to identify dietetic students on the recordings saved online at the website. The mean and median scores for both the SPG and RPG were good to excellent for all categories of CCOG as well as the total mean scores for CCOG and BECCI (Table 2). When scores for each encounter were inspected, only in the category of ‘‘Gathering Information’’, did the SPG have significantly greater scores (p = 0.020) than the RPG in encounter one.
V.S. Schwartz et al. / Patient Education and Counseling 98 (2015) 168–173 Table 1 Baseline demographic characteristics of participants (n = 75) in the standardized patient group (SPG = 42), compared to the real patient group (RPG n = 33). Characteristic
SPG (n = 42) Total no. (%)
RPG (n = 33) a
Total no. (%)
Gender Female Male
42 (100%) 0 (0%)
31 (94.1%) 2 (5.9%)
Education Graduate Undergraduate
27 (64.3%) 15 (35.7%)
18 (55.9%) 15 (44.1%)
Raced Caucasian Other
30 (71.4%) 12 (28.6)
28 (84.8%) 5 (15.2%)
x2
p-value
0.80
0.371b
Table 3 Difference between RPG 2011 and 2013 from RPG and differences between 2012 and 2013 SPG from SPG. Groups
a
RPG 2011 RPG 2013 0.381c
0.537c SPG 2012 SPG 2013
1.21c
0.271c a b
a
Total number and % within group assignment. b Fisher’s exact test was reported as cells had expected counts
