1040-5488/14/9101-0121/0 VOL. 91, NO. 1, PP. 121Y128 OPTOMETRY AND VISION SCIENCE Copyright * 2013 American Academy of Optometry
ORIGINAL ARTICLE
Training Students with Patient Actors Improves Communication: A Pilot Study Heather A. Anderson*, Jack Young†, Danica Marrelli‡, Rudolph Black‡, Kimberly Lambreghts§, and Michael D. Twa*
ABSTRACT Purpose. Effective patient communication is correlated with better health outcomes and patient satisfaction, but is challenging to train, particularly with difficult clinical scenarios such as loss of sight. In this pilot study, we evaluated the use of simulated patient encounters with actors to train optometric students. Methods. Students were recorded during encounters with actors and assigned to an enrichment group performing five interactions with instructor feedback (n = 6) or a no-enrichment group performing two interactions without feedback (n = 4). Student performance on first and last encounters was scored with (1) subjective rating of performance change using a visual analog scale (anchors: much worse/much better), (2) yes/no response: Would you recommend this doctor to a friend/ relative?, and (3) average score on questions from the American Board of Internal Medicine (ABIM) assessment of doctor communication skills. Three clinical instructors, masked to student group assignments and the order of patient encounters they viewed, provided scores in addition to self-evaluation by students and patient-actors. Results. Using the visual analog scale, students who received enrichment were rated more improved than the noenrichment group by masked examiners (+18 vs. j11% p = 0.04) and self-evaluation (+79 vs. +27% p = 0.009), but not by actors (+31 vs. +43%). The proportion of students recommended significantly increased following enrichment for masked examiners (61% vs. 94%; p G 0.001), but not actors (100 vs. 83%). Average ABIM assessment scores were not significantly different by any rating group: masked instructors, actors, or self-ratings. Conclusions. The findings of this study suggest five simulated patient encounters with feedback result in measurable improvement in student-patient communication skills as rated by masked examiners. (Optom Vis Sci 2014;91:121Y128) Key Words: actor, communication skills, feedback, optometric education, patient simulation, physician-patient communication, standardized patient, student training
E
ffective communication skills are essential in every healthcare discipline and recognition of their importance has resulted in specific provider-patient communication training programs in medicine, nursing, dentistry, and other health professions, yet there are few examples in vision care. There is a sizeable body of published research showing that effective
*OD, PhD, FAAO † MFA ‡ OD, FAAO § RN, OD, FAAO College of Optometry (HAA, DM, RB, KL, MDT) and The School of Theatre & Dance (JY), University of Houston, Houston, Texas. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.optvissci.com).
communication is highly correlated with better health outcomes,1,2 greater patient satisfaction,3 better patient compliance with recommended treatments,4 and lower cost of care.5Y7 Loss of sight is one of the most common fears, second only to fear of cancer and surpassing fear of several other important public health concerns such as diabetes or heart disease.4,8,9 Vision loss is often associated with a loss of mobility, loss of independence, and social isolation, and has been associated with an increased risk of suicide behavior.8,10Y13 Optometrists caring for patients with these concerns should be capable of compassionately conveying news about permanent loss of sight, loss of driving privileges, and the seriousness of acute and chronic ocular diseases. Nevertheless, student clinicians typically receive limited specific training to help them prepare for these challenging patient encounters. Training methods commonly used to improve providerpatient communication include individual mentoring, traditional
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
122 Patient Actors Improve Clinician TrainingVAnderson et al.
lectures on communication skills, role-playing, and demonstrations. Several studies have demonstrated the value of using actors as patients to augment the number and variety of clinical scenarios that student clinicians encounter during their training.14Y18 The use of patient-actors is ideally suited for training student clinicians to navigate emotionally charged and complex interviews where roleplaying can effectively simulate and control the encounter while reducing untoward consequences for students and patients alike. Moreover, there is evidence that these experiences can have a measurable and lasting impact on students’ ability to communicate effectively with their patients.19Y22 Providing student feedback is a critical component in training student clinicians and determining the most effective methods is a growing field of research, particularly among health educators.23Y26 A recent shift in feedback practices has been to utilize a studentcentered approach which incorporates self-assessments, such as the evaluation of video-recorded performances. Medical students report that this strategy is a useful learning tool and is more effective than traditional qualitative instructor feedback.27 A recent review article supports the use of self-evaluation of video recordings, noting that it is only effective when utilized in combination with expert feedback.28 The primary motivation for this pilot study was to develop training scenarios that would permit student clinicians to practice patient communication skills and to develop methods to assess the effects of this training. In this study, we evaluate the effects of active student participation in patient interviewing and counseling when combined with enrichment (instructor feedback, supplemented with viewing of video-recorded encounters).
METHODS This study was approved by the University of Houston Committee for the Protection of Human Subjects and adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from all study participants including written permission for publication of video segments from those who elected to give permission. Three actors were recruited through the University of Houston School of Theatre & Dance to portray patients in five different common clinical scenarios developed by the investigators (HAA, MDT). The five cases portrayed scenarios with significant demands on doctor to patient communication and included (1) recent onset retinal detachment in an individual scheduled to travel on vacation, (2) loss of eligibility for driving privileges in an elderly patient with age-related macular degeneration, (3) a patient with progressive cataracts taking homeopathic herbal supplements as treatment, (4) a patient wearing contact lenses who is non-compliant with lens hygiene and wearing schedule who is beginning to develop ocular complications, and (5) a sight-threatening corneal infection in a contact lensYwearing patient who refuses to wear glasses and is participating in a wedding in 2 days. Actors were provided information about their character (demographics, social behaviors, etc.), their reason for seeking care, additional specific details regarding their visual complaints, and pertinent medical history. They were directed to improvise additional details as necessary and given some (minimal) direction to encourage consistency during the
encounter with the students; however, they cannot be classified as professional standardized patients given that they did not undergo the extensive formalized training customary of that profession. Student clinicians were invited via email to participate in the study during the summer following completion of the second year of their 4-year professional training at University of Houston College of Optometry. Students at this stage in the curriculum have engaged in limited patient encounters (less than 40) and were thus considered novice, early-stage clinicians by study investigators due to their limited experience. The first ten students to respond were enrolled in the study. Students were assigned by the investigators to one of two study groups: an enrichment group that performed all five patient interactions with instructor feedback following each interaction (n = 6), or a comparison group that received no enrichment, performing only the first and last patient interactions without instructor feedback (n = 4). Because students in both groups were actively receiving clinical training and patient encounters outside of their participation in this study, we compared student performance at the beginning and the end of the study to account for improvements in performance that may have been attributable to other learning experiences. To encourage consistency across subject groups, a single actor who was unaware of student group assignments portrayed the first and last patient encounters (scenarios 1 and 5), which were used for statistical comparisons. Two other actors portrayed three additional clinical scenarios for the enrichment group (scenarios 2, 3, and 4). Group assignment was based upon student schedules with regards to actor availability and not upon student choice or investigators’ perceptions of student communication skills. Scheduling considerations were the determining factor for group assignments and the reason for the difference in the number of subjects between groups. Patient interactions occurred in the Family Practice Service at the University Eye Institute. Students were instructed to enter the examination room and perform an initial interview and case history with the patient. When finished, they summarized their findings for the investigators (HAA, MDT). The investigators then provided students with the associated clinical examination findings; students did not perform the physical examinations. Students were permitted to ask questions and discuss these findings and their understanding of the case with the investigators. The student clinicians were then instructed to return to the examination room and counsel the patient on their diagnosis and treatment plans. This sequence is consistent with patient encounters that students experience at the University Eye Institute. Two video cameras were placed in the examination room permitting investigators to record and monitor interactions between student clinicians and the patient actors. The student clinicians were aware of the cameras and consented to the video recording. A sample of one of these encounters is shown in Video 1 (available at http://links.lww.com/OPX/A150) with the student’s face blurred to maintain anonymity. The investigators made notes regarding the student’s performance while monitoring the live video feed during the initial interview and the final consultation of each encounter. Immediately following the encounter,
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
the student clinician, patient actor, and investigators each independently completed a written performance evaluation using instruments described in detail below. Students in the noenrichment group were not permitted to see their performance evaluations, were not given any verbal feedback, and were not allowed to view the recordings until completion of the study. Their performance evaluations were recorded for later analysis. For students in the enrichment group, the written evaluations were used along with verbal feedback in individual faceto-face discussions with the instructors (HAA, MDT) after each encounter. Both strengths and weaknesses of the encounter were discussed. The instructors shared specific notes regarding the individual encounters, as well as encouraged the student to selfevaluate by verbalizing their own perceived areas of strength and weaknesses. Strategies for improvement were offered directly from the instructors, but students were also led to discover their own solutions to weaknesses through facilitative discussions. Throughout the discussions, students viewed segments of the video recordings that were selected by the investigators to illustrate specific teaching points. Students were not given recordings to view on their own time until completion of the study. Feedback sessions lasted 20 minutes on average. Student performance was also evaluated at a later date by three additional, independent clinical instructors who watched the videos and were masked to the student group assignments, the order of the encounters (first/last vs. last/first), and from the ratings of other instructors. Three metrics were used to score student performance: (1) subjective rating of change in performance between first and last encounter using a visual analog scale (anchors: much worse/much better), (2) yes/no response: ‘‘Would you recommend this doctor to a friend or relative?’’, and (3) average cumulative score on a subset of questions based on the American Board of Internal Medicine (ABIM) assessment of doctor communication skills. The ABIM assessment is an instrument created as part of the continuous professional development program for maintenance of board certification for internal medicine physicians.29,30 It is designed to evaluate a variety of physician qualities related to patient communication on a scale from 1 to 5 and has also been used in modified form in previous studies evaluating the training of clinician/ patient interactions.1,22,31Y33 For this study, we selected nine questions which targeted characteristics applicable to patient history and consultation: (1) disclosure of information, (2) warmth, (3) respect, (4) listening skills, (5) genuineness, (6) involving the patient in decision making, (7) encouraging questions, (8) proper patient education, and (9) use of lay language. Questions were modified to use appropriate tense for self-evaluation, patient evaluation, and instructor evaluation (see Appendix A, available at http://links.lww.com/OPX/A151).
123
investigators. Visual analog scores were measured using a millimeter rule and normalized over a range of j1 to +1 where j1 represented maximum possible regression in performance and +1 represented maximum possible improvement in performance. Change in scores were compared between subject groups (enrichment or no enrichment) using Student t test (patient and self-rating: six observations for enrichment and four observations for noenrichment) and two-way ANOVA with repeated measures (masked graders: 18 observations for enrichment and 12 observations for no enrichment). Statistical significance was declared at p G 0.05. Statistical analysis was performed using Stata (SE 11.2 for x86-64; StataCorp LP, College Station, TX). A test of proportions was used to compare responses to whether or not student clinicians were recommended to friends or family. The ABIM survey responses were summarized as a mean response score according to methods described by Lipner and colleagues.34 Scores for each student group were compared using Student t test.
RESULTS Visual Analog Scale Students who received enrichment were rated more improved by the masked clinical instructors between their first and last encounters than students who did not receive enrichment (+18 vs. j11%, two-way repeated measures ANOVA, F = 4.59, df = 1, p = 0.04) (Fig. 1). Students who did not receive enrichment rated themselves improved between their first and last encounters, but the level of self-rated improvement was significantly higher for the students who received enrichment (+27 vs. +79%, ANOVA, F = 11.64, df = 1, p = 0.009). There was no significant difference in the level of improvement rated by the patient actor for the two student groups (+43% no enrichment vs. +31% enrichment, ANOVA, F = 0.08 df = 1, p = 0.78). Within the enrichment group, the level of improvement between first to last encounter reached significance as rated by the students themselves (+79%, one-sample t test, t = 8.77, df = 5, p G 0.001) and by the masked clinical instructors (+18%, one-sample t test, t = 2.20, df = 17, p = 0.04), but not by the patient actor (+31%, one-sample t test, t = 1.32, df = 5, p = 0.24). Given the small sample size, individual student performance was also evaluated as scored by the masked clinical instructors. Change scores for the three masked clinical instructors were averaged for each student and compared between the enrichment and no-enrichment groups. Five of the six students in the enrichment group had a positive change in performance from first to last encounter, whereas three of the four students in the no-enrichment group had a negative change in performance from first to last encounter.
Would You Recommend This Doctor? Analysis Student performance on each of the three metrics was analyzed separately for each rating group: student self-evaluations, patient-actors, and masked clinical instructors. Results are not reported for the evaluations provided by the study investigators (HAA, MDT) because these evaluations were used for student feedback and would therefore likely reflect the bias of the
The proportion of students recommended by the masked clinical instructors significantly increased in the enrichment group between first and last encounter (61 vs. 94%, one-sample test of proportion, z = j5.60, p G 0.001) as opposed to those who did not receive enrichment (83 vs. 92%, one-sample test of proportion, z = j1.15, p = 0.25; Figs. 2 and 3). The proportion of students recommended by the patient-actor on
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
124 Patient Actors Improve Clinician TrainingVAnderson et al.
FIGURE 1. Change in performance obtained from the visual analog scale grouped by enrichment. Combined mean scores are shown for masked clinical instructors, patient-actor, and self-evaluation by the student clinician. Positive values indicate that raters observed improvement between first and last encounters.
first encounter was 100% and was 83% on their last encounter for the enrichment group. This was 75% on first encounter versus 100% on the last encounter in the no-enrichment group.
Communications Assessment Survey This instrument was comprised of nine items (Likert scale), with individual item scores ranging from 1 to 5. Mean cumulative
scores on the ABIM assessment could range from 9 to 45, with higher scores indicating better performance. Mean scores from first and last encounters are shown for the enrichment and noenrichment groups as rated by the three types of evaluators (masked clinical instructors, patient-actor, and self-evaluation) in Table 1. Similar to the visual analog scale, the trend of positive change was greatest for the enrichment group as rated by the masked instructors (mean = 3.4 T 1.75) and the student’s
FIGURE 2. Responses from masked clinical instructors to the question: ‘‘Would you recommend this doctor to a friend or relative?’’ Responses are shown for the first and last encounters for students receiving no enrichment and enrichment. Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
125
FIGURE 3. Responses from patient-actors to the question: ‘‘Would you recommend this doctor to a friend or relative?’’ Responses are shown for the first and last encounters for students receiving no enrichment and enrichment.
self-evaluation (mean = 4.33 T 2.04); however, the difference in mean cumulative performance on the ABIM Communications Assessment did not reach statistical significance from first to last encounter for either the enrichment or no enrichment groups for any of the three types of evaluators (one-sample t test, range of p = 0.09 to 0.58). Individual student scores derived from a mean of the three masked clinical instructors were also compared between enrichment and no-enrichment groups. Five of six students in the enrichment group had an increase in ABIM score from first to last encounter, whereas three of four students in the no-enrichment group had a decrease or no change in ABIM score from first to last encounter. These findings were in agreement with the same individual trends in performance observed with the visual analog scale.
DISCUSSION This pilot study found a significant improvement in patient communication skills of student clinicians who participated in five clinical scenarios with immediate instructor feedback and the opportunity to view their video-taped sessions, versus students who participated in two encounters with no feedback. By design, the enrichment group received multiple types of instructional enrichment (feedback, multiple encounters, and viewing portions of the video recordings) that were not offered to the noenrichment group. This pilot study was not designed to show specifically which of these three elements of instructional enrichment resulted in the observed improvement in performance; it may have been a single element or some combination. Consistent with our results, previous studies have demonstrated that instructor feedback paired with self-evaluation of video recordings results in increased learning relative to isolated self-evaluation
of videos.28 This supports the hypothesis that feedback during enrichment is an essential component and would be consistent with the improvement in the enrichment group observed in this study. However, a recent study of patient communication in nursing students found no increase in performance between first and second encounters for those receiving immediate feedback and group discussion with video recordings.35 This finding may suggest that individual feedback is more effective than group encounters, or that the number of encounters is important as well. Because student performance was only evaluated by the masked clinical instructors for the first and last cases, it is not possible to determine at what point the students receiving enrichment began to improve. Given the inability to identify the source and timing of the improvement, further study is
TABLE 1.
Mean scores for the communication assessment from first and last encounters compared by group as rated by the three types of evaluators. No differences reached statistical significance Group: rater Enrichment Instructor Patient Self No enrichment Instructor Patient Self
First encounter (mean T SD)
Last encounter (mean T SD)
3.5 T 0.79 4.7 T 0.15 4.1 T 0.43
3.9 T 0.34 4.7 T 0.11 4.5 T 0.51
4.1 T 0.61 4.6 T 0.28 4.3 T 0.53
4.0 T 0.41 4.8 T 0.91 4.5 T 0.45
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
126 Patient Actors Improve Clinician TrainingVAnderson et al.
warranted to better understand the impact of each aspect of instructional enrichment provided in this study. Specifically, there is a need for evidence regarding the most influential methods of instructional enrichment (e.g., immediate instructor feedback or self-evaluation of video-taped encounters), the number of simulated clinical encounters necessary for measurable improvement, and the most influential types of patient encounters (e.g., simple or complex). The significant findings in this study were based upon an average of scores from three masked instructors. Two of the masked clinical instructors had previously taught these student clinicians in didactic courses in the first and second years of their professional training and therefore had some prior knowledge of the student’s academic performance. The third masked clinical instructor served as an attending clinical instructor to four of the study participants. However, these students were evenly distributed between the enrichment and no-enrichment groups. An individual analysis of the masked instructors revealed that evaluators varied in their overall ratings, but improvement from first to last encounter was consistently observed among all three instructors. More structured grading criteria for the instructors would have helped to reduce the variability in overall ratings and is planned for future evaluations. We did not find significant differences in student clinician communication skills between enrichment and no-enrichment groups as rated by the patient-actor. While real patient feedback is highly valued in the clinical setting, we found that the feedback of the patient actor was not a robust measure of student performance and that actors were reticent to score the students poorly. This limited the range of actor-reported scores to a narrow and relatively high range with each of the instruments. When asked, actors stated that they were concerned that scoring the students poorly would negatively impact the students’ standing in some way. This limitation could be addressed by using professional standardized patients trained to provide reliable feedback. Conversely, we found that the students tended to rate themselves highly when performing self-evaluations. This finding is consistent with other studies that report medical residents, particularly those with lower performance, overestimate their clinical communication skills when compared to ratings by external reviewers.36,37 The fact that students in this study knew they were being assessed on their communications skills and they were aware of their group assignments (enrichment vs. no enrichment) may have influenced and confounded how students rated their own skills. The students’ perceived improvement in communication skills may simply have been influenced by their knowledge that their communication skills were under study, rather than the direct effect of enrichment. Clearly, the students felt more confident in their ability following these enrichment experiences (Fig. 1), but their estimation of how much they improved was far greater than the improvement reported by masked graders. Nevertheless, significant improvement in performance was also observed by the masked instructors, further supporting our finding that enrichment in the form of additional mock-patient encounters and instructor feedback was associated with better patient communication skills. An observation that we did not anticipate was the emotional impact that these mock-patient encounters had on student
clinicians. Given that these were novice clinicians, they had limited or no previous experience with difficult patient counseling scenarios involving life-changing events, such as the loss of driving privileges or the loss of sight. Although students knew that these encounters were only simulations, they were highly invested in these experiences and at times were visibly or emotionally affected. This suggests that the simulated encounters designed for this study were realistic patient simulations. An example of a student having difficulty responding directly to a patient’s concern about going blind is shown in Video 2 (available at http://links.lww.com/OPX/A152). Other investigators have previously demonstrated the value of utilizing patient actors to simulate emotionally challenging counseling scenarios, such as the transition of oncology patients to palliative care.38 Our results suggest that this approach could be useful for training eye care providers to better manage difficult clinical encounters in a simulated and ‘‘low-risk’’ environment as well. Limitations of this study include the small sample size and the self-selection of the students who volunteered to participate. Despite the fact this was a pilot study with a small sample size, we did demonstrate significant improvement with both the visual analog scale of improvement and the test of proportions for recommending the doctor to a friend or relative; however, we were underpowered to detect a significant improvement with the ABIM assessment. Volunteers for this study may have chosen to participate because they were already capable and confident communicators or because they felt they were in need of additional training. With a small sample, this potential bias could influence our results. Student group assignments (enrichment and no enrichment) were based on logistical considerations (e.g., scheduling availability); they were not based on student preference or ability. The proportion of students recommended (Figs. 2 and 3) suggests that the students assigned to the no-enrichment group may have been stronger initially (83% recommended at first encounter vs. 61%). These results argue against the possibility that selection bias was an important factor in this study; however, a larger study confirming these observations is warranted. Another point to consider when interpreting these results is that the enrichment group had exposure to two contact lensYrelated cases (one of which was the final evaluation), while the noenrichment group had only one. This additional exposure could have led to better performance in the enrichment group. Nevertheless, these two contact lensYrelated clinical scenarios were substantially different and required very dissimilar patient communication skills. The enrichment case involved a chronic inflammatory condition related to poor adherence to lens wear and care recommendations, while the second contact lensYrelated scenario involved an acute, severe, sight-threatening infection. Although these cases shared a common component of contact lens wear, the nature of the problem and the counseling provided in each case was substantially different. The need to formalize training of patient interviewing and counseling has been recognized in many disciplines. Utilizing patient actors can lessen concerns regarding the number and type of patient encounters39 while providing opportunities to standardize the type of encounters students experience. Utilization of patient-actors also allows for the possibility to video record interactions in order to illustrate points to students, facilitate individualized feedback, and provide external resources for
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
self-evaluation, as was done in the present study. Other successful programs have utilized small group sessions with both peer and patient actor feedback.38,40 Both strategies have been shown to result in significant improvements in clinician skills as measured by masked examiner evaluations by clinical instructors.38 However, the problem of doctor communication skills is less often considered from the patient perspective, and that nuance remains critical to improving performance, as it is not sufficient to have clinician satisfaction if it cannot also be linked to patient needs and satisfaction. This study was not designed for follow-up and evaluation of patient communication skills of the student participants after conclusion of the training and thus there is no evidence that the training effects were enduring. Although five encounters were sufficient to demonstrate a significant improvement, it is unknown whether this short course of instruction will have lasting effects on student performance. Previous studies have demonstrated short training courses can have effects lasting at least 12 to 15 months,19 whereas others have indicated short courses to be ineffective in altering behavior over the long term.41 Future studies should address the long-term impact of this instructional design on student performance. Although we elected to enroll early-stage clinicians in an attempt to have similar baseline patient communication skills between groups, it was apparent that some students had a more natural ability to communicate than others. Given the variability in natural skills, it is possible that the amount of exposure necessary to attain competence could vary among students. Previous studies have demonstrated that communication skills improve significantly in student clinicians during the early stages of training, and that once students reach a satisfactory level of skills, there is no significant difference between upper-level students and expert clinicians with added years of experience.42 This evidence argues that programs to develop patient communication skills in clinicians should be incorporated early in the curriculum when students are most rapidly developing their strategies for patient interactions.
CONCLUSIONS We found a measurable improvement in patient communication skills as rated by masked instructors with as few as five clincal interactions when combined with immediate instructor feedback and video evaluation. This study design may serve as a model for curricular modifications to develop student clinician competency in patient communication by providing a ‘‘low-risk’’ environment in which instructors are comfortable allowing novice clinicians to provide consultation for challenging clinical scenarios. In addition, the use of patient actors may ensure standardization of student exposure to a specific subset of both common and challenging scenarios that are critical for practical competency in patient communication skills.
ACKNOWLEDGMENTS This work was funded by a Faculty Development Initiative Program Grant to Dr. Twa from the University of Houston. Received: June 24, 2013; accepted August 26, 2013.
127
SUPPLEMENTARY DIGITAL CONTENT AND APPENDIX Video 1, showing interactions between a student clinician and patient actor, is available at [http://links.lww.com/OPX/A150]. Video 2, an example of a student having difficulty responding directly to a patient’s concern about going blind, is available at [http://links.lww.com/OPX/A152]. Appendix A, a subset of questions modified from the American Board of Internal Medicine Assessment for self-evaluation, instructor evaluation, and patient evaluation, is available at [http://links.lww.com/OPX/A151].
REFERENCES 1. Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ 1995;152:1423Y33. 2. Symons AB, Swanson A, McGuigan D, Orrange S, Akl EA. A tool for self-assessment of communication skills and professionalism in residents. BMC Med Educ 2009;9:1. 3. Carter WB, Inui TS, Kukull WA, Haigh VH. Outcome-based doctor-patient interaction analysis: II. Identifying effective provider and patient behavior. Med Care 1982;20:550Y66. 4. Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prev Control 1999;3:25Y30. 5. Beck RS, Daughtridge R, Sloane PD. Physician-patient communication in the primary care office: a systematic review. J Am Board Fam Pract 2002;15:25Y38. 6. Boon H, Stewart M. Patient-physician communication assessment instruments: 1986 to 1996 in review. Patient Educ Couns 1998; 35:161Y76. 7. Levinson W, Roter DL, Mullooly JP, Dull VT, Frankel RM. Physician-patient communication. The relationship with malpractice claims among primary care physicians and surgeons. JAMA 1997;277:553Y9. 8. Shibata M, Kishi T, Iwata H. Clinical study of complications in dialyzed diabetics. Tohoku J Exp Med 1983;141(Suppl.):417Y25. 9. Simpson M, Buckman R, Stewart M, Maguire P, Lipkin M, Novack D, Till J. Doctor-patient communication: the Toronto consensus statement. BMJ 1991;303:1385Y7. 10. Adams GL, Pearlman JT. Emotional response and management of visually handicapped patients. Psychiatry Med 1970;1:233Y40. 11. Caplan LM. Pre-education of the potentially blind as a deterrent to suicide. Psychosomatics 1981;22:165, 169. 12. De Leo D, Hickey PA, Meneghel G, Cantor CH. Blindness, fear of sight loss, and suicide. Psychosomatics 1999;40:339Y44. 13. Fitzgerald RG. Reactions to blindness. An exploratory study of adults with recent loss of sight. Arch Gen Psychiatry 1970;22:370Y9. 14. Dowsett SM, Saul JL, Butow PN, Dunn SM, Boyer MJ, Findlow R, Dunsmore J. Communication styles in the cancer consultation: preferences for a patient-centred approach. Psychooncology 2000;9: 147Y56. 15. Kopecky-Wenzel M, Reiner F. [A video based training in communication skills for physicians]. Prax Kinderpsychol Kinderpsychiatr 2010;59:207Y23. 16. Moulene MV, de Lusignan S, Freeman G, van Vlymen J, Sheeler I, Singleton A, Kumarapeli P. Assessing the impact of recording quality target data on the GP consultation using multi-channel video. Stud Health Technol Inform 2007;129:1132Y6.
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
128 Patient Actors Improve Clinician TrainingVAnderson et al. 17. Neumann E, Obliers R, Schiessl C, Stosch C, Albus C. Student Evaluation Scale for Medical Courses with Simulations of the Doctor-Patient Interaction (SES-Sim). GMS Z Med Ausbild 2011; 28:Doc56. 18. Prose NS, Brown H, Murphy G, Nieves A. The morbidity and mortality conference: a unique opportunity for teaching empathic communication. J Grad Med Educ 2010;2:505Y7. 19. Fallowfield L, Jenkins V, Farewell V, Solis-Trapala I. Enduring impact of communication skills training: results of a 12-month follow-up. Br J Cancer 2003;89:1445Y9. 20. Maguire P. Can communication skills be taught? Br J Hosp Med 1990;43:215Y6. 21. Roter DL, Hall JA, Kern DE, Barker LR, Cole KA, Roca RP. Improving physicians’ interviewing skills and reducing patients’ emotional distress. A randomized clinical trial. Arch Intern Med 1995;155:1877Y84. 22. Stewart MA. What is a successful doctor-patient interview? A study of interactions and outcomes. Soc Sci Med 1984;19:167Y75. 23. Archer JC. State of the science in health professional education: effective feedback. Med Educ 2010;44:101Y8. 24. Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, Sargeant J. Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract 2012;17:15Y26. 25. Plakht Y, Shiyovich A, Nusbaum L, Raizer H. The association of positive and negative feedback with clinical performance, selfevaluation and practice contribution of nursing students. Nurse Educ Today 2013;33:1264Y8. 26. Rudland J, Wilkinson T, Wearn A, Nicol P, Tunny T, Owen C, O’Keefe M. A student-centred feedback model for educators. Clin Teach 2013;10:99Y102. 27. Plant JL, Corden M, Mourad M, O’Brien BC, van Schaik SM. Understanding self-assessment as an informed process: residents’ use of external information for self-assessment of performance in simulated resuscitations. Adv Health Sci Educ Theory Pract 2013; 18:181Y92. 28. Hammoud MM, Morgan HK, Edwards ME, Lyon JA, White C. Is video review of patient encounters an effective tool for medical student learning? A review of the literature. Adv Med Educ Pract 2012;3:19Y30. 29. Weaver MJ, Ow CL, Walker DJ, Degenhardt EF. A questionnaire for patients’ evaluations of their physicians’ humanistic behaviors. J Gen Intern Med 1993;8:135Y9. 30. Webster G. Final Report on the Patient Satisfaction Questionnaire Project: Executive Summary. Philadelphia, PA: American Board of Internal Medicine Committee on Evaluation of Clinical Competence; 1989.
31. Stewart M, Brown JB, Hammerton J, Donner A, Gavin A, Holliday RL, Whelan T, Leslie K, Cohen I, Weston W, Freeman T. Improving communication between doctors and breast cancer patients. Ann Fam Med 2007;5:387Y94. 32. Williams S, Weinman J, Dale J. Doctor-patient communication and patient satisfaction: a review. Fam Pract 1998;15:480Y92. 33. Yudkowsky R, Alseidi A, Cintron J. Beyond fulfilling the core competencies: an objective structured clinical examination to assess communication and interpersonal skills in a surgical residency. Curr Surg 2004;61:499Y503. 34. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med 2002;77:S64Y6. 35. Lin ECL, Chen SL, Chao SY, Chen YC. Using standardized patient with immediate feedback and group discussion to teach interpersonal and communication skills to advanced practice nursing students. Nurse Educ Today 2013;33:677Y83. 36. Gow KW. Self-evaluation: how well do surgery residents judge performance on a rotation? Am J Surg 2013;205:557Y62. 37. Lipsett PA, Harris I, Downing S. Resident self-other assessor agreement: influence of assessor, competency, and performance level. Arch Surg 2011;146:901Y6. 38. Goelz T, Wuensch A, Stubenrauch S, Ihorst G, de Figueiredo M, Bertz H, Wirsching M, Fritzsche K. Specific training program improves oncologists’ palliative care communication skills in a randomized controlled trial. J Clin Oncol 2011;29:3402Y7. 39. Elley CR, Clinick T, Wong C, Arroll B, Kennelly J, Doerr H, Moir F, Fishman T, Moyes SA, Kerse N. Effectiveness of simulated clinical teaching in general practice: randomised controlled trial. J Prim Health Care 2012;4:281Y7. 40. Novack DH, Dube C, Goldstein MG. Teaching medical interviewing. A basic course on interviewing and the physician-patient relationship. Arch Intern Med 1992;152:1814Y20. 41. Aspegren K. BEME Guide No. 2: Teaching and learning communication skills in medicineVa review with quality grading of articles. Med Teach 1999;21:563Y70. 42. Wouda JC, van de Wiel HB. The communication competency of medical students, residents and consultants. Patient Educ Couns 2012;86:57Y62.
Heather A. Anderson College of Optometry University of Houston 505 J Davis Armistead Bldg Houston, TX 77204-2020 e-mail: [email protected]
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
ORIGINAL ARTICLE
Training Students with Patient Actors Improves Communication: A Pilot Study Heather A. Anderson*, Jack Young†, Danica Marrelli‡, Rudolph Black‡, Kimberly Lambreghts§, and Michael D. Twa*
ABSTRACT Purpose. Effective patient communication is correlated with better health outcomes and patient satisfaction, but is challenging to train, particularly with difficult clinical scenarios such as loss of sight. In this pilot study, we evaluated the use of simulated patient encounters with actors to train optometric students. Methods. Students were recorded during encounters with actors and assigned to an enrichment group performing five interactions with instructor feedback (n = 6) or a no-enrichment group performing two interactions without feedback (n = 4). Student performance on first and last encounters was scored with (1) subjective rating of performance change using a visual analog scale (anchors: much worse/much better), (2) yes/no response: Would you recommend this doctor to a friend/ relative?, and (3) average score on questions from the American Board of Internal Medicine (ABIM) assessment of doctor communication skills. Three clinical instructors, masked to student group assignments and the order of patient encounters they viewed, provided scores in addition to self-evaluation by students and patient-actors. Results. Using the visual analog scale, students who received enrichment were rated more improved than the noenrichment group by masked examiners (+18 vs. j11% p = 0.04) and self-evaluation (+79 vs. +27% p = 0.009), but not by actors (+31 vs. +43%). The proportion of students recommended significantly increased following enrichment for masked examiners (61% vs. 94%; p G 0.001), but not actors (100 vs. 83%). Average ABIM assessment scores were not significantly different by any rating group: masked instructors, actors, or self-ratings. Conclusions. The findings of this study suggest five simulated patient encounters with feedback result in measurable improvement in student-patient communication skills as rated by masked examiners. (Optom Vis Sci 2014;91:121Y128) Key Words: actor, communication skills, feedback, optometric education, patient simulation, physician-patient communication, standardized patient, student training
E
ffective communication skills are essential in every healthcare discipline and recognition of their importance has resulted in specific provider-patient communication training programs in medicine, nursing, dentistry, and other health professions, yet there are few examples in vision care. There is a sizeable body of published research showing that effective
*OD, PhD, FAAO † MFA ‡ OD, FAAO § RN, OD, FAAO College of Optometry (HAA, DM, RB, KL, MDT) and The School of Theatre & Dance (JY), University of Houston, Houston, Texas. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.optvissci.com).
communication is highly correlated with better health outcomes,1,2 greater patient satisfaction,3 better patient compliance with recommended treatments,4 and lower cost of care.5Y7 Loss of sight is one of the most common fears, second only to fear of cancer and surpassing fear of several other important public health concerns such as diabetes or heart disease.4,8,9 Vision loss is often associated with a loss of mobility, loss of independence, and social isolation, and has been associated with an increased risk of suicide behavior.8,10Y13 Optometrists caring for patients with these concerns should be capable of compassionately conveying news about permanent loss of sight, loss of driving privileges, and the seriousness of acute and chronic ocular diseases. Nevertheless, student clinicians typically receive limited specific training to help them prepare for these challenging patient encounters. Training methods commonly used to improve providerpatient communication include individual mentoring, traditional
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
122 Patient Actors Improve Clinician TrainingVAnderson et al.
lectures on communication skills, role-playing, and demonstrations. Several studies have demonstrated the value of using actors as patients to augment the number and variety of clinical scenarios that student clinicians encounter during their training.14Y18 The use of patient-actors is ideally suited for training student clinicians to navigate emotionally charged and complex interviews where roleplaying can effectively simulate and control the encounter while reducing untoward consequences for students and patients alike. Moreover, there is evidence that these experiences can have a measurable and lasting impact on students’ ability to communicate effectively with their patients.19Y22 Providing student feedback is a critical component in training student clinicians and determining the most effective methods is a growing field of research, particularly among health educators.23Y26 A recent shift in feedback practices has been to utilize a studentcentered approach which incorporates self-assessments, such as the evaluation of video-recorded performances. Medical students report that this strategy is a useful learning tool and is more effective than traditional qualitative instructor feedback.27 A recent review article supports the use of self-evaluation of video recordings, noting that it is only effective when utilized in combination with expert feedback.28 The primary motivation for this pilot study was to develop training scenarios that would permit student clinicians to practice patient communication skills and to develop methods to assess the effects of this training. In this study, we evaluate the effects of active student participation in patient interviewing and counseling when combined with enrichment (instructor feedback, supplemented with viewing of video-recorded encounters).
METHODS This study was approved by the University of Houston Committee for the Protection of Human Subjects and adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from all study participants including written permission for publication of video segments from those who elected to give permission. Three actors were recruited through the University of Houston School of Theatre & Dance to portray patients in five different common clinical scenarios developed by the investigators (HAA, MDT). The five cases portrayed scenarios with significant demands on doctor to patient communication and included (1) recent onset retinal detachment in an individual scheduled to travel on vacation, (2) loss of eligibility for driving privileges in an elderly patient with age-related macular degeneration, (3) a patient with progressive cataracts taking homeopathic herbal supplements as treatment, (4) a patient wearing contact lenses who is non-compliant with lens hygiene and wearing schedule who is beginning to develop ocular complications, and (5) a sight-threatening corneal infection in a contact lensYwearing patient who refuses to wear glasses and is participating in a wedding in 2 days. Actors were provided information about their character (demographics, social behaviors, etc.), their reason for seeking care, additional specific details regarding their visual complaints, and pertinent medical history. They were directed to improvise additional details as necessary and given some (minimal) direction to encourage consistency during the
encounter with the students; however, they cannot be classified as professional standardized patients given that they did not undergo the extensive formalized training customary of that profession. Student clinicians were invited via email to participate in the study during the summer following completion of the second year of their 4-year professional training at University of Houston College of Optometry. Students at this stage in the curriculum have engaged in limited patient encounters (less than 40) and were thus considered novice, early-stage clinicians by study investigators due to their limited experience. The first ten students to respond were enrolled in the study. Students were assigned by the investigators to one of two study groups: an enrichment group that performed all five patient interactions with instructor feedback following each interaction (n = 6), or a comparison group that received no enrichment, performing only the first and last patient interactions without instructor feedback (n = 4). Because students in both groups were actively receiving clinical training and patient encounters outside of their participation in this study, we compared student performance at the beginning and the end of the study to account for improvements in performance that may have been attributable to other learning experiences. To encourage consistency across subject groups, a single actor who was unaware of student group assignments portrayed the first and last patient encounters (scenarios 1 and 5), which were used for statistical comparisons. Two other actors portrayed three additional clinical scenarios for the enrichment group (scenarios 2, 3, and 4). Group assignment was based upon student schedules with regards to actor availability and not upon student choice or investigators’ perceptions of student communication skills. Scheduling considerations were the determining factor for group assignments and the reason for the difference in the number of subjects between groups. Patient interactions occurred in the Family Practice Service at the University Eye Institute. Students were instructed to enter the examination room and perform an initial interview and case history with the patient. When finished, they summarized their findings for the investigators (HAA, MDT). The investigators then provided students with the associated clinical examination findings; students did not perform the physical examinations. Students were permitted to ask questions and discuss these findings and their understanding of the case with the investigators. The student clinicians were then instructed to return to the examination room and counsel the patient on their diagnosis and treatment plans. This sequence is consistent with patient encounters that students experience at the University Eye Institute. Two video cameras were placed in the examination room permitting investigators to record and monitor interactions between student clinicians and the patient actors. The student clinicians were aware of the cameras and consented to the video recording. A sample of one of these encounters is shown in Video 1 (available at http://links.lww.com/OPX/A150) with the student’s face blurred to maintain anonymity. The investigators made notes regarding the student’s performance while monitoring the live video feed during the initial interview and the final consultation of each encounter. Immediately following the encounter,
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
the student clinician, patient actor, and investigators each independently completed a written performance evaluation using instruments described in detail below. Students in the noenrichment group were not permitted to see their performance evaluations, were not given any verbal feedback, and were not allowed to view the recordings until completion of the study. Their performance evaluations were recorded for later analysis. For students in the enrichment group, the written evaluations were used along with verbal feedback in individual faceto-face discussions with the instructors (HAA, MDT) after each encounter. Both strengths and weaknesses of the encounter were discussed. The instructors shared specific notes regarding the individual encounters, as well as encouraged the student to selfevaluate by verbalizing their own perceived areas of strength and weaknesses. Strategies for improvement were offered directly from the instructors, but students were also led to discover their own solutions to weaknesses through facilitative discussions. Throughout the discussions, students viewed segments of the video recordings that were selected by the investigators to illustrate specific teaching points. Students were not given recordings to view on their own time until completion of the study. Feedback sessions lasted 20 minutes on average. Student performance was also evaluated at a later date by three additional, independent clinical instructors who watched the videos and were masked to the student group assignments, the order of the encounters (first/last vs. last/first), and from the ratings of other instructors. Three metrics were used to score student performance: (1) subjective rating of change in performance between first and last encounter using a visual analog scale (anchors: much worse/much better), (2) yes/no response: ‘‘Would you recommend this doctor to a friend or relative?’’, and (3) average cumulative score on a subset of questions based on the American Board of Internal Medicine (ABIM) assessment of doctor communication skills. The ABIM assessment is an instrument created as part of the continuous professional development program for maintenance of board certification for internal medicine physicians.29,30 It is designed to evaluate a variety of physician qualities related to patient communication on a scale from 1 to 5 and has also been used in modified form in previous studies evaluating the training of clinician/ patient interactions.1,22,31Y33 For this study, we selected nine questions which targeted characteristics applicable to patient history and consultation: (1) disclosure of information, (2) warmth, (3) respect, (4) listening skills, (5) genuineness, (6) involving the patient in decision making, (7) encouraging questions, (8) proper patient education, and (9) use of lay language. Questions were modified to use appropriate tense for self-evaluation, patient evaluation, and instructor evaluation (see Appendix A, available at http://links.lww.com/OPX/A151).
123
investigators. Visual analog scores were measured using a millimeter rule and normalized over a range of j1 to +1 where j1 represented maximum possible regression in performance and +1 represented maximum possible improvement in performance. Change in scores were compared between subject groups (enrichment or no enrichment) using Student t test (patient and self-rating: six observations for enrichment and four observations for noenrichment) and two-way ANOVA with repeated measures (masked graders: 18 observations for enrichment and 12 observations for no enrichment). Statistical significance was declared at p G 0.05. Statistical analysis was performed using Stata (SE 11.2 for x86-64; StataCorp LP, College Station, TX). A test of proportions was used to compare responses to whether or not student clinicians were recommended to friends or family. The ABIM survey responses were summarized as a mean response score according to methods described by Lipner and colleagues.34 Scores for each student group were compared using Student t test.
RESULTS Visual Analog Scale Students who received enrichment were rated more improved by the masked clinical instructors between their first and last encounters than students who did not receive enrichment (+18 vs. j11%, two-way repeated measures ANOVA, F = 4.59, df = 1, p = 0.04) (Fig. 1). Students who did not receive enrichment rated themselves improved between their first and last encounters, but the level of self-rated improvement was significantly higher for the students who received enrichment (+27 vs. +79%, ANOVA, F = 11.64, df = 1, p = 0.009). There was no significant difference in the level of improvement rated by the patient actor for the two student groups (+43% no enrichment vs. +31% enrichment, ANOVA, F = 0.08 df = 1, p = 0.78). Within the enrichment group, the level of improvement between first to last encounter reached significance as rated by the students themselves (+79%, one-sample t test, t = 8.77, df = 5, p G 0.001) and by the masked clinical instructors (+18%, one-sample t test, t = 2.20, df = 17, p = 0.04), but not by the patient actor (+31%, one-sample t test, t = 1.32, df = 5, p = 0.24). Given the small sample size, individual student performance was also evaluated as scored by the masked clinical instructors. Change scores for the three masked clinical instructors were averaged for each student and compared between the enrichment and no-enrichment groups. Five of the six students in the enrichment group had a positive change in performance from first to last encounter, whereas three of the four students in the no-enrichment group had a negative change in performance from first to last encounter.
Would You Recommend This Doctor? Analysis Student performance on each of the three metrics was analyzed separately for each rating group: student self-evaluations, patient-actors, and masked clinical instructors. Results are not reported for the evaluations provided by the study investigators (HAA, MDT) because these evaluations were used for student feedback and would therefore likely reflect the bias of the
The proportion of students recommended by the masked clinical instructors significantly increased in the enrichment group between first and last encounter (61 vs. 94%, one-sample test of proportion, z = j5.60, p G 0.001) as opposed to those who did not receive enrichment (83 vs. 92%, one-sample test of proportion, z = j1.15, p = 0.25; Figs. 2 and 3). The proportion of students recommended by the patient-actor on
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
124 Patient Actors Improve Clinician TrainingVAnderson et al.
FIGURE 1. Change in performance obtained from the visual analog scale grouped by enrichment. Combined mean scores are shown for masked clinical instructors, patient-actor, and self-evaluation by the student clinician. Positive values indicate that raters observed improvement between first and last encounters.
first encounter was 100% and was 83% on their last encounter for the enrichment group. This was 75% on first encounter versus 100% on the last encounter in the no-enrichment group.
Communications Assessment Survey This instrument was comprised of nine items (Likert scale), with individual item scores ranging from 1 to 5. Mean cumulative
scores on the ABIM assessment could range from 9 to 45, with higher scores indicating better performance. Mean scores from first and last encounters are shown for the enrichment and noenrichment groups as rated by the three types of evaluators (masked clinical instructors, patient-actor, and self-evaluation) in Table 1. Similar to the visual analog scale, the trend of positive change was greatest for the enrichment group as rated by the masked instructors (mean = 3.4 T 1.75) and the student’s
FIGURE 2. Responses from masked clinical instructors to the question: ‘‘Would you recommend this doctor to a friend or relative?’’ Responses are shown for the first and last encounters for students receiving no enrichment and enrichment. Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
125
FIGURE 3. Responses from patient-actors to the question: ‘‘Would you recommend this doctor to a friend or relative?’’ Responses are shown for the first and last encounters for students receiving no enrichment and enrichment.
self-evaluation (mean = 4.33 T 2.04); however, the difference in mean cumulative performance on the ABIM Communications Assessment did not reach statistical significance from first to last encounter for either the enrichment or no enrichment groups for any of the three types of evaluators (one-sample t test, range of p = 0.09 to 0.58). Individual student scores derived from a mean of the three masked clinical instructors were also compared between enrichment and no-enrichment groups. Five of six students in the enrichment group had an increase in ABIM score from first to last encounter, whereas three of four students in the no-enrichment group had a decrease or no change in ABIM score from first to last encounter. These findings were in agreement with the same individual trends in performance observed with the visual analog scale.
DISCUSSION This pilot study found a significant improvement in patient communication skills of student clinicians who participated in five clinical scenarios with immediate instructor feedback and the opportunity to view their video-taped sessions, versus students who participated in two encounters with no feedback. By design, the enrichment group received multiple types of instructional enrichment (feedback, multiple encounters, and viewing portions of the video recordings) that were not offered to the noenrichment group. This pilot study was not designed to show specifically which of these three elements of instructional enrichment resulted in the observed improvement in performance; it may have been a single element or some combination. Consistent with our results, previous studies have demonstrated that instructor feedback paired with self-evaluation of video recordings results in increased learning relative to isolated self-evaluation
of videos.28 This supports the hypothesis that feedback during enrichment is an essential component and would be consistent with the improvement in the enrichment group observed in this study. However, a recent study of patient communication in nursing students found no increase in performance between first and second encounters for those receiving immediate feedback and group discussion with video recordings.35 This finding may suggest that individual feedback is more effective than group encounters, or that the number of encounters is important as well. Because student performance was only evaluated by the masked clinical instructors for the first and last cases, it is not possible to determine at what point the students receiving enrichment began to improve. Given the inability to identify the source and timing of the improvement, further study is
TABLE 1.
Mean scores for the communication assessment from first and last encounters compared by group as rated by the three types of evaluators. No differences reached statistical significance Group: rater Enrichment Instructor Patient Self No enrichment Instructor Patient Self
First encounter (mean T SD)
Last encounter (mean T SD)
3.5 T 0.79 4.7 T 0.15 4.1 T 0.43
3.9 T 0.34 4.7 T 0.11 4.5 T 0.51
4.1 T 0.61 4.6 T 0.28 4.3 T 0.53
4.0 T 0.41 4.8 T 0.91 4.5 T 0.45
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
126 Patient Actors Improve Clinician TrainingVAnderson et al.
warranted to better understand the impact of each aspect of instructional enrichment provided in this study. Specifically, there is a need for evidence regarding the most influential methods of instructional enrichment (e.g., immediate instructor feedback or self-evaluation of video-taped encounters), the number of simulated clinical encounters necessary for measurable improvement, and the most influential types of patient encounters (e.g., simple or complex). The significant findings in this study were based upon an average of scores from three masked instructors. Two of the masked clinical instructors had previously taught these student clinicians in didactic courses in the first and second years of their professional training and therefore had some prior knowledge of the student’s academic performance. The third masked clinical instructor served as an attending clinical instructor to four of the study participants. However, these students were evenly distributed between the enrichment and no-enrichment groups. An individual analysis of the masked instructors revealed that evaluators varied in their overall ratings, but improvement from first to last encounter was consistently observed among all three instructors. More structured grading criteria for the instructors would have helped to reduce the variability in overall ratings and is planned for future evaluations. We did not find significant differences in student clinician communication skills between enrichment and no-enrichment groups as rated by the patient-actor. While real patient feedback is highly valued in the clinical setting, we found that the feedback of the patient actor was not a robust measure of student performance and that actors were reticent to score the students poorly. This limited the range of actor-reported scores to a narrow and relatively high range with each of the instruments. When asked, actors stated that they were concerned that scoring the students poorly would negatively impact the students’ standing in some way. This limitation could be addressed by using professional standardized patients trained to provide reliable feedback. Conversely, we found that the students tended to rate themselves highly when performing self-evaluations. This finding is consistent with other studies that report medical residents, particularly those with lower performance, overestimate their clinical communication skills when compared to ratings by external reviewers.36,37 The fact that students in this study knew they were being assessed on their communications skills and they were aware of their group assignments (enrichment vs. no enrichment) may have influenced and confounded how students rated their own skills. The students’ perceived improvement in communication skills may simply have been influenced by their knowledge that their communication skills were under study, rather than the direct effect of enrichment. Clearly, the students felt more confident in their ability following these enrichment experiences (Fig. 1), but their estimation of how much they improved was far greater than the improvement reported by masked graders. Nevertheless, significant improvement in performance was also observed by the masked instructors, further supporting our finding that enrichment in the form of additional mock-patient encounters and instructor feedback was associated with better patient communication skills. An observation that we did not anticipate was the emotional impact that these mock-patient encounters had on student
clinicians. Given that these were novice clinicians, they had limited or no previous experience with difficult patient counseling scenarios involving life-changing events, such as the loss of driving privileges or the loss of sight. Although students knew that these encounters were only simulations, they were highly invested in these experiences and at times were visibly or emotionally affected. This suggests that the simulated encounters designed for this study were realistic patient simulations. An example of a student having difficulty responding directly to a patient’s concern about going blind is shown in Video 2 (available at http://links.lww.com/OPX/A152). Other investigators have previously demonstrated the value of utilizing patient actors to simulate emotionally challenging counseling scenarios, such as the transition of oncology patients to palliative care.38 Our results suggest that this approach could be useful for training eye care providers to better manage difficult clinical encounters in a simulated and ‘‘low-risk’’ environment as well. Limitations of this study include the small sample size and the self-selection of the students who volunteered to participate. Despite the fact this was a pilot study with a small sample size, we did demonstrate significant improvement with both the visual analog scale of improvement and the test of proportions for recommending the doctor to a friend or relative; however, we were underpowered to detect a significant improvement with the ABIM assessment. Volunteers for this study may have chosen to participate because they were already capable and confident communicators or because they felt they were in need of additional training. With a small sample, this potential bias could influence our results. Student group assignments (enrichment and no enrichment) were based on logistical considerations (e.g., scheduling availability); they were not based on student preference or ability. The proportion of students recommended (Figs. 2 and 3) suggests that the students assigned to the no-enrichment group may have been stronger initially (83% recommended at first encounter vs. 61%). These results argue against the possibility that selection bias was an important factor in this study; however, a larger study confirming these observations is warranted. Another point to consider when interpreting these results is that the enrichment group had exposure to two contact lensYrelated cases (one of which was the final evaluation), while the noenrichment group had only one. This additional exposure could have led to better performance in the enrichment group. Nevertheless, these two contact lensYrelated clinical scenarios were substantially different and required very dissimilar patient communication skills. The enrichment case involved a chronic inflammatory condition related to poor adherence to lens wear and care recommendations, while the second contact lensYrelated scenario involved an acute, severe, sight-threatening infection. Although these cases shared a common component of contact lens wear, the nature of the problem and the counseling provided in each case was substantially different. The need to formalize training of patient interviewing and counseling has been recognized in many disciplines. Utilizing patient actors can lessen concerns regarding the number and type of patient encounters39 while providing opportunities to standardize the type of encounters students experience. Utilization of patient-actors also allows for the possibility to video record interactions in order to illustrate points to students, facilitate individualized feedback, and provide external resources for
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Patient Actors Improve Clinician TrainingVAnderson et al.
self-evaluation, as was done in the present study. Other successful programs have utilized small group sessions with both peer and patient actor feedback.38,40 Both strategies have been shown to result in significant improvements in clinician skills as measured by masked examiner evaluations by clinical instructors.38 However, the problem of doctor communication skills is less often considered from the patient perspective, and that nuance remains critical to improving performance, as it is not sufficient to have clinician satisfaction if it cannot also be linked to patient needs and satisfaction. This study was not designed for follow-up and evaluation of patient communication skills of the student participants after conclusion of the training and thus there is no evidence that the training effects were enduring. Although five encounters were sufficient to demonstrate a significant improvement, it is unknown whether this short course of instruction will have lasting effects on student performance. Previous studies have demonstrated short training courses can have effects lasting at least 12 to 15 months,19 whereas others have indicated short courses to be ineffective in altering behavior over the long term.41 Future studies should address the long-term impact of this instructional design on student performance. Although we elected to enroll early-stage clinicians in an attempt to have similar baseline patient communication skills between groups, it was apparent that some students had a more natural ability to communicate than others. Given the variability in natural skills, it is possible that the amount of exposure necessary to attain competence could vary among students. Previous studies have demonstrated that communication skills improve significantly in student clinicians during the early stages of training, and that once students reach a satisfactory level of skills, there is no significant difference between upper-level students and expert clinicians with added years of experience.42 This evidence argues that programs to develop patient communication skills in clinicians should be incorporated early in the curriculum when students are most rapidly developing their strategies for patient interactions.
CONCLUSIONS We found a measurable improvement in patient communication skills as rated by masked instructors with as few as five clincal interactions when combined with immediate instructor feedback and video evaluation. This study design may serve as a model for curricular modifications to develop student clinician competency in patient communication by providing a ‘‘low-risk’’ environment in which instructors are comfortable allowing novice clinicians to provide consultation for challenging clinical scenarios. In addition, the use of patient actors may ensure standardization of student exposure to a specific subset of both common and challenging scenarios that are critical for practical competency in patient communication skills.
ACKNOWLEDGMENTS This work was funded by a Faculty Development Initiative Program Grant to Dr. Twa from the University of Houston. Received: June 24, 2013; accepted August 26, 2013.
127
SUPPLEMENTARY DIGITAL CONTENT AND APPENDIX Video 1, showing interactions between a student clinician and patient actor, is available at [http://links.lww.com/OPX/A150]. Video 2, an example of a student having difficulty responding directly to a patient’s concern about going blind, is available at [http://links.lww.com/OPX/A152]. Appendix A, a subset of questions modified from the American Board of Internal Medicine Assessment for self-evaluation, instructor evaluation, and patient evaluation, is available at [http://links.lww.com/OPX/A151].
REFERENCES 1. Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ 1995;152:1423Y33. 2. Symons AB, Swanson A, McGuigan D, Orrange S, Akl EA. A tool for self-assessment of communication skills and professionalism in residents. BMC Med Educ 2009;9:1. 3. Carter WB, Inui TS, Kukull WA, Haigh VH. Outcome-based doctor-patient interaction analysis: II. Identifying effective provider and patient behavior. Med Care 1982;20:550Y66. 4. Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prev Control 1999;3:25Y30. 5. Beck RS, Daughtridge R, Sloane PD. Physician-patient communication in the primary care office: a systematic review. J Am Board Fam Pract 2002;15:25Y38. 6. Boon H, Stewart M. Patient-physician communication assessment instruments: 1986 to 1996 in review. Patient Educ Couns 1998; 35:161Y76. 7. Levinson W, Roter DL, Mullooly JP, Dull VT, Frankel RM. Physician-patient communication. The relationship with malpractice claims among primary care physicians and surgeons. JAMA 1997;277:553Y9. 8. Shibata M, Kishi T, Iwata H. Clinical study of complications in dialyzed diabetics. Tohoku J Exp Med 1983;141(Suppl.):417Y25. 9. Simpson M, Buckman R, Stewart M, Maguire P, Lipkin M, Novack D, Till J. Doctor-patient communication: the Toronto consensus statement. BMJ 1991;303:1385Y7. 10. Adams GL, Pearlman JT. Emotional response and management of visually handicapped patients. Psychiatry Med 1970;1:233Y40. 11. Caplan LM. Pre-education of the potentially blind as a deterrent to suicide. Psychosomatics 1981;22:165, 169. 12. De Leo D, Hickey PA, Meneghel G, Cantor CH. Blindness, fear of sight loss, and suicide. Psychosomatics 1999;40:339Y44. 13. Fitzgerald RG. Reactions to blindness. An exploratory study of adults with recent loss of sight. Arch Gen Psychiatry 1970;22:370Y9. 14. Dowsett SM, Saul JL, Butow PN, Dunn SM, Boyer MJ, Findlow R, Dunsmore J. Communication styles in the cancer consultation: preferences for a patient-centred approach. Psychooncology 2000;9: 147Y56. 15. Kopecky-Wenzel M, Reiner F. [A video based training in communication skills for physicians]. Prax Kinderpsychol Kinderpsychiatr 2010;59:207Y23. 16. Moulene MV, de Lusignan S, Freeman G, van Vlymen J, Sheeler I, Singleton A, Kumarapeli P. Assessing the impact of recording quality target data on the GP consultation using multi-channel video. Stud Health Technol Inform 2007;129:1132Y6.
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
128 Patient Actors Improve Clinician TrainingVAnderson et al. 17. Neumann E, Obliers R, Schiessl C, Stosch C, Albus C. Student Evaluation Scale for Medical Courses with Simulations of the Doctor-Patient Interaction (SES-Sim). GMS Z Med Ausbild 2011; 28:Doc56. 18. Prose NS, Brown H, Murphy G, Nieves A. The morbidity and mortality conference: a unique opportunity for teaching empathic communication. J Grad Med Educ 2010;2:505Y7. 19. Fallowfield L, Jenkins V, Farewell V, Solis-Trapala I. Enduring impact of communication skills training: results of a 12-month follow-up. Br J Cancer 2003;89:1445Y9. 20. Maguire P. Can communication skills be taught? Br J Hosp Med 1990;43:215Y6. 21. Roter DL, Hall JA, Kern DE, Barker LR, Cole KA, Roca RP. Improving physicians’ interviewing skills and reducing patients’ emotional distress. A randomized clinical trial. Arch Intern Med 1995;155:1877Y84. 22. Stewart MA. What is a successful doctor-patient interview? A study of interactions and outcomes. Soc Sci Med 1984;19:167Y75. 23. Archer JC. State of the science in health professional education: effective feedback. Med Educ 2010;44:101Y8. 24. Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, Sargeant J. Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract 2012;17:15Y26. 25. Plakht Y, Shiyovich A, Nusbaum L, Raizer H. The association of positive and negative feedback with clinical performance, selfevaluation and practice contribution of nursing students. Nurse Educ Today 2013;33:1264Y8. 26. Rudland J, Wilkinson T, Wearn A, Nicol P, Tunny T, Owen C, O’Keefe M. A student-centred feedback model for educators. Clin Teach 2013;10:99Y102. 27. Plant JL, Corden M, Mourad M, O’Brien BC, van Schaik SM. Understanding self-assessment as an informed process: residents’ use of external information for self-assessment of performance in simulated resuscitations. Adv Health Sci Educ Theory Pract 2013; 18:181Y92. 28. Hammoud MM, Morgan HK, Edwards ME, Lyon JA, White C. Is video review of patient encounters an effective tool for medical student learning? A review of the literature. Adv Med Educ Pract 2012;3:19Y30. 29. Weaver MJ, Ow CL, Walker DJ, Degenhardt EF. A questionnaire for patients’ evaluations of their physicians’ humanistic behaviors. J Gen Intern Med 1993;8:135Y9. 30. Webster G. Final Report on the Patient Satisfaction Questionnaire Project: Executive Summary. Philadelphia, PA: American Board of Internal Medicine Committee on Evaluation of Clinical Competence; 1989.
31. Stewart M, Brown JB, Hammerton J, Donner A, Gavin A, Holliday RL, Whelan T, Leslie K, Cohen I, Weston W, Freeman T. Improving communication between doctors and breast cancer patients. Ann Fam Med 2007;5:387Y94. 32. Williams S, Weinman J, Dale J. Doctor-patient communication and patient satisfaction: a review. Fam Pract 1998;15:480Y92. 33. Yudkowsky R, Alseidi A, Cintron J. Beyond fulfilling the core competencies: an objective structured clinical examination to assess communication and interpersonal skills in a surgical residency. Curr Surg 2004;61:499Y503. 34. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med 2002;77:S64Y6. 35. Lin ECL, Chen SL, Chao SY, Chen YC. Using standardized patient with immediate feedback and group discussion to teach interpersonal and communication skills to advanced practice nursing students. Nurse Educ Today 2013;33:677Y83. 36. Gow KW. Self-evaluation: how well do surgery residents judge performance on a rotation? Am J Surg 2013;205:557Y62. 37. Lipsett PA, Harris I, Downing S. Resident self-other assessor agreement: influence of assessor, competency, and performance level. Arch Surg 2011;146:901Y6. 38. Goelz T, Wuensch A, Stubenrauch S, Ihorst G, de Figueiredo M, Bertz H, Wirsching M, Fritzsche K. Specific training program improves oncologists’ palliative care communication skills in a randomized controlled trial. J Clin Oncol 2011;29:3402Y7. 39. Elley CR, Clinick T, Wong C, Arroll B, Kennelly J, Doerr H, Moir F, Fishman T, Moyes SA, Kerse N. Effectiveness of simulated clinical teaching in general practice: randomised controlled trial. J Prim Health Care 2012;4:281Y7. 40. Novack DH, Dube C, Goldstein MG. Teaching medical interviewing. A basic course on interviewing and the physician-patient relationship. Arch Intern Med 1992;152:1814Y20. 41. Aspegren K. BEME Guide No. 2: Teaching and learning communication skills in medicineVa review with quality grading of articles. Med Teach 1999;21:563Y70. 42. Wouda JC, van de Wiel HB. The communication competency of medical students, residents and consultants. Patient Educ Couns 2012;86:57Y62.
Heather A. Anderson College of Optometry University of Houston 505 J Davis Armistead Bldg Houston, TX 77204-2020 e-mail: [email protected]
Optometry and Vision Science, Vol. 91, No. 1, January 2014
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
