Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, 2014; 15: 47–54
ORIGINAL ARTICLE
Breaking bad news in amyotrophic lateral sclerosis: The need for medical education
KERRI L. SCHELLENBERG1, SUSIE J. SCHOFIELD2, SHOUFAN FANG3 & WENDY S. W. JOHNSTON1 1University
of Alberta, 2University of Dundee, and 3Alberta Health Services, Cardiovascular Health and Stroke Strategic Clinical Network, Edmonton, Canada
Abstract The manner in which physicians deliver difficult diagnoses is an area of discontent for patients with amyotrophic lateral sclerosis (ALS). The American Academy of Neurology’s Practice Parameter for care of the ALS Patient recommended teaching and evaluating strategies for disclosing the diagnosis (10). Our objective was to examine residents’ ability in and perceptions of communicating the diagnosis of ALS. Twenty-two resident physicians were videotaped and rated by two ALS neurologists as they delivered an ALS diagnosis to a standardized patient (SP) during an objective structured clinical examination (OSCE). Residents self-rated immediately after the OSCE, again after viewing their videotape, and completed a survey regarding the OSCE and delivering difficult diagnoses. OSCE performance was suboptimal, particularly for communication skills and empathy. The two examiners’ scores correlated except for the empathy subscore. Residents’ self-assessments did not align with the examiners’ scores either before or after watching their videotape. The survey uncovered residents’ apprehension and dissatisfaction with their training in diagnosis delivery. The results highlight a need for resident education in delivering an ALS diagnosis. The lack of correlation between residents’ and examiners’ scoring requires further study. Evaluation of empathy is particularly challenging. Residents agreed that OSCE participation was worthwhile. Key words: Methods of education, amyotrophic lateral sclerosis, palliative care, professional conduct and ethics
Introduction The manner in which physicians break bad news is an area of discontent for patients with ALS (1,2). The future psychological well-being of patients is profoundly affected by this interaction (3). Lack of physician training is one barrier to breaking bad news effectively (4). Fortunately, it has been shown that communication skills may be learned through experiential courses (5). Although courses have been developed for teaching neurology trainees communication skills, including the skill of breaking bad news (6–9), none has been reported in the context of ALS. Since general neurologists are often the first to deliver the diagnosis of ALS, it is important that all residents acquire the necessary skills. The main objective of this study was to explore the need for medical education with respect to residents’ ability to communicate the diagnosis of ALS.
The first step in determining appropriate education is to assess the existing skill level of neurology residents in delivering an ALS diagnosis. Secondary areas of interest pertain to this project’s use of self-assessment: do ratings provided by the examiners correlate to residents’ self-ratings; and how do residents perceive the exercise of self-assessment?
Material and methods The American Academy of Neurology (AAN) Practice Parameter for care of the ALS patient recommended teaching and evaluating strategies for disclosing the diagnosis (10). A Medline search combining results from the key words ‘neurology, residency and internship, education, ALS’ yielded 257 articles (limit English language); none per-
Correspondence: K. L. Schellenberg, 8440-112 St. NW 2E3 WMC, Edmonton, Alberta, Canada T6G 2B7. Tel: 780 407 6528. Fax: 780 407 1325. E-mail: [email protected] (Received 20 May 2013 ; accepted 4 September 2013) ISSN 2167-8421 print/ISSN 2167-9223 online © 2014 Informa Healthcare DOI: 10.3109/21678421.2013.843711
48
K. L. Schellenberg et al.
tained to resident education in breaking bad news in the context of ALS. Setting and participants Multiple objective structured clinical examination (OSCE) stations are organized annually at the University of Alberta for junior (year 1–3) and senior (year 4–5) neurology residents from several five-year residency programs to evaluate knowledge and skills. One examination station was created by the study coordinators to evaluate breaking bad news in ALS. After receiving ethics approval, all residents (n 28) were invited to participate, and written consent was obtained from participating residents prior to the OSCE.
of Neurological Societies (EFNS) (10,11) and on the six-step protocol for delivering bad news ‘SPIKES’ (Setting, Perception, Invitation, Knowledge, Empathy, Strategy and Summary) (12). Although developed for oncologists, ‘SPIKES’ has been used successfully within the ALS community (3). The components of ‘SPIKES’ have also been used for patients with ALS and their caregivers’ evaluation of physician performance in delivering an ALS diagnosis (1). The checklist had three subsections: action (22 points), empathy (6 points), and general communication (10 points) for a total of 38 points (Table I). A higher score corresponded to better performance. There were no established norms for this checklist since it had not been previously used in a study.
Design Neurology residents’ performances were videotaped during an OSCE as they disclosed the diagnosis of ALS to a standardized patient (SP). Two neurologists with specialized training and a current practice in the care of patients with ALS independently observed the videotapes and rated the residents who consented to participate according to a checklist developed for this study (see below). Residents assessed themselves using the same checklist as the examiners immediately after the OSCE and again after watching their own videotape. Residents also completed a survey pertaining to the OSCE, medical education, and breaking bad news. Formation of the checklist A rating checklist consisting of specific behavioral criteria was developed based upon the existing guidelines from the AAN and European Federation
OSCE station Instructions to the candidates and SP were formulated based on a scenario residents may encounter – imparting the ALS diagnosis to an inpatient at the request of the patient. The OSCE station was supervised and instructions given by a faculty member (not a study evaluator) who observed the encounter in the room. To standardize the encounter, the same SP with a performing arts background was used in all OSCEs. Instructions for the SP, including scripted questions and responses, were submitted to the SP and SP coordinator. The SP received 40 minutes of standard training with the SP coordinator, and met with a study investigator prior to the OSCEs to receive further instruction. The first two resident performances were used as practice runs for the SP, and were not included in the study data. One study investigator provided feedback to the SP after each practice OSCE.
Table I. Checklist subsections. Action Preparing for conversation: · attention to privacy · time management · inclusion of the patient’s support system Interviewing the patient: · appropriate greeting · assessing the patient’s existing knowledge and extent to which the patient wishes to be informed · giving a warning statement · delivering the diagnosis without equivocation · imparting ALS specific material including the known pathophysiology, prognosis, treatment, research opportunities, and support organizations · providing hope · supporting wishes for a second opinion · summarizing the discussion; confi rming understanding; and arranging follow-up
Empathy Empathetic diagnosis delivery and response: · delivering the diagnosis in a warm and sensitive manner · acknowledging and exploring the patient’s response · conveying the reaction is understandable
Communication Verbal communication: · speaking without technical jargon, excessive bluntness, or euphemisms · giving information in small chunks · checking for understanding and encouraging questions Non-verbal communication: · demonstrating attention to the patient and maintaining eye contact
Breaking bad news in ALS: the need for medical education The OSCE was selected as an assessment tool since it evaluates both cognitive and performance based aptitudes, including communication techniques. Delivering an ALS diagnosis involves ALS specific knowledge as well as empathy and general communication strategies. Utility of the OSCE to measure communication skills is well established (13,14). Statistical analysis Statistical analyses were conducted with statistical analysis software SAS (version 9.2, developed by SAS Institute Inc.). The 95% significance level (i.e. α 0.05) was taken to reject null hypotheses. When necessary, a 90% significance level (i.e. α 0.10) was used in looking for trend. Null hypotheses for specific analyses were given, and all tests were twotailed. Spearman’s rank correlation test was applied in testing the correlation of the two assessors’ scoring. The null hypothesis was Spearman’s RHO ρ 0 and the alternative hypothesis was ρ 0. Generalized Estimating Equation (GEE) was applied in analysis of the difference between residents’ self-scores and assessors’ scoring, the effect of viewing the video on self-scores, and the correlation between the residents’ demographic factors and their scores. GEE analysis of the demographic factors included three features: gender (male, female), year of residency (junior year 1–3, senior year 4–5) and site (residents from host site, residents from other centres). All GEE models had Poisson distribution as an SAS model option and exchangeable correlation was induced when residents’ ID numbers were used to identify repeated (pre-/post-DVD watching) measurement. After a GEE model was fitted, a Z-test was conducted to test the estimated coefficients, which represented the effect of evaluators (residents or examiners), viewing video, and the selected demographic features. The null hypotheses of the Z-tests were the estimated coefficients being equal to zero, and the alternative hypotheses were that they were not equal to zero.
49
Results Of the 28 residents involved in the OSCE, 25 consented to participate, three did not consent, the first two consenting residents were used as practice stations for the SP, one consented but did not return the required forms; thus 22 participated in the study. There were eight (36%) male and 14 (64%) female participants in the following years of residency: Year 1 3 (14%), Year 2 1 (5%), Year 3 6 (27%), Year 4 5 (23%), Year 5 7 (32%). OSCE results: examiner checklist scores, subscores and resident self-scoring There was positive correlation for the two examiners’ overall scores as well as the action and communication subscores, but no significant correlation for the empathy subscore (Spearman’s correlation coefficient) (Table II). The checklist items ‘assuring SP regarding opportunities for research involvement’ and ‘when directly asked by SP, stating there is no cure’ were performed adequately by all residents. Items with the poorest performance included ‘identifying the need for sufficient time’ and ‘summarizing the discussion, asking if there are further questions, and confirming understanding’. In GEE models for comparison of the scores from residents and examiners, the examiners’ scores or pre-video self-score were treated as the basis of comparison. Therefore, the coefficients of GEE models reflect the difference of the residents’ scores – the positive/negative coefficients represented that residents’ scores were higher/lower than that of examiners or pre-video. Comparing scores from examiners and residents prior to viewing video, GEE models presented positive coefficients and small p-values ( 0.05) for total score and all subscores (Table II), which reflected that the residents significantly overevaluated their performance. All coefficients from GEE models to represent video effect were negative, which meant that residents lowered their scores to more closely resemble those of the examiners. However,
Table II. Checklist results. Total Examiner 1 (mean scores, standard deviation) Examiner 2 (mean scores, standard deviation) Correlation between the examiners’ scores (Spearman’s RHO, p -value) Both examiners (mean, standard deviation) Resident scores compared to examiner scores pre-video (coefficient, standard error, Z-value, p -value from GEE) Resident self-score change after viewing video (coefficient, standard error, Z-value, p -value from GEE) *Indicates 95% significance level was reached.
Action
Communication
Empathy
24.6/38, 3.634
15.5/22, 2.041
5.9/10, 1.540
3.2/6, 1.152
25.8/38, 4.393
16.2/22, 2.423
6.1/10, 1.670
3.5/6, 1.185
0.56, 0.0065*
0.80, 0.0001*
0.44, 0.0415*
0.23, 0.2955
25.2, 4.032
15.8, 2.244
6.0, 1.592
3.4, 1.163
0.145, 0.045, 3.23, 0.0012* 0.027, 0.034, 0.78, 0.4331
0.085, 0.041, 2.08, 0.0374* 0.005, 0.037, 0.14, 0.8857
0.219, 0.064, 3.45, 0.0006* 0.025, 0.059, 0.41, 0.6794
0.291, 0.083, 3.53, 0.0004* 0.141, 0.057, 2.48, 0.0131*
50
K. L. Schellenberg et al.
only the empathy subscore had significant reduction; the remaining p-values were not small enough to reject the null hypotheses (Table II). Relationship between demographic information and resident and examiner ratings Demographic information was compared to both the residents’ and examiners’ ratings using a generalized estimating equation. With respect to the residents’ self-score in this small sample size, there was a trend approaching significance in that female residents scored themselves lower than males on the communication subscore with a 90% significance level (Table III). Different and sometimes opposing relationships were uncovered when the examiners’ scoring of residents and demographic information were evaluated. Female residents achieved higher scores for the grand total and action and empathy subscores (95% significance level) with a trend approaching significance for the communication subscore (90% significance level). Seniors earned greater scores than juniors for grand total and empathy subscores (95% significance level) (Table IV). Survey results: residents’ perception of the OSCE and the skill of breaking bad news All participating residents agreed the OSCE was worthwhile. Thematic analysis of the free-text responses revealed two aspects residents found useful: the exercise of self-assessment, and the practice/ exam preparation value. Virtually all (21/22) residents felt using an SP was effective (95%). However, nine (41%) mentioned unhelpful aspects of the OSCE station: five had no prior experience with certain aspects of the exam (23%); three found the setting artificial (14%); and one wished for immediate feedback (5%). Eighteen (82%) felt watching their videotape was useful and 15 (68%) believed they would be more
effective in breaking bad news after having watched their video. Thematic analysis of free-text comments revealed that residents felt they could improve on various aspects of communication and empathy. Several trainees commented on areas for self-improvement: “I need to be more compassionate and less clinical in the interaction. I need to take some time to acknowledge the human aspect of the situation; the fact that I have given some devastating news. I want to be less verbose and more direct when I do attempt to relay information.” “I think I learned that I was a bit too reserved in my approach to the patient (partially out of my own nervousness), but that it may have come across as cold and unfeeling. I think I could make more effort to reach out to the patient, seek their emotional responses and answer their questions.” “The non-verbal communication seemed inadequate. Eye contact was not sufficient and could have been more empathetic. I think it looked a little bit like it was rehearsed instead of spontaneous.” “I was able to pick up on the parts of the disclosure that made me uncomfortable because my body language clearly reflected it.” “I will pay more attention in the future to use less jargon and be clear in conversation.” All residents felt breaking bad news effectively was important, but 10 (45%) were unsure about their own effectiveness. The majority of residents (17, 77%) reported prior experience; however, 15 found it stressful (68%). They identified several issues that made breaking bad news particularly difficult: “I feel partially responsible for the patient’s diagnosis when breaking bad news, even though I know it is not my fault that they have a terrible disease. I dislike the feeling of powerlessness, and my own ignorance to help them. I find it difficult to deal with their strong emotions.” “Use of appropriate words/language, as English is not my first language. Cultural differences. Need to
Table III. GEE parameter estimates based on residents’ self-score. Model dependent Total
Action
Empathy
Communication
Parameter
Estimate
SE
Z
p -value
Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female
3.342 0.031 0.079 0.019 2.798 0.024 0.055 0.013 1.412 0.077 0.082 0.065 2.049 0.126 0.145 0.150
0.080 0.066 0.061 0.061 0.073 0.057 0.050 0.051 0.094 0.124 0.126 0.104 0.106 0.087 0.078 0.087
41.56 0.46 1.30 0.30 38.18 0.42 1.09 0.25 14.98 0.62 0.65 0.63 19.44 1.44 1.85 1.72
.0001 0.6451 0.1936 0.7607 .0001 0.6723 0.2746 0.8041 .0001 0.5359 0.5155 0.5313 .0001 0.1501 0.0645 0.0849
Breaking bad news in ALS: the need for medical education
51
Table IV. GEE parameter estimates based on examiners’ scores. Model dependent Total
Action
Empathy
Communication
Parameter
Estimate
SE
Z
p -value
Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1
3.125 0.083 0.035 0.167 0.049 2.671 0.082 0.018 0.155 0.046 0.979 0.171 0.130 0.279 0.081 1.733 0.033 0.027 0.133 0.038
0.046 0.042 0.044 0.042 0.030 0.045 0.049 0.048 0.043 0.020 0.098 0.073 0.085 0.080 0.086 0.065 0.069 0.084 0.073 0.061
68.01 1.98 0.78 3.97 1.62 59.40 1.67 0.38 3.59 2.29 9.98 2.34 1.53 3.49 0.94 26.88 0.47 0.32 1.82 0.62
.0001 0.0481* 0.4371 .0001* 0.1057 .0001 0.0946 0.7006 0.0003* 0.0222* .0001 0.0195* 0.1255 0.0005* 0.3454 .0001 0.6387 0.7473 0.0692 0.5371
*Covariates which reached 95% significance level.
know enough about the disease especially prognosis, treatment options, ongoing research to answer patient’s questions.” “Not being too verbose and rambling. Being direct while remaining empathetic.” “The silence and seeing people hurt. Also, wondering if I’m forgetting any important information.” “It’s emotionally draining.” Only three (14%) residents had prior experience in breaking bad news to patients with ALS. All three observed their staff members relay the diagnosis, and one resident participated in giving the diagnosis. The residents’ impressions of their staff members’ execution of the task varied widely: “Positive experience. Was able to observe staff, broke news slowly in waves throughout day as to not overwhelm patient. Discussed with patient and family once diagnosis fully given.” “Another time was in the clinic, and the staff did a horrible job in about three minutes and left the guy sitting in the exam room with a box of tissues,
crying – I still remember vividly how bad I felt leaving the patient like that (but I couldn’t stay behind and talk with the patient when the staff guy was walking away).” Only three (14%) residents were aware of any professional guidelines for breaking bad news to ALS patients. The minority of residents (9%) believed they received adequate instruction on breaking bad news, and the majority (17%) believed more emphasis should be placed on this during training. The residents identified a range of potential topics for further training. (Table V). Discussion Need for resident education in delivering the diagnosis of ALS This study indicates a need for resident education in breaking bad news to patients with ALS, evidenced both by expert examiner scores and residents’ opinions. The examiners’ rating of residents demonstrated
Table V. Suggestions for further training identified in resident survey. Didactic · · · ·
organization of approach further training with SPIKES how to achieve an appropriate setting access to workshops, lectures, demonstrations · sensitivity to cultural values
Practice · practice sessions, including sessions in ward rotations · opportunity to watch staff neurologists impart bad news · opportunity to be observed breaking bad news, given feedback · videotape practice, watch own performance
Communication skills · · · ·
cultivate empathic responses useful phrases what NOT to say better communication skills including non-verbal communication
How to . . . · · · ·
emphasize hope be direct handle patients emotional responses break bad news in least traumatic way
52
K. L. Schellenberg et al.
suboptimal overall scores, with particularly poor performance within the empathy and communication subscores. The residents identified the same two areas of weakness in the free-text survey questions. Observations regarding resident performance in the empathy subscores must be interpreted with caution since there was no correlation between the two examiners’ scores for these subscores. However, it is of interest that, although skills of self-review by checklist were themselves suboptimal, trainees were able to identify where they were weak when asked qualitatively. The importance of skilled delivery of the diagnosis of ALS is demonstrated in the existing literature. Patient satisfaction is linked to the ability to ask questions when receiving the diagnosis of ALS (2). Both patients with ALS and their caregivers have higher satisfaction in communicating with the physician if they feel the doctor understands their feelings (15). Since, in this study, resident performance was poorest for ‘identifying the need for sufficient time’ and ‘summarizing the discussion, asking if there are further questions, and confirming understanding’, it appears that there is an overlap between the behaviors patients/caregivers desire and those behaviours that residents must improve upon. In the survey, residents eloquently described reasons for why breaking bad news was difficult: feeling powerless to help, finding it emotionally draining, feeling partially responsible for the diagnosis, cultural and language barriers, inadequate knowledge, and difficulty dealing with strong emotions. Residency training programs may wish to focus both on these specific areas of weakness uncovered in the checklist as well as the residents’ stated barriers to breaking bad news. All residents endorsed the importance of competence in breaking bad news, and the majority desired further training since they believe education in this area is lacking. Specifically, residents are largely unaware of the available guidelines for diagnosis delivery in ALS. The residents’ discomfort with the situation was clear in their survey responses. Use of standardized patients for teaching and testing communication skills during residency training might translate into greater resident confidence and efficacy in practice. Resident responses indicate that both the OSCE and SP were well received. Residents have limited exposure in breaking bad news to patients with ALS since it is sometimes deemed inappropriate for junior members of the team to be involved in such difficult conversations. Johnston et al. (2) found that patient satisfaction with diagnosis delivery was in part related to a physician consultant relaying the information. If concerns regarding participation exist, then videotaping a standardized scenario with an SP might be an adequate approximation of this experience. An important lesson for staff physicians may also be extrapolated from the resident survey. Not all demonstrations of diagnosis delivery were viewed
favourably, and close examination of technique even at the staff physician level may be appropriate. Findings with respect to self-assessment Although expert and learner assessments are often unrelated (16,17), use of explicit behavioural guidelines (18) and later viewing recorded performances may improve the correlation between expert grading and learner self-assessment (18–20). Despite this, our study found alignment between examiners and residents was lacking both prior to and after watching the video. This was evident for both overall scores and subscores. The residents found the experience beneficial, with the majority thinking it would improve their future effectiveness in breaking bad news. This finding is consistent with the literature (16,20,21). Females scored higher than males in all subgroups according to the examiners’ grading. Despite this, there was a trend toward female participants scoring themselves lower on measures of self-assessment than their male counterparts. This may either reflect under-rating by the females or over-rating by the males in the group. Symons et al. had a similar finding wherein males believed they were more adept in communication than their assessors (22). Aspegren’s literature review found that males were slower to learn communication skills (5). Although this generalization is not applicable to all individuals, it may be that some male learners within a residency group might require more aid in developing the skill of breaking bad news, and may need to be made aware of their need for improvement. That senior residents achieved higher grand total scores, as well as higher empathy subscores in this study, suggests that this skill may be acquired during residency. This is also in keeping with findings from other studies which emphasize that communication and empathy can be taught (5,23). Empathy scoring An unexpected finding was the lack of agreement between the two examiners’ empathy scores. Empathy assessment is problematic, and many measures lack sufficient validity, internal consistency, and reliability (24). The lack of correlation on the empathy subscore may reflect a lack of agreement on what constitutes empathy, or that a checklist score is not an appropriate assessment tool. Although this study’s checklist was based upon existing guidelines from the AAN and EFNS, and on the ‘SPIKES’ protocol, the checklist including the empathy subsection has not been previously validated. Limitations and future directions One limitation of the study was its small sample size (n 22); it may be useful to repeat the study with
Breaking bad news in ALS: the need for medical education more participants. Use of an SP and the fixed time requirement of the OSCE may decrease the realism of the situation. The lack of correlation between assessors’ empathy subscores might indicate a need to standardize assessment methods and training of assessors in this more subjective area of evaluation. Having the SP evaluate the residents would have been helpful. Another limitation of the study is the weight given to checklists that may not reflect overall competence. Providing formal feedback (25,26) and showing a gold standard video (27) in future studies might improve self-assessment and enhance the experience for residents. Participation might also be beneficial for staff members, perhaps as continuing medical education. Further research may assess whether the content and method of training identified by residents results in improved performance and comfort in breaking bad news. Feedback from people with ALS and their caregivers in future studies may prove invaluable. The results of this study indicate the need for resident education in breaking the bad news of an ALS diagnosis. Participants in the study valued the experience and endorsed the importance of acquiring this specific skill. Our study adds to the evidence that specific training in communicating a difficult diagnosis is essential in neurology residency. Acknowledgements The authors thank Ted Roberts for his work in checklist development, Penny Smyth for facilitating the OSCE. Declaration of interest: Kerri L. Schellenberg was the recipient of the Betty Norman research fellowship of the ALS Society of Canada. Wendy S.W. Johnston receives research support from the Canadian Institutes of Health Research. The authors alone are responsible for the content and writing of the paper. References 1. McCluskey L, Casarett D, Siderowf A. Breaking the news: a survey of ALS patients and their caregivers. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5:131–5. 2. Johnston M, Earll L, Mitchell E, Morrison V, Wright S. Communicating the diagnosis of motor neuron disease. Palliat Med. 1996;10:23–34. 3. Chio A, Borasio GD. Breaking the news in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5:195–201. 4. Maguire P, Fairbairn S, Fletcher C. Consultation skills of young doctors: II. Most young doctors are bad at giving information. Br Med J (Clin Res Ed). 1986;292: 1576–8. 5. Aspegren K. BEME Guide No. 2. Teaching and learning communication skills in medicine: a review with quality grading of articles. Medical Teacher. 1999;21:563–70.
53
6. Watling CJ, Brown JB. Education research: communication skills for neurology residents. Structured teaching and reflective practice. Neurology. 2007;69:20–6. 7. Schuh LA, Biondo A, An A, Newman D, Rykzko S, Remer S, et al. Neurology resident learning in an end-of-life/ palliative care course. J Palliat Med. 2007;10:178–81. 8. Fuller G, Smith P. Rediscovering the wheel-teaching communication skills using videotaped clinical consultations in specialist training. Clin Med. 2001;1:203–4. 9. Smith PE, Fuller GN, Kinnersley P, Brigley S, Elwyn G. Using simulated consultations to develop communications skills for neurology trainees. Eur J Neurol. 2002; 9:83–7. 10. Miller RG, Rosenberg JA, Gelinas DF, Mitsumoto H, Newman D, Sufit R, et al. Practice parameter: the care of the patient with amyotrophic lateral sclerosis (an evidencebased review): report of the Quality Standards Subcommittee of the American Academy of Neurology: ALS Practice Parameters Task Force. Neurology. 1999;52:1311–23. 11. Andersen PM, Borasio GD, Dengler R, Hardiman O, Kollewe K, Leigh PN, et al. EFNS task force on management of amyotrophic lateral sclerosis: guidelines for diagnosing and clinical care of patients and relatives. Eur J Neurol. 2005;12:921–38. 12. Baile WF, Buckman R, Lenzi R, Glober G, Beale, EA, Kudelka AP. SPIKES – a six-step protocol for delivering bad news: application to the patient with cancer. Oncologist. 2000;5:302–11. 13. Casey PM, Goepfert AR, Espey EL, Hammoud MM, Kaczmarczuk JM, Katz NT, et al. To the point: reviews in medical education – the Objective Structured Clinical Examination. Journal of Obstetrics. 2009;200:25–34. 14. Yudkowsky R, Alseidi A, Cintron J. Beyond fulfilling the core competencies: an objective structured clinical examination to assess communication and interpersonal skills in a surgical residency. Current Surgery. 2004;61:499–503. 15. Chio A, Montuschi A, Cammarosano S, de Mercanti S, Cavallo E, Ilardi A, et al. ALS patients and caregivers’ communication preferences and information seeking behaviour. Eur J Neurol. 2008;15:55–60. 16. Jones A, McArdle PJ, O’Neill PA. How well prepared are graduates for the role of pre-registration house officer? A comparison of the perceptions of new graduates and educational supervisors. Med Educ. 2001;35:578–84. 17. Barnsley L, Lyon PM, Ralston SJ, Hibbert EJ, Cunningham I, Gordon FC, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38:358–67. 18. Gordon MJ. A review of the validity and accuracy of selfassessments in health professions training. Acad Med. 1991;66:762–9. 19. Barrows H. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. Acad Med. 1993;68:443–51. 20. Stuart MR, Goldstein HS, Snope FC. Self-evaluation by residents in family medicine. J Fam Pract. 1980; 10:639–42. 21. Calhoun JG, Woolliscroft JO, Hockman EM, Wolf FM, Davis WK. Evaluating medical student clinical skill performance: relationships among self, peer, and expert ratings. Res Med Educ. 1984;23:205–10. 22. Symons AB, Swanson A, McGuigan D, Orrange S, Akl EA. A tool for self-assessment of communication skills and professionalism in residents. BMC Med Educ. 2009; 9:1–7. 23. Riess H, Kelley JM, Bailey RW, Dunn EJ, Phillips M. Empathy training for resident physicians: a randomized controlled trial of a neuroscience-informed curriculum. J Gen Intern Med. 2012;27:1280–6. 24. Hemmerdinger JM, Stoddart SDR, Lilford RJ. A systematic review of tests of empathy in medicine. BMC Med Educ. 2007;7:24.
54
K. L. Schellenberg et al.
25. Martin D, Regehr G, Hodges B, McNaughton N. Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Acad Med. 1998; 73:1201–6. 26. Srinivasan M, Hauer K, Der-Martirosian C, Wilkes M, Gesundheit N. Does feedback matter? Practice-based
Supplementary material available online Supplementary Appendix 1–3 available online at http://www.informahealthcare.com/doi/abs/10.3109/ 21678421.2013.843711.
learning for medical students after a multi-institutional clinical performance examination. Med Educ. 2007;41:857–65. 27. Hawkins SC, Osborne A, Schofield SJ, Pournaris DJ, Chester JF. Improving the accuracy of self-assessment of practical skills using video feedback: the importance of using benchmarks. Medical Teacher. 2012;34:279–84.
Copyright of Amyotrophic Lateral Sclerosis & Frontotemporal Degeneration is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
ORIGINAL ARTICLE
Breaking bad news in amyotrophic lateral sclerosis: The need for medical education
KERRI L. SCHELLENBERG1, SUSIE J. SCHOFIELD2, SHOUFAN FANG3 & WENDY S. W. JOHNSTON1 1University
of Alberta, 2University of Dundee, and 3Alberta Health Services, Cardiovascular Health and Stroke Strategic Clinical Network, Edmonton, Canada
Abstract The manner in which physicians deliver difficult diagnoses is an area of discontent for patients with amyotrophic lateral sclerosis (ALS). The American Academy of Neurology’s Practice Parameter for care of the ALS Patient recommended teaching and evaluating strategies for disclosing the diagnosis (10). Our objective was to examine residents’ ability in and perceptions of communicating the diagnosis of ALS. Twenty-two resident physicians were videotaped and rated by two ALS neurologists as they delivered an ALS diagnosis to a standardized patient (SP) during an objective structured clinical examination (OSCE). Residents self-rated immediately after the OSCE, again after viewing their videotape, and completed a survey regarding the OSCE and delivering difficult diagnoses. OSCE performance was suboptimal, particularly for communication skills and empathy. The two examiners’ scores correlated except for the empathy subscore. Residents’ self-assessments did not align with the examiners’ scores either before or after watching their videotape. The survey uncovered residents’ apprehension and dissatisfaction with their training in diagnosis delivery. The results highlight a need for resident education in delivering an ALS diagnosis. The lack of correlation between residents’ and examiners’ scoring requires further study. Evaluation of empathy is particularly challenging. Residents agreed that OSCE participation was worthwhile. Key words: Methods of education, amyotrophic lateral sclerosis, palliative care, professional conduct and ethics
Introduction The manner in which physicians break bad news is an area of discontent for patients with ALS (1,2). The future psychological well-being of patients is profoundly affected by this interaction (3). Lack of physician training is one barrier to breaking bad news effectively (4). Fortunately, it has been shown that communication skills may be learned through experiential courses (5). Although courses have been developed for teaching neurology trainees communication skills, including the skill of breaking bad news (6–9), none has been reported in the context of ALS. Since general neurologists are often the first to deliver the diagnosis of ALS, it is important that all residents acquire the necessary skills. The main objective of this study was to explore the need for medical education with respect to residents’ ability to communicate the diagnosis of ALS.
The first step in determining appropriate education is to assess the existing skill level of neurology residents in delivering an ALS diagnosis. Secondary areas of interest pertain to this project’s use of self-assessment: do ratings provided by the examiners correlate to residents’ self-ratings; and how do residents perceive the exercise of self-assessment?
Material and methods The American Academy of Neurology (AAN) Practice Parameter for care of the ALS patient recommended teaching and evaluating strategies for disclosing the diagnosis (10). A Medline search combining results from the key words ‘neurology, residency and internship, education, ALS’ yielded 257 articles (limit English language); none per-
Correspondence: K. L. Schellenberg, 8440-112 St. NW 2E3 WMC, Edmonton, Alberta, Canada T6G 2B7. Tel: 780 407 6528. Fax: 780 407 1325. E-mail: [email protected] (Received 20 May 2013 ; accepted 4 September 2013) ISSN 2167-8421 print/ISSN 2167-9223 online © 2014 Informa Healthcare DOI: 10.3109/21678421.2013.843711
48
K. L. Schellenberg et al.
tained to resident education in breaking bad news in the context of ALS. Setting and participants Multiple objective structured clinical examination (OSCE) stations are organized annually at the University of Alberta for junior (year 1–3) and senior (year 4–5) neurology residents from several five-year residency programs to evaluate knowledge and skills. One examination station was created by the study coordinators to evaluate breaking bad news in ALS. After receiving ethics approval, all residents (n 28) were invited to participate, and written consent was obtained from participating residents prior to the OSCE.
of Neurological Societies (EFNS) (10,11) and on the six-step protocol for delivering bad news ‘SPIKES’ (Setting, Perception, Invitation, Knowledge, Empathy, Strategy and Summary) (12). Although developed for oncologists, ‘SPIKES’ has been used successfully within the ALS community (3). The components of ‘SPIKES’ have also been used for patients with ALS and their caregivers’ evaluation of physician performance in delivering an ALS diagnosis (1). The checklist had three subsections: action (22 points), empathy (6 points), and general communication (10 points) for a total of 38 points (Table I). A higher score corresponded to better performance. There were no established norms for this checklist since it had not been previously used in a study.
Design Neurology residents’ performances were videotaped during an OSCE as they disclosed the diagnosis of ALS to a standardized patient (SP). Two neurologists with specialized training and a current practice in the care of patients with ALS independently observed the videotapes and rated the residents who consented to participate according to a checklist developed for this study (see below). Residents assessed themselves using the same checklist as the examiners immediately after the OSCE and again after watching their own videotape. Residents also completed a survey pertaining to the OSCE, medical education, and breaking bad news. Formation of the checklist A rating checklist consisting of specific behavioral criteria was developed based upon the existing guidelines from the AAN and European Federation
OSCE station Instructions to the candidates and SP were formulated based on a scenario residents may encounter – imparting the ALS diagnosis to an inpatient at the request of the patient. The OSCE station was supervised and instructions given by a faculty member (not a study evaluator) who observed the encounter in the room. To standardize the encounter, the same SP with a performing arts background was used in all OSCEs. Instructions for the SP, including scripted questions and responses, were submitted to the SP and SP coordinator. The SP received 40 minutes of standard training with the SP coordinator, and met with a study investigator prior to the OSCEs to receive further instruction. The first two resident performances were used as practice runs for the SP, and were not included in the study data. One study investigator provided feedback to the SP after each practice OSCE.
Table I. Checklist subsections. Action Preparing for conversation: · attention to privacy · time management · inclusion of the patient’s support system Interviewing the patient: · appropriate greeting · assessing the patient’s existing knowledge and extent to which the patient wishes to be informed · giving a warning statement · delivering the diagnosis without equivocation · imparting ALS specific material including the known pathophysiology, prognosis, treatment, research opportunities, and support organizations · providing hope · supporting wishes for a second opinion · summarizing the discussion; confi rming understanding; and arranging follow-up
Empathy Empathetic diagnosis delivery and response: · delivering the diagnosis in a warm and sensitive manner · acknowledging and exploring the patient’s response · conveying the reaction is understandable
Communication Verbal communication: · speaking without technical jargon, excessive bluntness, or euphemisms · giving information in small chunks · checking for understanding and encouraging questions Non-verbal communication: · demonstrating attention to the patient and maintaining eye contact
Breaking bad news in ALS: the need for medical education The OSCE was selected as an assessment tool since it evaluates both cognitive and performance based aptitudes, including communication techniques. Delivering an ALS diagnosis involves ALS specific knowledge as well as empathy and general communication strategies. Utility of the OSCE to measure communication skills is well established (13,14). Statistical analysis Statistical analyses were conducted with statistical analysis software SAS (version 9.2, developed by SAS Institute Inc.). The 95% significance level (i.e. α 0.05) was taken to reject null hypotheses. When necessary, a 90% significance level (i.e. α 0.10) was used in looking for trend. Null hypotheses for specific analyses were given, and all tests were twotailed. Spearman’s rank correlation test was applied in testing the correlation of the two assessors’ scoring. The null hypothesis was Spearman’s RHO ρ 0 and the alternative hypothesis was ρ 0. Generalized Estimating Equation (GEE) was applied in analysis of the difference between residents’ self-scores and assessors’ scoring, the effect of viewing the video on self-scores, and the correlation between the residents’ demographic factors and their scores. GEE analysis of the demographic factors included three features: gender (male, female), year of residency (junior year 1–3, senior year 4–5) and site (residents from host site, residents from other centres). All GEE models had Poisson distribution as an SAS model option and exchangeable correlation was induced when residents’ ID numbers were used to identify repeated (pre-/post-DVD watching) measurement. After a GEE model was fitted, a Z-test was conducted to test the estimated coefficients, which represented the effect of evaluators (residents or examiners), viewing video, and the selected demographic features. The null hypotheses of the Z-tests were the estimated coefficients being equal to zero, and the alternative hypotheses were that they were not equal to zero.
49
Results Of the 28 residents involved in the OSCE, 25 consented to participate, three did not consent, the first two consenting residents were used as practice stations for the SP, one consented but did not return the required forms; thus 22 participated in the study. There were eight (36%) male and 14 (64%) female participants in the following years of residency: Year 1 3 (14%), Year 2 1 (5%), Year 3 6 (27%), Year 4 5 (23%), Year 5 7 (32%). OSCE results: examiner checklist scores, subscores and resident self-scoring There was positive correlation for the two examiners’ overall scores as well as the action and communication subscores, but no significant correlation for the empathy subscore (Spearman’s correlation coefficient) (Table II). The checklist items ‘assuring SP regarding opportunities for research involvement’ and ‘when directly asked by SP, stating there is no cure’ were performed adequately by all residents. Items with the poorest performance included ‘identifying the need for sufficient time’ and ‘summarizing the discussion, asking if there are further questions, and confirming understanding’. In GEE models for comparison of the scores from residents and examiners, the examiners’ scores or pre-video self-score were treated as the basis of comparison. Therefore, the coefficients of GEE models reflect the difference of the residents’ scores – the positive/negative coefficients represented that residents’ scores were higher/lower than that of examiners or pre-video. Comparing scores from examiners and residents prior to viewing video, GEE models presented positive coefficients and small p-values ( 0.05) for total score and all subscores (Table II), which reflected that the residents significantly overevaluated their performance. All coefficients from GEE models to represent video effect were negative, which meant that residents lowered their scores to more closely resemble those of the examiners. However,
Table II. Checklist results. Total Examiner 1 (mean scores, standard deviation) Examiner 2 (mean scores, standard deviation) Correlation between the examiners’ scores (Spearman’s RHO, p -value) Both examiners (mean, standard deviation) Resident scores compared to examiner scores pre-video (coefficient, standard error, Z-value, p -value from GEE) Resident self-score change after viewing video (coefficient, standard error, Z-value, p -value from GEE) *Indicates 95% significance level was reached.
Action
Communication
Empathy
24.6/38, 3.634
15.5/22, 2.041
5.9/10, 1.540
3.2/6, 1.152
25.8/38, 4.393
16.2/22, 2.423
6.1/10, 1.670
3.5/6, 1.185
0.56, 0.0065*
0.80, 0.0001*
0.44, 0.0415*
0.23, 0.2955
25.2, 4.032
15.8, 2.244
6.0, 1.592
3.4, 1.163
0.145, 0.045, 3.23, 0.0012* 0.027, 0.034, 0.78, 0.4331
0.085, 0.041, 2.08, 0.0374* 0.005, 0.037, 0.14, 0.8857
0.219, 0.064, 3.45, 0.0006* 0.025, 0.059, 0.41, 0.6794
0.291, 0.083, 3.53, 0.0004* 0.141, 0.057, 2.48, 0.0131*
50
K. L. Schellenberg et al.
only the empathy subscore had significant reduction; the remaining p-values were not small enough to reject the null hypotheses (Table II). Relationship between demographic information and resident and examiner ratings Demographic information was compared to both the residents’ and examiners’ ratings using a generalized estimating equation. With respect to the residents’ self-score in this small sample size, there was a trend approaching significance in that female residents scored themselves lower than males on the communication subscore with a 90% significance level (Table III). Different and sometimes opposing relationships were uncovered when the examiners’ scoring of residents and demographic information were evaluated. Female residents achieved higher scores for the grand total and action and empathy subscores (95% significance level) with a trend approaching significance for the communication subscore (90% significance level). Seniors earned greater scores than juniors for grand total and empathy subscores (95% significance level) (Table IV). Survey results: residents’ perception of the OSCE and the skill of breaking bad news All participating residents agreed the OSCE was worthwhile. Thematic analysis of the free-text responses revealed two aspects residents found useful: the exercise of self-assessment, and the practice/ exam preparation value. Virtually all (21/22) residents felt using an SP was effective (95%). However, nine (41%) mentioned unhelpful aspects of the OSCE station: five had no prior experience with certain aspects of the exam (23%); three found the setting artificial (14%); and one wished for immediate feedback (5%). Eighteen (82%) felt watching their videotape was useful and 15 (68%) believed they would be more
effective in breaking bad news after having watched their video. Thematic analysis of free-text comments revealed that residents felt they could improve on various aspects of communication and empathy. Several trainees commented on areas for self-improvement: “I need to be more compassionate and less clinical in the interaction. I need to take some time to acknowledge the human aspect of the situation; the fact that I have given some devastating news. I want to be less verbose and more direct when I do attempt to relay information.” “I think I learned that I was a bit too reserved in my approach to the patient (partially out of my own nervousness), but that it may have come across as cold and unfeeling. I think I could make more effort to reach out to the patient, seek their emotional responses and answer their questions.” “The non-verbal communication seemed inadequate. Eye contact was not sufficient and could have been more empathetic. I think it looked a little bit like it was rehearsed instead of spontaneous.” “I was able to pick up on the parts of the disclosure that made me uncomfortable because my body language clearly reflected it.” “I will pay more attention in the future to use less jargon and be clear in conversation.” All residents felt breaking bad news effectively was important, but 10 (45%) were unsure about their own effectiveness. The majority of residents (17, 77%) reported prior experience; however, 15 found it stressful (68%). They identified several issues that made breaking bad news particularly difficult: “I feel partially responsible for the patient’s diagnosis when breaking bad news, even though I know it is not my fault that they have a terrible disease. I dislike the feeling of powerlessness, and my own ignorance to help them. I find it difficult to deal with their strong emotions.” “Use of appropriate words/language, as English is not my first language. Cultural differences. Need to
Table III. GEE parameter estimates based on residents’ self-score. Model dependent Total
Action
Empathy
Communication
Parameter
Estimate
SE
Z
p -value
Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female Intercept Senior Visiting centre Female
3.342 0.031 0.079 0.019 2.798 0.024 0.055 0.013 1.412 0.077 0.082 0.065 2.049 0.126 0.145 0.150
0.080 0.066 0.061 0.061 0.073 0.057 0.050 0.051 0.094 0.124 0.126 0.104 0.106 0.087 0.078 0.087
41.56 0.46 1.30 0.30 38.18 0.42 1.09 0.25 14.98 0.62 0.65 0.63 19.44 1.44 1.85 1.72
.0001 0.6451 0.1936 0.7607 .0001 0.6723 0.2746 0.8041 .0001 0.5359 0.5155 0.5313 .0001 0.1501 0.0645 0.0849
Breaking bad news in ALS: the need for medical education
51
Table IV. GEE parameter estimates based on examiners’ scores. Model dependent Total
Action
Empathy
Communication
Parameter
Estimate
SE
Z
p -value
Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1 Intercept Senior Visiting sites Female Examiner 1
3.125 0.083 0.035 0.167 0.049 2.671 0.082 0.018 0.155 0.046 0.979 0.171 0.130 0.279 0.081 1.733 0.033 0.027 0.133 0.038
0.046 0.042 0.044 0.042 0.030 0.045 0.049 0.048 0.043 0.020 0.098 0.073 0.085 0.080 0.086 0.065 0.069 0.084 0.073 0.061
68.01 1.98 0.78 3.97 1.62 59.40 1.67 0.38 3.59 2.29 9.98 2.34 1.53 3.49 0.94 26.88 0.47 0.32 1.82 0.62
.0001 0.0481* 0.4371 .0001* 0.1057 .0001 0.0946 0.7006 0.0003* 0.0222* .0001 0.0195* 0.1255 0.0005* 0.3454 .0001 0.6387 0.7473 0.0692 0.5371
*Covariates which reached 95% significance level.
know enough about the disease especially prognosis, treatment options, ongoing research to answer patient’s questions.” “Not being too verbose and rambling. Being direct while remaining empathetic.” “The silence and seeing people hurt. Also, wondering if I’m forgetting any important information.” “It’s emotionally draining.” Only three (14%) residents had prior experience in breaking bad news to patients with ALS. All three observed their staff members relay the diagnosis, and one resident participated in giving the diagnosis. The residents’ impressions of their staff members’ execution of the task varied widely: “Positive experience. Was able to observe staff, broke news slowly in waves throughout day as to not overwhelm patient. Discussed with patient and family once diagnosis fully given.” “Another time was in the clinic, and the staff did a horrible job in about three minutes and left the guy sitting in the exam room with a box of tissues,
crying – I still remember vividly how bad I felt leaving the patient like that (but I couldn’t stay behind and talk with the patient when the staff guy was walking away).” Only three (14%) residents were aware of any professional guidelines for breaking bad news to ALS patients. The minority of residents (9%) believed they received adequate instruction on breaking bad news, and the majority (17%) believed more emphasis should be placed on this during training. The residents identified a range of potential topics for further training. (Table V). Discussion Need for resident education in delivering the diagnosis of ALS This study indicates a need for resident education in breaking bad news to patients with ALS, evidenced both by expert examiner scores and residents’ opinions. The examiners’ rating of residents demonstrated
Table V. Suggestions for further training identified in resident survey. Didactic · · · ·
organization of approach further training with SPIKES how to achieve an appropriate setting access to workshops, lectures, demonstrations · sensitivity to cultural values
Practice · practice sessions, including sessions in ward rotations · opportunity to watch staff neurologists impart bad news · opportunity to be observed breaking bad news, given feedback · videotape practice, watch own performance
Communication skills · · · ·
cultivate empathic responses useful phrases what NOT to say better communication skills including non-verbal communication
How to . . . · · · ·
emphasize hope be direct handle patients emotional responses break bad news in least traumatic way
52
K. L. Schellenberg et al.
suboptimal overall scores, with particularly poor performance within the empathy and communication subscores. The residents identified the same two areas of weakness in the free-text survey questions. Observations regarding resident performance in the empathy subscores must be interpreted with caution since there was no correlation between the two examiners’ scores for these subscores. However, it is of interest that, although skills of self-review by checklist were themselves suboptimal, trainees were able to identify where they were weak when asked qualitatively. The importance of skilled delivery of the diagnosis of ALS is demonstrated in the existing literature. Patient satisfaction is linked to the ability to ask questions when receiving the diagnosis of ALS (2). Both patients with ALS and their caregivers have higher satisfaction in communicating with the physician if they feel the doctor understands their feelings (15). Since, in this study, resident performance was poorest for ‘identifying the need for sufficient time’ and ‘summarizing the discussion, asking if there are further questions, and confirming understanding’, it appears that there is an overlap between the behaviors patients/caregivers desire and those behaviours that residents must improve upon. In the survey, residents eloquently described reasons for why breaking bad news was difficult: feeling powerless to help, finding it emotionally draining, feeling partially responsible for the diagnosis, cultural and language barriers, inadequate knowledge, and difficulty dealing with strong emotions. Residency training programs may wish to focus both on these specific areas of weakness uncovered in the checklist as well as the residents’ stated barriers to breaking bad news. All residents endorsed the importance of competence in breaking bad news, and the majority desired further training since they believe education in this area is lacking. Specifically, residents are largely unaware of the available guidelines for diagnosis delivery in ALS. The residents’ discomfort with the situation was clear in their survey responses. Use of standardized patients for teaching and testing communication skills during residency training might translate into greater resident confidence and efficacy in practice. Resident responses indicate that both the OSCE and SP were well received. Residents have limited exposure in breaking bad news to patients with ALS since it is sometimes deemed inappropriate for junior members of the team to be involved in such difficult conversations. Johnston et al. (2) found that patient satisfaction with diagnosis delivery was in part related to a physician consultant relaying the information. If concerns regarding participation exist, then videotaping a standardized scenario with an SP might be an adequate approximation of this experience. An important lesson for staff physicians may also be extrapolated from the resident survey. Not all demonstrations of diagnosis delivery were viewed
favourably, and close examination of technique even at the staff physician level may be appropriate. Findings with respect to self-assessment Although expert and learner assessments are often unrelated (16,17), use of explicit behavioural guidelines (18) and later viewing recorded performances may improve the correlation between expert grading and learner self-assessment (18–20). Despite this, our study found alignment between examiners and residents was lacking both prior to and after watching the video. This was evident for both overall scores and subscores. The residents found the experience beneficial, with the majority thinking it would improve their future effectiveness in breaking bad news. This finding is consistent with the literature (16,20,21). Females scored higher than males in all subgroups according to the examiners’ grading. Despite this, there was a trend toward female participants scoring themselves lower on measures of self-assessment than their male counterparts. This may either reflect under-rating by the females or over-rating by the males in the group. Symons et al. had a similar finding wherein males believed they were more adept in communication than their assessors (22). Aspegren’s literature review found that males were slower to learn communication skills (5). Although this generalization is not applicable to all individuals, it may be that some male learners within a residency group might require more aid in developing the skill of breaking bad news, and may need to be made aware of their need for improvement. That senior residents achieved higher grand total scores, as well as higher empathy subscores in this study, suggests that this skill may be acquired during residency. This is also in keeping with findings from other studies which emphasize that communication and empathy can be taught (5,23). Empathy scoring An unexpected finding was the lack of agreement between the two examiners’ empathy scores. Empathy assessment is problematic, and many measures lack sufficient validity, internal consistency, and reliability (24). The lack of correlation on the empathy subscore may reflect a lack of agreement on what constitutes empathy, or that a checklist score is not an appropriate assessment tool. Although this study’s checklist was based upon existing guidelines from the AAN and EFNS, and on the ‘SPIKES’ protocol, the checklist including the empathy subsection has not been previously validated. Limitations and future directions One limitation of the study was its small sample size (n 22); it may be useful to repeat the study with
Breaking bad news in ALS: the need for medical education more participants. Use of an SP and the fixed time requirement of the OSCE may decrease the realism of the situation. The lack of correlation between assessors’ empathy subscores might indicate a need to standardize assessment methods and training of assessors in this more subjective area of evaluation. Having the SP evaluate the residents would have been helpful. Another limitation of the study is the weight given to checklists that may not reflect overall competence. Providing formal feedback (25,26) and showing a gold standard video (27) in future studies might improve self-assessment and enhance the experience for residents. Participation might also be beneficial for staff members, perhaps as continuing medical education. Further research may assess whether the content and method of training identified by residents results in improved performance and comfort in breaking bad news. Feedback from people with ALS and their caregivers in future studies may prove invaluable. The results of this study indicate the need for resident education in breaking the bad news of an ALS diagnosis. Participants in the study valued the experience and endorsed the importance of acquiring this specific skill. Our study adds to the evidence that specific training in communicating a difficult diagnosis is essential in neurology residency. Acknowledgements The authors thank Ted Roberts for his work in checklist development, Penny Smyth for facilitating the OSCE. Declaration of interest: Kerri L. Schellenberg was the recipient of the Betty Norman research fellowship of the ALS Society of Canada. Wendy S.W. Johnston receives research support from the Canadian Institutes of Health Research. The authors alone are responsible for the content and writing of the paper. References 1. McCluskey L, Casarett D, Siderowf A. Breaking the news: a survey of ALS patients and their caregivers. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5:131–5. 2. Johnston M, Earll L, Mitchell E, Morrison V, Wright S. Communicating the diagnosis of motor neuron disease. Palliat Med. 1996;10:23–34. 3. Chio A, Borasio GD. Breaking the news in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5:195–201. 4. Maguire P, Fairbairn S, Fletcher C. Consultation skills of young doctors: II. Most young doctors are bad at giving information. Br Med J (Clin Res Ed). 1986;292: 1576–8. 5. Aspegren K. BEME Guide No. 2. Teaching and learning communication skills in medicine: a review with quality grading of articles. Medical Teacher. 1999;21:563–70.
53
6. Watling CJ, Brown JB. Education research: communication skills for neurology residents. Structured teaching and reflective practice. Neurology. 2007;69:20–6. 7. Schuh LA, Biondo A, An A, Newman D, Rykzko S, Remer S, et al. Neurology resident learning in an end-of-life/ palliative care course. J Palliat Med. 2007;10:178–81. 8. Fuller G, Smith P. Rediscovering the wheel-teaching communication skills using videotaped clinical consultations in specialist training. Clin Med. 2001;1:203–4. 9. Smith PE, Fuller GN, Kinnersley P, Brigley S, Elwyn G. Using simulated consultations to develop communications skills for neurology trainees. Eur J Neurol. 2002; 9:83–7. 10. Miller RG, Rosenberg JA, Gelinas DF, Mitsumoto H, Newman D, Sufit R, et al. Practice parameter: the care of the patient with amyotrophic lateral sclerosis (an evidencebased review): report of the Quality Standards Subcommittee of the American Academy of Neurology: ALS Practice Parameters Task Force. Neurology. 1999;52:1311–23. 11. Andersen PM, Borasio GD, Dengler R, Hardiman O, Kollewe K, Leigh PN, et al. EFNS task force on management of amyotrophic lateral sclerosis: guidelines for diagnosing and clinical care of patients and relatives. Eur J Neurol. 2005;12:921–38. 12. Baile WF, Buckman R, Lenzi R, Glober G, Beale, EA, Kudelka AP. SPIKES – a six-step protocol for delivering bad news: application to the patient with cancer. Oncologist. 2000;5:302–11. 13. Casey PM, Goepfert AR, Espey EL, Hammoud MM, Kaczmarczuk JM, Katz NT, et al. To the point: reviews in medical education – the Objective Structured Clinical Examination. Journal of Obstetrics. 2009;200:25–34. 14. Yudkowsky R, Alseidi A, Cintron J. Beyond fulfilling the core competencies: an objective structured clinical examination to assess communication and interpersonal skills in a surgical residency. Current Surgery. 2004;61:499–503. 15. Chio A, Montuschi A, Cammarosano S, de Mercanti S, Cavallo E, Ilardi A, et al. ALS patients and caregivers’ communication preferences and information seeking behaviour. Eur J Neurol. 2008;15:55–60. 16. Jones A, McArdle PJ, O’Neill PA. How well prepared are graduates for the role of pre-registration house officer? A comparison of the perceptions of new graduates and educational supervisors. Med Educ. 2001;35:578–84. 17. Barnsley L, Lyon PM, Ralston SJ, Hibbert EJ, Cunningham I, Gordon FC, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38:358–67. 18. Gordon MJ. A review of the validity and accuracy of selfassessments in health professions training. Acad Med. 1991;66:762–9. 19. Barrows H. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. Acad Med. 1993;68:443–51. 20. Stuart MR, Goldstein HS, Snope FC. Self-evaluation by residents in family medicine. J Fam Pract. 1980; 10:639–42. 21. Calhoun JG, Woolliscroft JO, Hockman EM, Wolf FM, Davis WK. Evaluating medical student clinical skill performance: relationships among self, peer, and expert ratings. Res Med Educ. 1984;23:205–10. 22. Symons AB, Swanson A, McGuigan D, Orrange S, Akl EA. A tool for self-assessment of communication skills and professionalism in residents. BMC Med Educ. 2009; 9:1–7. 23. Riess H, Kelley JM, Bailey RW, Dunn EJ, Phillips M. Empathy training for resident physicians: a randomized controlled trial of a neuroscience-informed curriculum. J Gen Intern Med. 2012;27:1280–6. 24. Hemmerdinger JM, Stoddart SDR, Lilford RJ. A systematic review of tests of empathy in medicine. BMC Med Educ. 2007;7:24.
54
K. L. Schellenberg et al.
25. Martin D, Regehr G, Hodges B, McNaughton N. Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Acad Med. 1998; 73:1201–6. 26. Srinivasan M, Hauer K, Der-Martirosian C, Wilkes M, Gesundheit N. Does feedback matter? Practice-based
Supplementary material available online Supplementary Appendix 1–3 available online at http://www.informahealthcare.com/doi/abs/10.3109/ 21678421.2013.843711.
learning for medical students after a multi-institutional clinical performance examination. Med Educ. 2007;41:857–65. 27. Hawkins SC, Osborne A, Schofield SJ, Pournaris DJ, Chester JF. Improving the accuracy of self-assessment of practical skills using video feedback: the importance of using benchmarks. Medical Teacher. 2012;34:279–84.
Copyright of Amyotrophic Lateral Sclerosis & Frontotemporal Degeneration is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
