The American Journal of Surgery (2015) -, -–-
Informed consent training improves surgery resident performance in simulated encounters with standardized patients Britta M. Thompson, Ph.D., M.S.a,b,*, Rhonda A. Sparks, M.D.c, Jonathan Seavey, M.D.d,e, Michelle D. Wallace, B.S.c, Jeremy Irvan, M.D.d, Alexander R. Raines, M.D.d, Heather McClure, J.D.f, Mikio A. Nihira, M.D.g, Jason S. Lees, M.D.d a
University of Oklahoma College of Medicine, Office of Medical Education and Clinical Skills Education and Testing Center, 941 Stanton L. Young Boulevard, BSEB 115A, Oklahoma City, OK 73104, USA; bPennState Hershey College of Medicine, Hershey, PA, USA; cUniversity of Oklahoma College of Medicine, Clinical Skills Education and Testing Center, Oklahoma City, OK, USA; d Department of Surgery, University of Oklahoma College of Medicine, Oklahoma City, OK, USA; e National Naval Medical Center, Bethesda, MD, USA; fOU Physicians, Professional Liability, Patient Safety & Risk Management, Oklahoma City, OK, USA; gDepartment of Obstetrics and Gynecology, University of Oklahoma College of Medicine, Oklahoma City, OK, USA
KEYWORDS: Informed consent; Resident education; Standardized patients; Communication
Abstract BACKGROUND: Although informed consent is vital to patient–physician communication, little training is provided to surgical trainees. We hypothesized that highlighting critical aspects of informed consent would improve resident performance. METHODS: Eighty (out of 88) surgical postgraduate year 1 surgical residents were randomly assigned to one of the 2 cases (laparoscopic cholecystectomy or ventral herniorrhaphy) and instructed to obtain and document informed consent with a standardized patient (SP) followed by a didactic training session. The residents then obtained and documented informed consent with the other case with the other SP. SPs graded encounters (‘‘Checklist’’); trained raters graded notes. Repeated measures multivariate analysis of variance (MANOVA) was used to determine differences between pre- and post-training and Checklist versus ‘‘Note’’ scores. RESULTS: Statistically significant pre- to post differences for Note (P , .01) and Checklist (P , .01) along with significant differences between Note and Checklist (P , .01) were noted. CONCLUSIONS: Training improved surgery residents’ ability to discuss and document informed consent. Despite this improvement, significant differences between discussion and documentation persisted. Documentation training is a future area for improvement. Ó 2015 Elsevier Inc. All rights reserved.
This project was funded, in part, through OU Physicians. No other support was provided. * Corresponding author. Tel.: 717-531-0003 x281249; fax: 717-531-3925. E-mail address: [email protected] Manuscript received September 12, 2014; revised manuscript November 17, 2014 0002-9610/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2014.12.044
2
The American Journal of Surgery, Vol -, No -, - 2015
Informed consent is the communication between a patient and a physician resulting in a patient’s agreement for a medical intervention. According to the American College of Surgeons, informed consent is a standard of ethical surgical practice that serves to enhance the patient/surgeon relationship, which may result in improved patient care and outcomes.1 During the informed consent process, the physician should explain the illness itself and the natural course of the illness, the proposed procedure, benefits and common risks (including death) of the procedure, alternatives to the procedure (including no surgery), and who will comprise the surgery team.1 In a systematic review, researchers have documented that many physicians do not meet the minimum standards when they conduct informed consent with patients.2 Physicians tend to discuss the procedure but rarely discuss alternatives, risks, and benefits of the procedure. Even when they deliver complete informed consent, they tend to overestimate patient’s comprehension of the information.2,3 Properly conducted and documented informed consent decreases litigation risks. Approximately 40% of all physicians and almost 60% of surgical subspecialists have been sued.4 Root cause analysis of the lawsuits indicates that communication failures between the physician and the patient rather than treatment failures are the provoking factors.5 Analysis of malpractice claims indicates that properly conducted and documented informed consent is associated with decreased indemnity risk.6 A recent review of gastroenterologists found that inadequate documentation was one of the primary reasons that lawsuits were awarded to patients.7 This review showed that limited documentation in the medical record was critical to provide evidence that the informed consent process had occurred.7 Surprisingly, there have been few published studies regarding interventions that improve residents’ skills in communicating and documenting informed consent. Leclercq et al5 reported that informed consent training interventions with faculty and residents were time consuming, costly, and of questionable success. A small pilot study almost a decade ago with only 8 surgical residents showed significant improvements from pre- to post-test in communication skills with standardized patients (SP) (establishing rapport, discussing the surgical diagnosis and procedure, explaining the risks and complications of the procedure, and providing appropriate feedback to the patient).8 Grossman et al9 reported that case-based instruction of ethics (including informed consent) resulted in increased confidence of residents to discuss options with patients. A more recent educational intervention included an online module, a small group discussion with faculty, and 2 SP cases. Data collected from the training included resident skill self-assessment, which significantly increased pre to post.10 However, most of the reported trainings have included only a few participants; none of them evaluated improvements in informed consent communication and documentation skills. We hypothesized that a short intervention for residents highlighting critical aspects of informed consent would improve resident performance in both communication and documentation skills.
Patients and Methods Participants Eighty-eight postgraduate year (PGY) 1 surgical residents (general, orthopedics, otorhinolaryngology, neurosurgery, urology) who matriculated at the University of Oklahoma College of Medicine during the 2009 to 2013 years were eligible to participate in the study. Approval was obtained prior to this study from the University of Oklahoma Health Sciences Center Institutional Review Board.
Standardized patient cases and training intervention Two SP cases were developed by board-certified surgeons (J.S.L. and M.A.N.), a board-certified family medicine practitioner (R.A.S.), and an attorney specializing in professional liability and healthcare risk management (H.M.). The SP cases (laparoscopic cholecystectomy and ventral herniorrhaphy) were chosen based on commonly performed surgical procedures and risks of complications for the surgery. Both cases were 15 minutes in length for the encounter followed by 10 minutes for postencounter documentation. At pretraining, approximately half of the residents were randomly assigned to the ventral herniorrhaphy case, while the other half were assigned the cholecystectomy case. The case was followed by a didactic training session (approximately 1.5 hours) with an attorney specializing in professional liability and healthcare risk management (H.M.) and the surgery residency program director (J.S.L.). This didactic training with targeted discussion covered basic tenants of malpractice with emphasis on informed consent. After the training (post-training), residents completed the second SP case. We determined direct costs for the training and the 2 cases. We also estimated the amount of time needed for the resident training. To determine the costs associated with the training, we used publically available mean estimates of a mid-career general attorney ($116/hour) (www.salary.com) and mean compensation rates for a general surgeon ($142/ hour).11 To determine the cost of the cases, we used publically available mean estimates for SPs.12 These costs were not inclusive of benefits. We also reported the amount of time required for residents to complete the training.
Instrument development and scoring After an extensive review of the literature, we noted no existing informed consent cases or checklists. Therefore, we created each SP performance checklist based on the American College of Surgeons (2008) guidelines1 for obtaining informed consent, along with experts in the field (J.S.L. and R.A.S.). Each of the items was dichotomously scored (0 5 did not discuss, 1 5 discussed) in accordance with published recommendations for creation of performance tests.13 The checklist covered areas such as nature of the illness and the natural
B.M. Thompson et al.
Informed consent training improves performance
consequences of no treatment, the proposed operation, risks and benefits, and alternative forms of treatment including nonoperative techniques. Representative items included ‘‘The clinician explained the problem with my gallbladder.’’ The post ‘‘Notes’’ were scored in the same manner using the same items (0 5 not included in note, 1 5 included in note). For this study, we were interested in whether the resident had documented each item, not whether they covered an exhaustive list of benefits, risks, alternatives, and so on. Criterion was determined in advance regarding the specific items that were given credit. To ensure consistency, SPs were trained by first having them read through the case. Then, the SPs were extensively trained using role play. We used the same SPs for the cases to ensure consistency, and used SPs who had over 1,000 hours of experience. To ensure consistency between Note ratings, a surgical educator (J.S.L.) and a medical student (J.S.) first independently rated 6 notes from the 2009 to 2010 matriculants. The raters met and reviewed all note grading. Where there were discrepancies in ratings, they came to consensus. The rater agreement for Note grading on each of the 6 post notes was 90.9%, and Kappa was .82, or almost perfect agreement according to published standards.14 A note rating guide was created by the 2 raters and was used by another investigator (B.M.T.) to grade the notes from the 2011 to 2013 matriculants.
and inferential statistics (repeated measures multivariate analysis of variance) to determine differences pre to post and between Note and Checklist scores and between matriculant cohort years using IBM SPSS 22.0 (IBM Corp, Armonk, NY). We set alpha at .05 and determined effect size (educational significance) using h2; we used recommendations of h2 5 .01 as a small effect, h2 5 .06 as a medium effect size, and h2 5 .16 and above as a large effect.15
Results Eighty of the 88 surgery residents at the University of Oklahoma College of Medicine completed both pre- and post-training for this study, for a completion rate of 90.91%. All residents were PGY1. Analysis of the Checklist summed scores from pre- to post-training indicated that at pretraining, residents discussed an average of 6.85 (standard deviation [SD] 51.16) of the 8 items and documented an average of 4.80 (SD 5 1.52) items. At post-training, they discussed an average of 7.40 (SD 5 .89) items and documented an average of 5.89 (SD 5 1.56) items (Fig. 1). Multivariate analysis of variance indicated statistically significant differences overall (P , .001). Checklist (F 5 14.73, P , .01, h2 5 .16) and Note documentation (F 5 33.31, P , .01, h2 5 .30) statistically significantly increased from pre- to post-training. We noted no differences between the 5 matriculant cohorts at pretraining for either Checklist or Note (P . .05). We examined differences between what residents discussed with the SPs as indicated by the Checklist and what they documented in the Note. At pretraining, residents discussed significantly more items (Checklist) than they
Statistics To determine differences between pre- and posttraining, we analyzed each checklist item separately and then combined items to create a summed score for the checklist rated by the SPs (‘‘Checklist’’) and the written post-note (Note). We analyzed our data using descriptive
8
7.4 6.85
7 5.89
Number of Items
6 5
3
4.8
4
Pre Training Encounter
3
Post Training Encounter
2 1 0 Note (p
Informed consent training improves surgery resident performance in simulated encounters with standardized patients Britta M. Thompson, Ph.D., M.S.a,b,*, Rhonda A. Sparks, M.D.c, Jonathan Seavey, M.D.d,e, Michelle D. Wallace, B.S.c, Jeremy Irvan, M.D.d, Alexander R. Raines, M.D.d, Heather McClure, J.D.f, Mikio A. Nihira, M.D.g, Jason S. Lees, M.D.d a
University of Oklahoma College of Medicine, Office of Medical Education and Clinical Skills Education and Testing Center, 941 Stanton L. Young Boulevard, BSEB 115A, Oklahoma City, OK 73104, USA; bPennState Hershey College of Medicine, Hershey, PA, USA; cUniversity of Oklahoma College of Medicine, Clinical Skills Education and Testing Center, Oklahoma City, OK, USA; d Department of Surgery, University of Oklahoma College of Medicine, Oklahoma City, OK, USA; e National Naval Medical Center, Bethesda, MD, USA; fOU Physicians, Professional Liability, Patient Safety & Risk Management, Oklahoma City, OK, USA; gDepartment of Obstetrics and Gynecology, University of Oklahoma College of Medicine, Oklahoma City, OK, USA
KEYWORDS: Informed consent; Resident education; Standardized patients; Communication
Abstract BACKGROUND: Although informed consent is vital to patient–physician communication, little training is provided to surgical trainees. We hypothesized that highlighting critical aspects of informed consent would improve resident performance. METHODS: Eighty (out of 88) surgical postgraduate year 1 surgical residents were randomly assigned to one of the 2 cases (laparoscopic cholecystectomy or ventral herniorrhaphy) and instructed to obtain and document informed consent with a standardized patient (SP) followed by a didactic training session. The residents then obtained and documented informed consent with the other case with the other SP. SPs graded encounters (‘‘Checklist’’); trained raters graded notes. Repeated measures multivariate analysis of variance (MANOVA) was used to determine differences between pre- and post-training and Checklist versus ‘‘Note’’ scores. RESULTS: Statistically significant pre- to post differences for Note (P , .01) and Checklist (P , .01) along with significant differences between Note and Checklist (P , .01) were noted. CONCLUSIONS: Training improved surgery residents’ ability to discuss and document informed consent. Despite this improvement, significant differences between discussion and documentation persisted. Documentation training is a future area for improvement. Ó 2015 Elsevier Inc. All rights reserved.
This project was funded, in part, through OU Physicians. No other support was provided. * Corresponding author. Tel.: 717-531-0003 x281249; fax: 717-531-3925. E-mail address: [email protected] Manuscript received September 12, 2014; revised manuscript November 17, 2014 0002-9610/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2014.12.044
2
The American Journal of Surgery, Vol -, No -, - 2015
Informed consent is the communication between a patient and a physician resulting in a patient’s agreement for a medical intervention. According to the American College of Surgeons, informed consent is a standard of ethical surgical practice that serves to enhance the patient/surgeon relationship, which may result in improved patient care and outcomes.1 During the informed consent process, the physician should explain the illness itself and the natural course of the illness, the proposed procedure, benefits and common risks (including death) of the procedure, alternatives to the procedure (including no surgery), and who will comprise the surgery team.1 In a systematic review, researchers have documented that many physicians do not meet the minimum standards when they conduct informed consent with patients.2 Physicians tend to discuss the procedure but rarely discuss alternatives, risks, and benefits of the procedure. Even when they deliver complete informed consent, they tend to overestimate patient’s comprehension of the information.2,3 Properly conducted and documented informed consent decreases litigation risks. Approximately 40% of all physicians and almost 60% of surgical subspecialists have been sued.4 Root cause analysis of the lawsuits indicates that communication failures between the physician and the patient rather than treatment failures are the provoking factors.5 Analysis of malpractice claims indicates that properly conducted and documented informed consent is associated with decreased indemnity risk.6 A recent review of gastroenterologists found that inadequate documentation was one of the primary reasons that lawsuits were awarded to patients.7 This review showed that limited documentation in the medical record was critical to provide evidence that the informed consent process had occurred.7 Surprisingly, there have been few published studies regarding interventions that improve residents’ skills in communicating and documenting informed consent. Leclercq et al5 reported that informed consent training interventions with faculty and residents were time consuming, costly, and of questionable success. A small pilot study almost a decade ago with only 8 surgical residents showed significant improvements from pre- to post-test in communication skills with standardized patients (SP) (establishing rapport, discussing the surgical diagnosis and procedure, explaining the risks and complications of the procedure, and providing appropriate feedback to the patient).8 Grossman et al9 reported that case-based instruction of ethics (including informed consent) resulted in increased confidence of residents to discuss options with patients. A more recent educational intervention included an online module, a small group discussion with faculty, and 2 SP cases. Data collected from the training included resident skill self-assessment, which significantly increased pre to post.10 However, most of the reported trainings have included only a few participants; none of them evaluated improvements in informed consent communication and documentation skills. We hypothesized that a short intervention for residents highlighting critical aspects of informed consent would improve resident performance in both communication and documentation skills.
Patients and Methods Participants Eighty-eight postgraduate year (PGY) 1 surgical residents (general, orthopedics, otorhinolaryngology, neurosurgery, urology) who matriculated at the University of Oklahoma College of Medicine during the 2009 to 2013 years were eligible to participate in the study. Approval was obtained prior to this study from the University of Oklahoma Health Sciences Center Institutional Review Board.
Standardized patient cases and training intervention Two SP cases were developed by board-certified surgeons (J.S.L. and M.A.N.), a board-certified family medicine practitioner (R.A.S.), and an attorney specializing in professional liability and healthcare risk management (H.M.). The SP cases (laparoscopic cholecystectomy and ventral herniorrhaphy) were chosen based on commonly performed surgical procedures and risks of complications for the surgery. Both cases were 15 minutes in length for the encounter followed by 10 minutes for postencounter documentation. At pretraining, approximately half of the residents were randomly assigned to the ventral herniorrhaphy case, while the other half were assigned the cholecystectomy case. The case was followed by a didactic training session (approximately 1.5 hours) with an attorney specializing in professional liability and healthcare risk management (H.M.) and the surgery residency program director (J.S.L.). This didactic training with targeted discussion covered basic tenants of malpractice with emphasis on informed consent. After the training (post-training), residents completed the second SP case. We determined direct costs for the training and the 2 cases. We also estimated the amount of time needed for the resident training. To determine the costs associated with the training, we used publically available mean estimates of a mid-career general attorney ($116/hour) (www.salary.com) and mean compensation rates for a general surgeon ($142/ hour).11 To determine the cost of the cases, we used publically available mean estimates for SPs.12 These costs were not inclusive of benefits. We also reported the amount of time required for residents to complete the training.
Instrument development and scoring After an extensive review of the literature, we noted no existing informed consent cases or checklists. Therefore, we created each SP performance checklist based on the American College of Surgeons (2008) guidelines1 for obtaining informed consent, along with experts in the field (J.S.L. and R.A.S.). Each of the items was dichotomously scored (0 5 did not discuss, 1 5 discussed) in accordance with published recommendations for creation of performance tests.13 The checklist covered areas such as nature of the illness and the natural
B.M. Thompson et al.
Informed consent training improves performance
consequences of no treatment, the proposed operation, risks and benefits, and alternative forms of treatment including nonoperative techniques. Representative items included ‘‘The clinician explained the problem with my gallbladder.’’ The post ‘‘Notes’’ were scored in the same manner using the same items (0 5 not included in note, 1 5 included in note). For this study, we were interested in whether the resident had documented each item, not whether they covered an exhaustive list of benefits, risks, alternatives, and so on. Criterion was determined in advance regarding the specific items that were given credit. To ensure consistency, SPs were trained by first having them read through the case. Then, the SPs were extensively trained using role play. We used the same SPs for the cases to ensure consistency, and used SPs who had over 1,000 hours of experience. To ensure consistency between Note ratings, a surgical educator (J.S.L.) and a medical student (J.S.) first independently rated 6 notes from the 2009 to 2010 matriculants. The raters met and reviewed all note grading. Where there were discrepancies in ratings, they came to consensus. The rater agreement for Note grading on each of the 6 post notes was 90.9%, and Kappa was .82, or almost perfect agreement according to published standards.14 A note rating guide was created by the 2 raters and was used by another investigator (B.M.T.) to grade the notes from the 2011 to 2013 matriculants.
and inferential statistics (repeated measures multivariate analysis of variance) to determine differences pre to post and between Note and Checklist scores and between matriculant cohort years using IBM SPSS 22.0 (IBM Corp, Armonk, NY). We set alpha at .05 and determined effect size (educational significance) using h2; we used recommendations of h2 5 .01 as a small effect, h2 5 .06 as a medium effect size, and h2 5 .16 and above as a large effect.15
Results Eighty of the 88 surgery residents at the University of Oklahoma College of Medicine completed both pre- and post-training for this study, for a completion rate of 90.91%. All residents were PGY1. Analysis of the Checklist summed scores from pre- to post-training indicated that at pretraining, residents discussed an average of 6.85 (standard deviation [SD] 51.16) of the 8 items and documented an average of 4.80 (SD 5 1.52) items. At post-training, they discussed an average of 7.40 (SD 5 .89) items and documented an average of 5.89 (SD 5 1.56) items (Fig. 1). Multivariate analysis of variance indicated statistically significant differences overall (P , .001). Checklist (F 5 14.73, P , .01, h2 5 .16) and Note documentation (F 5 33.31, P , .01, h2 5 .30) statistically significantly increased from pre- to post-training. We noted no differences between the 5 matriculant cohorts at pretraining for either Checklist or Note (P . .05). We examined differences between what residents discussed with the SPs as indicated by the Checklist and what they documented in the Note. At pretraining, residents discussed significantly more items (Checklist) than they
Statistics To determine differences between pre- and posttraining, we analyzed each checklist item separately and then combined items to create a summed score for the checklist rated by the SPs (‘‘Checklist’’) and the written post-note (Note). We analyzed our data using descriptive
8
7.4 6.85
7 5.89
Number of Items
6 5
3
4.8
4
Pre Training Encounter
3
Post Training Encounter
2 1 0 Note (p
