Pediatric Anesthesiology Fellows’ Perception of Quality of Attending Supervision and Medical Errors Hubert A. Benzon, MD, MPH,* John Hajduk, BS,* Gildasio De Oliveira Jr, MD, MSCI, MBA,† Santhanam Suresh, MD,* Sarah L. Nizamuddin, MD,‡ Robert McCarthy, PharmD,† and Narasimhan Jagannathan, MD* BACKGROUND: Appropriate supervision has been shown to reduce medical errors in anesthesiology residents and other trainees across various specialties. Nonetheless, supervision of pediatric anesthesiology fellows has yet to be evaluated. The main objective of this survey investigation was to evaluate supervision of pediatric anesthesiology fellows in the United States. We hypothesized that there was an indirect association between perceived quality of faculty supervision of pediatric anesthesiology fellow trainees and the frequency of medical errors reported. METHODS: A survey of pediatric fellows from 53 pediatric anesthesiology fellowship programs in the United States was performed. The primary outcome was the frequency of self-reported errors by fellows, and the primary independent variable was supervision scores. Questions also assessed barriers for effective faculty supervision. RESULTS: One hundred seventy-six pediatric anesthesiology fellows were invited to participate, and 104 (59%) responded to the survey. Nine of 103 (9%, 95% confidence interval [CI], 4%–16%) respondents reported performing procedures, on >1 occasion, for which they were not properly trained for. Thirteen of 101 (13%, 95% CI, 7%–21%) reported making >1 mistake with negative consequence to patients, and 23 of 104 (22%, 95% CI, 15%–31%) reported >1 medication error in the last year. There were no differences in median (interquartile range) supervision scores between fellows who reported >1 medication error compared to those reporting ≤1 errors (3.4 [3.0–3.7] vs 3.4 [3.1–3.7]; median difference, 0; 99% CI, −0.3 to 0.3; P = .96). Similarly, there were no differences in those who reported >1 mistake with negative patient consequences, 3.3 (3.0–3.7), compared with those who did not report mistakes with negative patient consequences (3.4 [3.3–3.7]; median difference, 0.1; 99% CI, −0.2 to 0.6; P = .35). CONCLUSIONS: We detected a high rate of self-reported medication errors in pediatric anesthesiology fellows in the United States. Interestingly, fellows’ perception of quality of faculty supervision was not associated with the frequency of reported errors. The current results with a narrow CI suggest the need to evaluate other potential factors that can be associated with the high frequency of reported errors by pediatric fellows (eg, fatigue, burnout). The identification of factors that lead to medical errors by pediatric anesthesiology fellows should be a main research priority to improve both trainee education and best practices of pediatric anesthesia. (Anesth Analg 2017;XXX:00–00)
C
linical supervision of anesthesiology trainees plays a major role in patient care and medical education. Trainees actively learn from attending physicians who closely supervise and provide consistent education and constructive feedback. Inadequate supervision may have negative consequences to trainees and to patient care.1–3 Medical errors are the third leading cause of all deaths and are commonly reported in the anesthesiology specialty.4–6 It has recently been demonstrated that there is an association between perception of resident trainee supervision in From the *Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, Illinois; †Department of Anesthesiology, Northwestern University, Chicago, Illinois; and ‡University of Chicago Medical Center, University of Chicago, Chicago, Illinois. Accepted for publication July 27, 2017.
Funding: This study was funded by the Department of Pediatric Anesthesiology, Ann & Robert H. Lurie Children’s Hospital of Chicago, Northwestern University, Chicago, Illinois. The authors declare no conflicts of interest. Reprints will not be available from the authors. Address correspondence to Hubert A. Benzon, MD, MPH, Department of Pediatric Anesthesiology, Ann & Robert H. Lurie Children’s Hospital of Chicago, 225 E Chicago Ave, PO Box 19, Chicago, IL 60611. Address e-mail to [email protected]. Copyright © 2017 International Anesthesia Research Society DOI: 10.1213/ANE.0000000000002445
anesthesiology and medical errors leading to negative consequences to patients during general anesthesiology residency.7 Pediatric anesthesiology fellows are more likely to be covering higher acuity rooms that can include a more challenging patient population such as high-risk neonates, patients with significant congenital cardiac lesions, or syndromic patients with more challenging airways.8–10 While practice patterns vary across institutions, these operating rooms likely require closer supervision from an attending physician.11 Nonetheless, quality of supervision for pediatric anesthesiology fellows has yet to be evaluated. One may argue that supervision of anesthesiology fellows may significantly differ from supervision of anesthesiology residents. The primary objective of this survey investigation was to evaluate supervision of pediatric anesthesiology fellows in the United States and medical errors reported by fellows. We hypothesized that there was an indirect association between perceived quality of faculty supervision of pediatric anesthesiology fellow trainees and the frequency of medical errors reported by the fellows.
METHODS The study was approved by the institutional review board of Ann & Robert H. Lurie Children’s Hospital of Chicago.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 1
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
Pediatric Fellow Supervision and Medical Errors
A mailing list for the 56 pediatric anesthesiology fellowship programs in the United States was obtained from the Accreditation Council for Graduate Medical Education, and contact information was compiled from the Accreditation Council for Graduate Medical Education website. Of the 56 listed programs, 53 indicated they had at least 1 fellow. An e-mail list of current pediatric anesthesiology fellows was obtained from the respective program coordinators and/ or fellowship directors, and a database (PostgreSQL version 9.4.1, PostgreSQL Global Development Group) was constructed. We followed similar methods previously used to evaluate supervision of general anesthesiology trainees.7 The survey was created using Survey Monkey software (SurveyMonkey Inc, Portland, OR). To assure confidentiality of the participants, the survey was set up to delink the responses to the respondent’s e-mail address but retained the Internet protocol address of the respondents. The software uses an internal tracking system to allow only 1 response per survey invitation and generates a list of nonresponders. The participants who did not respond to the electronic questionnaire received 3 subsequent requests to complete the survey. The questionnaire was divided into 4 parts and included 21 questions. Multiple-choice questions were used. Likert scales were used to quantify respondents’ level of agreement with a statement. The first 5 questions were designed to capture characteristics of the respondents, including age, gender, number of fellows in their class, and number of hours worked per week. The second part of the survey included all 9 questions from the de Oliveira Filho et al12 instrument specifically developed to examine anesthesiology residents’ perception of quality of faculty supervision. Each question represents a dimension of supervision. The instrument uses a 4-point Likert scale (never = 1, rarely = 2, frequently = 3, and always = 4). The supervision score was calculated as the average of the individual responses to the 9 questions. The instrument has been demonstrated to have very good internal consistency of results (Cronbach α coefficient = .93; G and φ coefficients = .93).12 The third part of the survey evaluated frequency of selfreported errors using 3 questions developed by previous investigators and used in other medical specialties but with applicable relevance to anesthesiology.13,14 Frequency was evaluated using a 5-point Likert scale (5 = often, 4 = multiple times, 3 = a couple of times, 2 = once, and 1 = never). The 3 questions pertained to the following statements: “I perform procedures for which I am not properly trained,” “I have made mistakes that have negative consequences for the patient,” and “I have made medication errors (dose or incorrect drug) in the last year.” The fourth part of the survey represented the trainee’s level of agreement to causes that contribute to poor supervision from anesthesia attendings using a 5-point Likert scale (5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree, and 1 = strongly disagree). Factors that contributed as a barrier to adequate supervision were assessed (lack of interest for teaching, lack of emphasis on supervision by departmental leadership, excessive supervisor clinical work, and lack of capability of the attending to teach as factors contributing to poor supervision). The primary outcome was the frequency of self-reported errors by the pediatric fellows. Confidence intervals (CIs)
2 www.anesthesia-analgesia.org
for the rate of reported errors were calculated using the Pearson-Klopper method. Fellow-reported supervision scores were the independent variables of interest. The primary comparison of interest was the difference in supervision scores in fellows who reported >1 errors (performing a procedure for which they were not properly trained, mistakes with negative consequences to the patient, or medication errors) with those who reported ≤1 errors in the last year. The independent variables were assessed for normality using the Shapiro-Wilk test and did not meet the criteria for a normal distribution and were compared between reported-error frequency groups using the Mann-Whitney U test. To correct for multiple comparisons of supervision scores, a P 1 occasion, and 13 of 101 (13%; 95% CI, 7%–21%) reported >1 mistake with negative consequences to patients. Twenty-three (22%; 95% CI, 15%–32%) reported >1 medication error in the last year. There were no differences in median supervision scores between fellows who reported >1 error (performing a procedure for which they were not properly trained, mistakes with negative consequences to the patient, or medication errors) with those who reported ≤1 errors in the last year (Table 2).
Secondary Outcomes Supervision scores were not correlated with the Likert response for the questions of performing procedures for which trainees were not adequately trained (ρ = .02; 95% CI, −0.19 to 0.25; P = .82), negative mistakes with consequences to patients (ρ = .03; 95% CI, −0.17 to 0.24; P = .73), and medication errors within the last year (ρ = .00; 95% CI, −0.19 to 0.19; P = .99). The most frequently cited (Likert 4 or 5) barriers to fellow supervision were a lack of interest in teaching by the faculty, 26 of 104 (25%; 95% CI, 17%–34%) respondents, and lack of ability of the anesthesia attending to teach, 20 of 104 (19%; 95% CI, 12%–18%) respondents. Supervision
scores were negatively correlated with the Likert response to the questions regarding faculty barriers for supervision (Table 3).
DISCUSSION The most important finding of the current study was the lack of association between the perceived supervision by pediatric anesthesiology fellows and reported medical errors. Furthermore, there was no association between the perceived supervision by pediatric anesthesiology fellows and reported mistakes with significant negative consequences to patient care. This is in contrast to the findings observed in anesthesiology residents where a significant association occurred between lower perceived supervision and greater incidence of medical errors with negative consequences to patients.7 The data in the current study suggest that perceived supervision reported by pediatric fellows is likely independent of the frequency of reported error occurrences when compared with the association observed in resident physicians. There are several possible explanations that may have contributed to the results of the current study. First, pediatric anesthesiology fellows are more likely to cover higher acuity patients undergoing more complex surgical and medical procedures.15,16 Second, due to anatomical and physiological differences in children and adults, it is likely that the general pediatric population undergoing surgery and anesthesia are at high risk for the development of rapid complications (eg, laryngospasm, oxygen desaturations) when compared with adults and thus may require greater active participation by the supervising attending with the fellow trainee.17,18 Finally, pediatric cases are more likely to have 1-on-1 coverage (anesthesia attending to anesthesia fellow trainee) versus 2 sites concurrently supervised by the same attending in adult anesthesia practice in the United States. Given that staffing models in the United States are complex and dependent on a number of variable factors (number of attendings and trainees, surgical procedure, acuity of patient’s medical problems, etc), our results should not be used to alter current staffing models. Since
Table 1. Characteristics of Survey Respondents and Correlation With Supervision Scores Number of Respondents (n = 104) Age (y) 25–29 30–34 35–40 >40 Gender Male Female Fellow class size 9 Weekly hours 70
Supervision Scores
2 (2) 74 (71) 20 (19) 8 (8)
3.7 3.4 3.4 3.5
(3.5–3.7) (3.0–3.7) (2.9–3.7) (3.1–3.9)
47 (45) 57 (55)
3.4 (3.0–3.7) 3.4 (3.0–3.7)
23 34 23 24
(22) (32) (22) (23)
3.6 3.3 3.4 3.3
(3.2–3.8) (3.0–2.6) (3.1–3.9) (3.0–3.7)
16 (15) 67 (64) 19 (18) 2 (3)
3.4 3.3 3.3 3.6
(3.1–3.6) (3.0–3.7) (3.0–3.7) (3.5–3.6)
P
Correlation With Supervision Score, ρ (95% CI) (−0.22 to 0.22)
.27 0.01 (−0.18 to 0.20) .91 −0.14 (−0.31 to 0.06) .14
−0.02 (−0.19 to 0.16) .74
Data presented as n (%) or median interquartile range. Abbreviation: CI, confidence interval.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 3
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
Pediatric Fellow Supervision and Medical Errors
Figure . Scatter plot of program size versus response rate.
Table 2. Association of Supervision Scores With Reported Errors Perform a procedure for which they are not properly trained Made a mistake with negative consequences to the patient Made a medication error
Reported Frequency ≤1 Time >1 Time 3.4 (3.0–3.7) 3.1 (2.9–3.8) 3.3 (3.0–3.7) 3.4 (3.3–3.7) 3.4 (3.0–3.7) 3.4 (3.1–3.7)
Difference (99% CI of Difference) −0.3 (−0.6 to 0.6) 0.1 (−0.2 to 0.6) 0 (−0.3 to 0.3)
P .83 .35 .96
Data presented as median interquartile range or median difference (99% CI of the difference). Abbreviation: CI, confidence interval.
Table 3. Correlations Between Supervision Scores and Faculty Barriers for Supervision
Lack of interest of teaching by attending Lack of emphasis on supervision by department leadership Lack of time due to excessive clinical work Lack of anesthesia attending’s capability to teach
Correlation With Supervision Scores, ρ (95% CI) −0.33 (−0.51 to −0.13)
P .001
−0.29 (−0.47 to −0.10)
.003
−0.21 (−0.39 to −0.03)
.04
−0.29 (−0.46 to −0.07)
.003
Abbreviation: CI, confidence interval.
we did not find an association between errors and supervision, future studies regarding staffing models of fellows may help determine the optimal staffing models of pediatric fellows in the United States. Another important finding of the current investigation was the percentage of fellow respondents stating they were performing procedures for which they were not properly trained. This reported percentage (9%) is greater than what has been reported in anesthesiology residents (7.5%).7 It is possible that fellow trainees have been given more independence to perform invasive procedures or that the attending anesthesiologist was unavailable. Additionally, 13% of the fellow trainees also had experienced >1 medical error with negative consequences during their fellowship, compared to 3% of the residents. Given the fragile patient population and the reduced margin of safety in children, it is conceivable that mistakes occurring at pediatric centers could lead to more severe consequences compared to adults. Few studies have addressed the role of supervision in anesthesiology training in general. Schmidt et al19 observed that close supervision of anesthesiology residents by attending anesthesiologists was associated with a reduction
4 www.anesthesia-analgesia.org
in complications during emergency airway management when performed by trainees. Survey tools have been used and validated across different specialties in anesthesiology, but none have specifically focused on pediatric fellows.20,21 To our knowledge, this is the first study evaluating the relationship between attending supervision and anesthesia training at the fellowship level in the field anesthesiology. Future studies should evaluate the importance and quality of supervision of anesthesiology trainees, particularly toward the evaluation of different fellowships (eg, critical care, pain medicine). The current study should only be interpreted within the context of its limitations. The responses were subjective and reflected the perception of supervision by anesthesiology fellows. The perception of quantity and quality of supervision is subject to opinion and variable among fellow trainees, which may have influenced the results. This study only evaluated the perception of supervision and not actual supervision (1-to-1 coverage versus 2-to-1 coverage, the amount of time the attending anesthesiologist is actually present in the room with the trainee, if the attending was present during medication administration, etc). We only examined pediatric anesthesiology fellows, and therefore, we cannot generalize our findings to different anesthesiology fellowships. Finally, it is possible that an examination of advance pediatric anesthesiology training in other countries may result in different findings. We detected a higher rate of self-reported errors in pediatric anesthesiology fellows in the United States than previously reported in anesthesiology residents. This difference may reflect the perception of supervision between the 2 populations. Interestingly, fellows’ perception of quality of faculty supervision was not associated with the frequency of reported errors. Nevertheless, the inverse associations of supervisions scores with perceived barriers to faculty supervision were similar among resident and pediatric
ANESTHESIA & ANALGESIA
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
fellow respondents. The current results suggest the need to evaluate other potential factors that can be associated with the high frequency of reported errors by pediatric fellows (eg, fatigue, burnout). The identification of root-cause factors that lead to medical errors by pediatric fellows should be a main research priority to improve both trainee education and best practices of pediatric anesthesia. E DISCLOSURES Name: Hubert A. Benzon, MD, MPH. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: John Hajduk, BS. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Gildasio De Oliveira Jr, MD, MSCI, MBA. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Santhanam Suresh, MD. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Sarah L. Nizamuddin, MD. Contribution: This author helped design the study and prepare the manuscript. Name: Robert McCarthy, PharmD. Contribution: This author helped contribute the statistical analysis and revise the manuscript. Name: Narasimhan Jagannathan, MD. Contribution: This author helped design and conduct the study, and prepare the manuscript. This manuscript was handled by: Edward C. Nemergut, MD. REFERENCES 1. Farnan JM, Petty LA, Georgitis E, et al. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Acad Med. 2012;87:428–442. 2. Baldwin DWC Jr, Daugherty SR, Ryan PM. How residents view their clinical supervision: a reanalysis of classic national survey data. J Grad Med Educ. 2010;2: 37–45 3. Sakai T, Emerick TD, Patel RM. A retrospective review of required projects in systems-based practice in a single anesthesiology residency: a 10-year experience. J Clin Anesth. 2015;27:451–456. 4. Charles R, Hood B, Derosier JM, et al. How to perform a root cause analysis for workup and future prevention of medical errors: a review. Patient Saf Surg. 2016;10:20. 5. Cobas MA, Martin ND, Barkin HB. Two lost airways and one unexpected problem: undiagnosed tracheal stenosis in a morbidly obese patient. J Clin Anesth. 2016;35:225–227. 6. Benkhadra M, Rivory JC, Wessels C, et al. Accuracy in obtaining 100 μg from 10 mg of morphine for spinal anesthesia. J Clin Anesth. 2015;27:638–645. 7. De Oliveira GS Jr, Rahmani R, Fitzgerald PC, Chang R, McCarthy RJ. The association between frequency of self-reported medical
errors and anesthesia trainee supervision: a survey of United States anesthesiology residents-in-training. Anesth Analg. 2013;116:892–897. 8. Braden A, Maani C, Nagy C. Anesthetic management of an ex utero intrapartum treatment procedure: a novel balanced approach. J Clin Anesth. 2016;31:60–63. 9. Goonasekera C, Ali K, Hickey A, et al. Mortality following congenital diaphragmatic hernia repair: the role of anesthesia. Paediatr Anaesth. 2016;26:1197–1201. 10. Rajan S, Khanna A, Argalious M, et al. Comparison of 2 resident learning tools-interactive screen-based simulated case scenarios versus problem-based learning discussions: a prospective quasi-crossover cohort study. J Clin Anesth. 2016;28:4–11. 11. Daverio M, Fino G, Luca B, et al. Failure mode and effective analysis ameliorate awareness of medical errors: a 4-year prospective observational study in critically ill children. Paediatr Anaesth. 2015;25:1227–1234. 12. de Oliveira Filho GR, Dal Mago AJ, Garcia JH, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg. 2008;107:1316–1322. 13. West CP, Huschka MM, Novotny PJ, et al. Association of perceived medical errors with resident distress and empathy: a prospective longitudinal study. JAMA. 2006;296:1071–1078. 14. Prins JT, van der Heijden FM, Hoekstra-Weebers JE, et al. Burnout, engagement and resident physicians’ self-reported errors. Psychol Health Med. 2009;14:654–666. 15. Saettele AK, Christensen JL, Chilson KL, Murray DJ. Children with heart disease: risk stratification for non-cardiac surgery. J Clin Anesth. 2016;35:479–484. 16. Bairdain S, Dodson B, Zurakowski D, Waisel DB, Jennings RW, Boretsky KR. Paravertebral nerve block catheters using chloroprocaine in infants with prolonged mechanical ventilation for treatment of long-gap esophageal atresia. Paediatr Anaesth. 2015;25:1151–1157. 17. King MR, Anderson TA, Sui J, He G, Poon KY, Coté CJ. Agerelated incidence of desaturation events and the cardiac responses on stroke index, cardiac index, and heart rate measured by continuous bioimpedance noninvasive cardiac output monitoring in infants and children undergoing general anesthesia. J Clin Anesth. 2016;32:181–188. 18. Schleelein LE, Vincent AM, Jawad AF, et al. Pediatric perioperative adverse events requiring rapid response: a retrospective case-control study. Paediatr Anaesth. 2016;26:734–741. 19. Schmidt UH, Kumwilaisak K, Bittner E, George E, Hess D. Effects of supervision by attending anesthesiologists on complications of emergency tracheal intubation. Anesthesiology. 2008;109:973–977. 20. Hindman BJ, Dexter F, Kreiter CD, Wachtel RE. Determinants, associations, and psychometric properties of resident assessments of anesthesiologist operating room supervision. Anesth Analg. 2013;116:1342–1351. 21. De Oliveira GS Jr, Dexter F, Bialek JM, McCarthy RJ. Reliability and validity of assessing subspecialty level of faculty anesthesiologists’ supervision of anesthesiology residents. Anesth Analg. 2015;120:209–213.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 5
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
C
linical supervision of anesthesiology trainees plays a major role in patient care and medical education. Trainees actively learn from attending physicians who closely supervise and provide consistent education and constructive feedback. Inadequate supervision may have negative consequences to trainees and to patient care.1–3 Medical errors are the third leading cause of all deaths and are commonly reported in the anesthesiology specialty.4–6 It has recently been demonstrated that there is an association between perception of resident trainee supervision in From the *Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, Illinois; †Department of Anesthesiology, Northwestern University, Chicago, Illinois; and ‡University of Chicago Medical Center, University of Chicago, Chicago, Illinois. Accepted for publication July 27, 2017.
Funding: This study was funded by the Department of Pediatric Anesthesiology, Ann & Robert H. Lurie Children’s Hospital of Chicago, Northwestern University, Chicago, Illinois. The authors declare no conflicts of interest. Reprints will not be available from the authors. Address correspondence to Hubert A. Benzon, MD, MPH, Department of Pediatric Anesthesiology, Ann & Robert H. Lurie Children’s Hospital of Chicago, 225 E Chicago Ave, PO Box 19, Chicago, IL 60611. Address e-mail to [email protected]. Copyright © 2017 International Anesthesia Research Society DOI: 10.1213/ANE.0000000000002445
anesthesiology and medical errors leading to negative consequences to patients during general anesthesiology residency.7 Pediatric anesthesiology fellows are more likely to be covering higher acuity rooms that can include a more challenging patient population such as high-risk neonates, patients with significant congenital cardiac lesions, or syndromic patients with more challenging airways.8–10 While practice patterns vary across institutions, these operating rooms likely require closer supervision from an attending physician.11 Nonetheless, quality of supervision for pediatric anesthesiology fellows has yet to be evaluated. One may argue that supervision of anesthesiology fellows may significantly differ from supervision of anesthesiology residents. The primary objective of this survey investigation was to evaluate supervision of pediatric anesthesiology fellows in the United States and medical errors reported by fellows. We hypothesized that there was an indirect association between perceived quality of faculty supervision of pediatric anesthesiology fellow trainees and the frequency of medical errors reported by the fellows.
METHODS The study was approved by the institutional review board of Ann & Robert H. Lurie Children’s Hospital of Chicago.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 1
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
Pediatric Fellow Supervision and Medical Errors
A mailing list for the 56 pediatric anesthesiology fellowship programs in the United States was obtained from the Accreditation Council for Graduate Medical Education, and contact information was compiled from the Accreditation Council for Graduate Medical Education website. Of the 56 listed programs, 53 indicated they had at least 1 fellow. An e-mail list of current pediatric anesthesiology fellows was obtained from the respective program coordinators and/ or fellowship directors, and a database (PostgreSQL version 9.4.1, PostgreSQL Global Development Group) was constructed. We followed similar methods previously used to evaluate supervision of general anesthesiology trainees.7 The survey was created using Survey Monkey software (SurveyMonkey Inc, Portland, OR). To assure confidentiality of the participants, the survey was set up to delink the responses to the respondent’s e-mail address but retained the Internet protocol address of the respondents. The software uses an internal tracking system to allow only 1 response per survey invitation and generates a list of nonresponders. The participants who did not respond to the electronic questionnaire received 3 subsequent requests to complete the survey. The questionnaire was divided into 4 parts and included 21 questions. Multiple-choice questions were used. Likert scales were used to quantify respondents’ level of agreement with a statement. The first 5 questions were designed to capture characteristics of the respondents, including age, gender, number of fellows in their class, and number of hours worked per week. The second part of the survey included all 9 questions from the de Oliveira Filho et al12 instrument specifically developed to examine anesthesiology residents’ perception of quality of faculty supervision. Each question represents a dimension of supervision. The instrument uses a 4-point Likert scale (never = 1, rarely = 2, frequently = 3, and always = 4). The supervision score was calculated as the average of the individual responses to the 9 questions. The instrument has been demonstrated to have very good internal consistency of results (Cronbach α coefficient = .93; G and φ coefficients = .93).12 The third part of the survey evaluated frequency of selfreported errors using 3 questions developed by previous investigators and used in other medical specialties but with applicable relevance to anesthesiology.13,14 Frequency was evaluated using a 5-point Likert scale (5 = often, 4 = multiple times, 3 = a couple of times, 2 = once, and 1 = never). The 3 questions pertained to the following statements: “I perform procedures for which I am not properly trained,” “I have made mistakes that have negative consequences for the patient,” and “I have made medication errors (dose or incorrect drug) in the last year.” The fourth part of the survey represented the trainee’s level of agreement to causes that contribute to poor supervision from anesthesia attendings using a 5-point Likert scale (5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree, and 1 = strongly disagree). Factors that contributed as a barrier to adequate supervision were assessed (lack of interest for teaching, lack of emphasis on supervision by departmental leadership, excessive supervisor clinical work, and lack of capability of the attending to teach as factors contributing to poor supervision). The primary outcome was the frequency of self-reported errors by the pediatric fellows. Confidence intervals (CIs)
2 www.anesthesia-analgesia.org
for the rate of reported errors were calculated using the Pearson-Klopper method. Fellow-reported supervision scores were the independent variables of interest. The primary comparison of interest was the difference in supervision scores in fellows who reported >1 errors (performing a procedure for which they were not properly trained, mistakes with negative consequences to the patient, or medication errors) with those who reported ≤1 errors in the last year. The independent variables were assessed for normality using the Shapiro-Wilk test and did not meet the criteria for a normal distribution and were compared between reported-error frequency groups using the Mann-Whitney U test. To correct for multiple comparisons of supervision scores, a P 1 occasion, and 13 of 101 (13%; 95% CI, 7%–21%) reported >1 mistake with negative consequences to patients. Twenty-three (22%; 95% CI, 15%–32%) reported >1 medication error in the last year. There were no differences in median supervision scores between fellows who reported >1 error (performing a procedure for which they were not properly trained, mistakes with negative consequences to the patient, or medication errors) with those who reported ≤1 errors in the last year (Table 2).
Secondary Outcomes Supervision scores were not correlated with the Likert response for the questions of performing procedures for which trainees were not adequately trained (ρ = .02; 95% CI, −0.19 to 0.25; P = .82), negative mistakes with consequences to patients (ρ = .03; 95% CI, −0.17 to 0.24; P = .73), and medication errors within the last year (ρ = .00; 95% CI, −0.19 to 0.19; P = .99). The most frequently cited (Likert 4 or 5) barriers to fellow supervision were a lack of interest in teaching by the faculty, 26 of 104 (25%; 95% CI, 17%–34%) respondents, and lack of ability of the anesthesia attending to teach, 20 of 104 (19%; 95% CI, 12%–18%) respondents. Supervision
scores were negatively correlated with the Likert response to the questions regarding faculty barriers for supervision (Table 3).
DISCUSSION The most important finding of the current study was the lack of association between the perceived supervision by pediatric anesthesiology fellows and reported medical errors. Furthermore, there was no association between the perceived supervision by pediatric anesthesiology fellows and reported mistakes with significant negative consequences to patient care. This is in contrast to the findings observed in anesthesiology residents where a significant association occurred between lower perceived supervision and greater incidence of medical errors with negative consequences to patients.7 The data in the current study suggest that perceived supervision reported by pediatric fellows is likely independent of the frequency of reported error occurrences when compared with the association observed in resident physicians. There are several possible explanations that may have contributed to the results of the current study. First, pediatric anesthesiology fellows are more likely to cover higher acuity patients undergoing more complex surgical and medical procedures.15,16 Second, due to anatomical and physiological differences in children and adults, it is likely that the general pediatric population undergoing surgery and anesthesia are at high risk for the development of rapid complications (eg, laryngospasm, oxygen desaturations) when compared with adults and thus may require greater active participation by the supervising attending with the fellow trainee.17,18 Finally, pediatric cases are more likely to have 1-on-1 coverage (anesthesia attending to anesthesia fellow trainee) versus 2 sites concurrently supervised by the same attending in adult anesthesia practice in the United States. Given that staffing models in the United States are complex and dependent on a number of variable factors (number of attendings and trainees, surgical procedure, acuity of patient’s medical problems, etc), our results should not be used to alter current staffing models. Since
Table 1. Characteristics of Survey Respondents and Correlation With Supervision Scores Number of Respondents (n = 104) Age (y) 25–29 30–34 35–40 >40 Gender Male Female Fellow class size 9 Weekly hours 70
Supervision Scores
2 (2) 74 (71) 20 (19) 8 (8)
3.7 3.4 3.4 3.5
(3.5–3.7) (3.0–3.7) (2.9–3.7) (3.1–3.9)
47 (45) 57 (55)
3.4 (3.0–3.7) 3.4 (3.0–3.7)
23 34 23 24
(22) (32) (22) (23)
3.6 3.3 3.4 3.3
(3.2–3.8) (3.0–2.6) (3.1–3.9) (3.0–3.7)
16 (15) 67 (64) 19 (18) 2 (3)
3.4 3.3 3.3 3.6
(3.1–3.6) (3.0–3.7) (3.0–3.7) (3.5–3.6)
P
Correlation With Supervision Score, ρ (95% CI) (−0.22 to 0.22)
.27 0.01 (−0.18 to 0.20) .91 −0.14 (−0.31 to 0.06) .14
−0.02 (−0.19 to 0.16) .74
Data presented as n (%) or median interquartile range. Abbreviation: CI, confidence interval.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 3
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
Pediatric Fellow Supervision and Medical Errors
Figure . Scatter plot of program size versus response rate.
Table 2. Association of Supervision Scores With Reported Errors Perform a procedure for which they are not properly trained Made a mistake with negative consequences to the patient Made a medication error
Reported Frequency ≤1 Time >1 Time 3.4 (3.0–3.7) 3.1 (2.9–3.8) 3.3 (3.0–3.7) 3.4 (3.3–3.7) 3.4 (3.0–3.7) 3.4 (3.1–3.7)
Difference (99% CI of Difference) −0.3 (−0.6 to 0.6) 0.1 (−0.2 to 0.6) 0 (−0.3 to 0.3)
P .83 .35 .96
Data presented as median interquartile range or median difference (99% CI of the difference). Abbreviation: CI, confidence interval.
Table 3. Correlations Between Supervision Scores and Faculty Barriers for Supervision
Lack of interest of teaching by attending Lack of emphasis on supervision by department leadership Lack of time due to excessive clinical work Lack of anesthesia attending’s capability to teach
Correlation With Supervision Scores, ρ (95% CI) −0.33 (−0.51 to −0.13)
P .001
−0.29 (−0.47 to −0.10)
.003
−0.21 (−0.39 to −0.03)
.04
−0.29 (−0.46 to −0.07)
.003
Abbreviation: CI, confidence interval.
we did not find an association between errors and supervision, future studies regarding staffing models of fellows may help determine the optimal staffing models of pediatric fellows in the United States. Another important finding of the current investigation was the percentage of fellow respondents stating they were performing procedures for which they were not properly trained. This reported percentage (9%) is greater than what has been reported in anesthesiology residents (7.5%).7 It is possible that fellow trainees have been given more independence to perform invasive procedures or that the attending anesthesiologist was unavailable. Additionally, 13% of the fellow trainees also had experienced >1 medical error with negative consequences during their fellowship, compared to 3% of the residents. Given the fragile patient population and the reduced margin of safety in children, it is conceivable that mistakes occurring at pediatric centers could lead to more severe consequences compared to adults. Few studies have addressed the role of supervision in anesthesiology training in general. Schmidt et al19 observed that close supervision of anesthesiology residents by attending anesthesiologists was associated with a reduction
4 www.anesthesia-analgesia.org
in complications during emergency airway management when performed by trainees. Survey tools have been used and validated across different specialties in anesthesiology, but none have specifically focused on pediatric fellows.20,21 To our knowledge, this is the first study evaluating the relationship between attending supervision and anesthesia training at the fellowship level in the field anesthesiology. Future studies should evaluate the importance and quality of supervision of anesthesiology trainees, particularly toward the evaluation of different fellowships (eg, critical care, pain medicine). The current study should only be interpreted within the context of its limitations. The responses were subjective and reflected the perception of supervision by anesthesiology fellows. The perception of quantity and quality of supervision is subject to opinion and variable among fellow trainees, which may have influenced the results. This study only evaluated the perception of supervision and not actual supervision (1-to-1 coverage versus 2-to-1 coverage, the amount of time the attending anesthesiologist is actually present in the room with the trainee, if the attending was present during medication administration, etc). We only examined pediatric anesthesiology fellows, and therefore, we cannot generalize our findings to different anesthesiology fellowships. Finally, it is possible that an examination of advance pediatric anesthesiology training in other countries may result in different findings. We detected a higher rate of self-reported errors in pediatric anesthesiology fellows in the United States than previously reported in anesthesiology residents. This difference may reflect the perception of supervision between the 2 populations. Interestingly, fellows’ perception of quality of faculty supervision was not associated with the frequency of reported errors. Nevertheless, the inverse associations of supervisions scores with perceived barriers to faculty supervision were similar among resident and pediatric
ANESTHESIA & ANALGESIA
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
fellow respondents. The current results suggest the need to evaluate other potential factors that can be associated with the high frequency of reported errors by pediatric fellows (eg, fatigue, burnout). The identification of root-cause factors that lead to medical errors by pediatric fellows should be a main research priority to improve both trainee education and best practices of pediatric anesthesia. E DISCLOSURES Name: Hubert A. Benzon, MD, MPH. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: John Hajduk, BS. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Gildasio De Oliveira Jr, MD, MSCI, MBA. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Santhanam Suresh, MD. Contribution: This author helped design and conduct the study, and prepare the manuscript. Name: Sarah L. Nizamuddin, MD. Contribution: This author helped design the study and prepare the manuscript. Name: Robert McCarthy, PharmD. Contribution: This author helped contribute the statistical analysis and revise the manuscript. Name: Narasimhan Jagannathan, MD. Contribution: This author helped design and conduct the study, and prepare the manuscript. This manuscript was handled by: Edward C. Nemergut, MD. REFERENCES 1. Farnan JM, Petty LA, Georgitis E, et al. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Acad Med. 2012;87:428–442. 2. Baldwin DWC Jr, Daugherty SR, Ryan PM. How residents view their clinical supervision: a reanalysis of classic national survey data. J Grad Med Educ. 2010;2: 37–45 3. Sakai T, Emerick TD, Patel RM. A retrospective review of required projects in systems-based practice in a single anesthesiology residency: a 10-year experience. J Clin Anesth. 2015;27:451–456. 4. Charles R, Hood B, Derosier JM, et al. How to perform a root cause analysis for workup and future prevention of medical errors: a review. Patient Saf Surg. 2016;10:20. 5. Cobas MA, Martin ND, Barkin HB. Two lost airways and one unexpected problem: undiagnosed tracheal stenosis in a morbidly obese patient. J Clin Anesth. 2016;35:225–227. 6. Benkhadra M, Rivory JC, Wessels C, et al. Accuracy in obtaining 100 μg from 10 mg of morphine for spinal anesthesia. J Clin Anesth. 2015;27:638–645. 7. De Oliveira GS Jr, Rahmani R, Fitzgerald PC, Chang R, McCarthy RJ. The association between frequency of self-reported medical
errors and anesthesia trainee supervision: a survey of United States anesthesiology residents-in-training. Anesth Analg. 2013;116:892–897. 8. Braden A, Maani C, Nagy C. Anesthetic management of an ex utero intrapartum treatment procedure: a novel balanced approach. J Clin Anesth. 2016;31:60–63. 9. Goonasekera C, Ali K, Hickey A, et al. Mortality following congenital diaphragmatic hernia repair: the role of anesthesia. Paediatr Anaesth. 2016;26:1197–1201. 10. Rajan S, Khanna A, Argalious M, et al. Comparison of 2 resident learning tools-interactive screen-based simulated case scenarios versus problem-based learning discussions: a prospective quasi-crossover cohort study. J Clin Anesth. 2016;28:4–11. 11. Daverio M, Fino G, Luca B, et al. Failure mode and effective analysis ameliorate awareness of medical errors: a 4-year prospective observational study in critically ill children. Paediatr Anaesth. 2015;25:1227–1234. 12. de Oliveira Filho GR, Dal Mago AJ, Garcia JH, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg. 2008;107:1316–1322. 13. West CP, Huschka MM, Novotny PJ, et al. Association of perceived medical errors with resident distress and empathy: a prospective longitudinal study. JAMA. 2006;296:1071–1078. 14. Prins JT, van der Heijden FM, Hoekstra-Weebers JE, et al. Burnout, engagement and resident physicians’ self-reported errors. Psychol Health Med. 2009;14:654–666. 15. Saettele AK, Christensen JL, Chilson KL, Murray DJ. Children with heart disease: risk stratification for non-cardiac surgery. J Clin Anesth. 2016;35:479–484. 16. Bairdain S, Dodson B, Zurakowski D, Waisel DB, Jennings RW, Boretsky KR. Paravertebral nerve block catheters using chloroprocaine in infants with prolonged mechanical ventilation for treatment of long-gap esophageal atresia. Paediatr Anaesth. 2015;25:1151–1157. 17. King MR, Anderson TA, Sui J, He G, Poon KY, Coté CJ. Agerelated incidence of desaturation events and the cardiac responses on stroke index, cardiac index, and heart rate measured by continuous bioimpedance noninvasive cardiac output monitoring in infants and children undergoing general anesthesia. J Clin Anesth. 2016;32:181–188. 18. Schleelein LE, Vincent AM, Jawad AF, et al. Pediatric perioperative adverse events requiring rapid response: a retrospective case-control study. Paediatr Anaesth. 2016;26:734–741. 19. Schmidt UH, Kumwilaisak K, Bittner E, George E, Hess D. Effects of supervision by attending anesthesiologists on complications of emergency tracheal intubation. Anesthesiology. 2008;109:973–977. 20. Hindman BJ, Dexter F, Kreiter CD, Wachtel RE. Determinants, associations, and psychometric properties of resident assessments of anesthesiologist operating room supervision. Anesth Analg. 2013;116:1342–1351. 21. De Oliveira GS Jr, Dexter F, Bialek JM, McCarthy RJ. Reliability and validity of assessing subspecialty level of faculty anesthesiologists’ supervision of anesthesiology residents. Anesth Analg. 2015;120:209–213.
XXX 2017 • Volume XXX • Number XXX www.anesthesia-analgesia.org 5
Copyright © 2017 International Anesthesia Research Society. Unauthorized reproduction of this article is prohibited.
