COMMENTARY
ON
MEDICAL EDUCATION
Trends in the History of Certification and Recertification of the American Board of Internal Medicine Michael T. Flannery, MD
Abstract: This commentary reviews the trends of pass rates for certification and recertification in internal medicine. This is true for certification since the 1930s and recertification since 2000. Predictors of performance, such as program director ratings and the in-training examination, are discussed in addition to positive clinical outcomes in relation to recertification. Differences in examinations pass rates due to gender, geographic location and number of attempts are reviewed. Recent trends in internal medicine demonstrate a decline, which may be multifactorial in reasoning. This is not unique to internal medicine as declines in certification rates have been noted in general surgery as well. Methods of preparing for the examination are discussed to maximize performance on the examination. Key Indexing Terms: American Board of Internal Medicine; Certification; Recertification; Pass rates. [Am J Med Sci 2014;347(1):74–77.]
T
TRENDS
he American Board of Internal Medicine (ABIM) 1st began issuing certificates to candidates in 1936. By the end of the 1st decade, 2,197 certificates had been issued. One of the 1st reviews, on initial certification over the 1st 30 years, compared the differences between the examinations given in the 1950s versus the 1980s. In the early years, the examinations were considered voluntary personal achievements and less than 50% of internists took the examination.1 The other major change was that certification before 1976 required both written and oral components. The written examination in the 1950’s was described as approximately 150 questions with 32 patient management problems.1 By 1982, the written examination occurred over 2 days and had approximately 200 questions with several patient management problems. In 1952, 56% of the 1sttime takers passed the written and the prevailing pass rate on the orals for the 1st-time takers was 70% meaning that only 40% became certified on their 1st try.1 In 1982, 82% of the 1st-time takers who were U.S./Canadian candidates passed on their 1st try.1 The overall passing curves were quite similar due to the inclusion of the poor performance by U.S. and non-U.S. international medical graduates (IMGs). In another study, looking at certification examination rates between 1980 and 1985, pass rates correlated with program directors (PDs) ratings on ABIM tracking. ABIM ratings by the PDs strongly correlated with their subsequent passage of the certification examination.2 Findings ranged an average of 46% passing for those receiving a rating of 4% to 93% passing for those candidates receiving a rating of 9.2 In another study, the pass rates for different PDs rating levels roughly remained the same or dropped slightly during the 5-year period.3 There was From the Department of Internal Medicine, Morsani College of Medicine, University of South Florida, Tampa, Florida. The author has no financial or other conflicts of interest to disclose. Correspondence: Michael T. Flannery, MD, Department of Medicine, Morsani College of Medicine, University of South Florida, 12901 Bruce B Downs Boulevard, MDC Box 19, Room L1041, Tampa, FL 33612 (E-mail: mfl[email protected]).
74
also a review from 1983 to 1988 focusing on medical knowledge for certification, which showed a slight decrease in the performance of candidates from the United States during that time frame.4 When globally looking at the overall score scale, the decreases were roughly 1 to 3 question items on the examination.4 This occurred at a time when interest in internal medicine had declined by 37%.5 It is difficult to blame these slight changes on the change of interest in internal medicine. Again, this is especially true given the fact that the raw score difference between any 2 years was roughly 1 to 3 common items, which are very small. Also, the PD ratings from the ABIM, during that time frame, on clinical competence, had been increasing, suggesting that while medical students choosing internal medicine as a career could be getting worse at test taking but better at patient care. Applicants entering the certification examination in 1982 or 1983 were tracked in the ABIM database for 5 years. Ultimately, 85% of the residents were certified in internal medicine; the pass rates were 87% for men, 81% for women, 92% for U.S. medical graduates and 60% for IMGs.6 Rates increased minimally after the 2nd attempt. Most residents, 87% passed on their 1st attempt of the certifying examination in the year in which their training had been completed. Delaying the examination, the 1st year was associated with lower pass rates. The other 13% who waited to take the examination had a pass rate of 63%.6 Half of the candidates who had passed the internal medicine examination attempted subspecialty certification. At the end of the tracking period, over the 9 subspecialty examinations, the pass rate was 87%.6 The certification examination pass rates ranged from 81% in 1982 to nearly 83% in 1985 for an average over that time frame of 81.5% passing.3 Another review examined the certification pass rates in the early 1990s and found a decrease in the pass rate of 1st-time takers from 70% in 1989 to 63% in 1992.7 The reduction was seen regardless of the type of training program or the level of PD ratings. In addition, it was suggested that internal medicine may be accepting less able students to fill training spots. The number of 1st-year postgraduates who were IMGs more than doubled during this time frame from 11% to 21%.7 Likewise, the number of osteopathic medical graduates opting for allopathic training in internal medicine also increased. IMGs had a 12% decrease in their pass rate between 1991 and 1992. The number of osteopathic medical graduates, taking the examination, increased by 50% from 1989 to 1992.7 While their numbers were still small, their performance remained relatively constant.
PREDICTORS OF PERFORMANCE On reviewing the 3rd-year internal medicine residents by the type of medical school as reported by the ABIM, by 2011, the percentages of U.S. medical graduates and IMGs were roughly the same around 47%. The remaining 6% represented the osteopathic group, which held steady at 6% since 2002.8 One study demonstrated that internal medicine programs with a higher proportion of IMGs were associated with a shorter certification by the Residency Review Committee for Internal Medicine. In
The American Journal of the Medical Sciences
Volume 347, Number 1, January 2014
Changes in ABIM Pass Rates
2011, the higher proportion of IMGs was associated with 1.17 years shorter of Residency Review Committee for Internal Medicine accreditation.9 There are very little published data on initial certification pass rates for the ABIM in the 1990s. One study looking at the ABIM certification found that in 1994, 72% of U.S. medical graduates and 56% of IMGs passed the boards on the 1st attempt. From 1991 to 1994, the number of IMGs taking the examination for the 1st time increased from 31.3% to 44.4%.10 Selecting the examination data, assessing physician performance and profiling physicians became more important during this time frame as recertification examinations were to start in the year 2000. Another area reviewed in the early 1990s was that of the pass rates from a gender perspective. In the 1970s and 1980s, men outscored women, but by 1993, the pass rates for women and men taking the initial internal medicine examination differed by less than 1%.11 The thought was that similar performances on the National Board of Medical Examiners, part II examination and its closer timing to the certification examination were part of the reasons for the improved pass rates for women compared with men in internal medicine. Additional data examination reviewed the performance on the in-training examination (ITE) in relation to the initial certification pass rates for internal medicine. Results from that study demonstrated a statistically significant prediction of the ABIM pass or fail performance based on ITE scores.12 Therefore, the ITE was thought to be a useful tool in assessing the likelihood of a resident passing or failing the ABIM certification. This was particularly true at the extreme low or high values but more difficult in the mid level (ITE) score performance.12 In addition, comparison was made on those who completed the maintenance of certification (MOC) for their subspecialty and how many of those who maintain their certification in core internal medicine. This is particularly focusing on a group that certified in a subspecialty between 1990 and 2000. It was published by the ABIM in February 2012.13 Programs less likely to maintain their core internal medicine certification after subspecialty certification were cardiac electrophysiology, interventional cardiology and critical care medicine. The top 3 programs most likely to continue core certification in addition to specialty certification were geriatrics, pathology and general internal medicine. Between 2000 and 2005, the pass rate gradually increased from 86% to 92%. Between 2003 and 2005, the pass rate for 1st-time takers on the initial certification peaked in the year 2007 at 94%, again, with roughly 7,100 candidates. However, over the next 5 years, the pass rate dropped from 94% to 84%, reflecting a 2% reduction per year over the past 5-year period (Table 1).14 There are many theories of why the current pass rates on the initial certification have returned to pre-2000 levels. One of the most prevailing considerations was the importance of board certification in the 1990s to obtain a job in the candidate’s area of interest. One theory was that when looking at other primary care areas, students might turn to family medicine, which they thought had a mid-90% pass rate for many years. In reality, according to the American Board of Family Medicine, the pass rate for initial certification was in the mid-80% range between 2004 and 2012.15 This would be attractive for students interested in primary care who considered board performance as an important feature for job attainment. However, over the past 10 years, looking at the National Resident Matching Program data, interest in family medicine has leveled off with a narrowing between positions offered and filled.16 Therefore, there may be less concern about the overall passing percentage for the internal medicine certification examination. Students will still want to specialize through an internal medicine pathway. Ó 2013 Lippincott Williams & Wilkins
TABLE 1. First time pass rates for the American Board of Internal Medicine over various points of time Year Pass rate (%)a 1952b 1982 1989 1994 2000 2005 2007 2011 2012 a b
40 82 70 72 86 92 94 84 85
Pass rate on 1st attempt. Oral and written examinations required.
As many PDs know, to avoid an American College Graduate Medical Education citation, they must maintain a 3-year rolling pass rate of 80% or higher on the certification examination for candidates from their program. For our program, by the late 1990s, we were slightly below the national pass rate in internal medicine with an overall pass rate of 68%. To solve that problem, we initiated Board Review upon which data had already shown improvement in board scores in residents who participated in reviews at their program.17 While clearly attending a board review course may be beneficial, it is unknown how many programs have board reviews or how many of their residents take advantage of such. Subsequently, our program peaked similar to the national level to a 94% threeyear rolling pass rate in 2007. In our own program, the decline over the past 5 years has mirrored that of the national decline with a 10% decrease in our 3-year rolling pass rate. Looking at the ABIM data on the number of programs and percent passing, there has been a marked change as well over the past 10 years. Ten years ago, there were roughly 250 programs that had a 91% to 100% 3-year rolling pass rate with another near 150 with an 81% to 90% 3-year rolling pass rate. Now, roughly 175 programs have a 91% to 100% pass rate with 150 programs having an 81% to 90% pass rate. There has been a slight increase in the number of programs with a 3-year rolling pass rates between 71% and 80%, which would then generate a citation from the American College Graduate Medical Education.13
DIFFERENCES IN PASS RATES There is quite a difference looking at subspecialty certification 1st-time taker pass rates over the past 10 years. Initially in 2000, the range was a 68% pass rate in cardiac electrophysiology to a high of 89% in pulmonary medicine. By 2011, the pass rate was 77% in clinical cardiac physiology and 93% in pulmonary medicine.16 This was true for most areas in the subspecialties for 2011, the pass rate range was 65% in sleep medicine to 96% in infectious disease. Therefore, whereas for initial certification of core programs, the pass rates have gone down by approximately 10 percentage points in the past 5 years, in the 1st 10 years of recertification, the percentage points went up from 5 to 10 for most subspecialties.16 One might speculate that the candidates do not underestimate the recertification process and are better prepared for the examination compared with those who took the examination in 2000. Similar to internal medicine, general surgery has seen a 7% decrease in examination pass rates from 2006 to 2011. This also varied geographically with Puerto Rico and the southern states underperforming
75
Flannery
compared with pacific programs.18 Core internal medicine had a pass rate of 83% in 2007 and then 88% in 2011 for 1st-time takers. The 88% remains the pass rate for the 1st attempt, and the ultimate pass rate, up to 3 subsequent examinations, is 96%.16 Some of those candidates may not have needed 3 examinations to ultimately pass the recertification examination for internal medicine. There are many questions why physicians needed to go through the MOC, after 10 years, beginning in 1990. In 1 study, the American Board of Medical Specialties demonstrated that approximately 23% of general internist and 14% of subspecialist did not renew their certification.17 The complaint focused on the amount of time, money and a lack of useful need for employment. Other studies point out quality benefits before the MOC, including better outcomes,19 15% low mortality in myocardial infarction,20 17% higher rates of preventative services,21 40% lower colon resection mortality22 and 20% lower birth rate babies.23 There are a number of other quality studies associated with MOC and studies that demonstrate that the public expects MOC.24,25 In addition, studies have shown that physicians value the MOC process.26,27 There are a number of components for MOC, including practice improvement modules, the examination, and supervisory support letters. Obviously, the goal is to maintain and improve the quality of patient care and obtain increasing support by a number of organizations and physicians themselves.28 To support that concept, the number of candidates issued certificates in core medicine and the various subspecialties have remained relatively stable over the past decade.29 Of course, some physicians do not take the MOC examination. A study of U.S. internists demonstrated that although 91% of applicants were still working in the field, less so for general medicine (79%), approximately 1/2 report being required to recertify by their employer. However, after completion of the process, only 1/3 report this requirement by their employer. The most common reasons for noncompliance were too much time, expense and not required for employment.17 In building an evidence base support for recertification, there must be an appropriate focus on learning and improvement activities while building the necessary body of evidence to demonstrate the value of recertification.29
METHODS TO IMPROVE PREPARATION The keys to board certification have remained fairly constant over the years. Use all opportunities during residency to prepare morning report, grand rounds, noon conference didactics, patient loads, journal club, morbidity/mortality, simulation experiences. This author recommends that residents spend 2 hours per day for 100 days before the examination leaving the last day for rest. Use a program’s internal board review course and cross reference that learning with questions and material from an external reliable board review course. Examination content areas typically complained about year to year, from our own residents over many years, include gynecology/obstetrics, ophthalmology, vaccinations, urinary incontinence/geriatrics and statistics. Applicants should spend extra time in their areas of weakness and read the material provided on test taking skills. Given this amount of learning devotion, applicant confidence increases and such may improve examination score rates. The ABIM also provides a blueprint of the examination, which reviews the numbers and types of questions per content area, which is invaluable for review preparation but is likely rarely used. Residents and fellows in the various disciplines should review this content and recent trends on examination score pass rates to maximize their own performance.30
76
REFERENCES 1. Wallerstein RO. ABIM certification revisited—what a difference 30 years make. J Trans Am Clin Climatol Assoc 1983;94:182–8. 2. Norcini JJ, Webster GD, Grosso LJ, et al. Ratings of residents’ clinical competence and performance on certification examination. J Med Educ 1987;62:457–62. 3. Shea JA, Norcini JJ, Kimball HR. Relationships of ratings of clinical competence and ABIM scores to certification status. Acad Med 1993;68 (suppl 10):S22–4. 4. Norcini JJ, Shea JA, Benson JA. Changes in the medical knowledge of candidates for certification. Ann Intern Med 1991;114:33–5. 5. Norcini JJ, Maihoff NA, Day SC, et al. Trends in medical knowledge as assessed by the certifying examination in internal medicine. JAMA 1989;262:2402–4. 6. Shea JA, Norcini JJ, Day SC, et al. A longitudinal description of patterns of certification in internal medicine and the subspecialties. J Gen Intern Med 1991;6:553–7. 7. Norcini JJ, Kimball HR, Grosso LJ, et al. Certification in internal medicine: 1989-1992. J Gen Intern Med 1994;9:361–5. 8. American Board of Internal Medicine (ABIM). Figure 4—percentage of third year internal medicine residents by type of medical school attended. 2004–2013. Available at: http://www.abim.org. Accessed June 2013. 9. Sisson SD, Casagrande SS, Dalal D, et al. Associations between quality indicators of internal medicine residency training programs. BMC Med Educ 2011;11:30. 10. Kimball HR, Benson JA. The ABIM perspective: past, present, future. Internist 1995;36:11–4. 11. Norcini JJ. Examining the examinations for licensure and certification in medicine. JAMA 1994;272:713–4. 12. Rollins LK, Martindale JR, Edmond M, et al. Predicting pass rates on the American board of internal medicine certifying examination. J Gen Intern Med 1998;13:414–6. 13. American Board of Internal Medicine (ABIM). Maintenance of certification program completion rates. 2012. Available at: http://www.abim.org. Accessed June 2013. 14. American Board of Internal Medicine (ABIM). First-time taker pass rates—initial certification. 2007–2011. Available at: http://www.abim.org. Accessed June 2013. 15. American Board of Internal Medicine (ABIM). ACGME accredited programs with at least ten first-time takers over the three-year period. 2012. Available at: http://www.abim.org. Accessed June 2013. 16. American Board of Internal Medicine (ABIM). First-time taker pass rates-maintenance of certification. 1997–2011. Available at: http://www.abim.org. Accessed June 2013. 17. Lipner RS, Bylsma WH, Arnold GK, et al. Who is maintaining certification in internal medicine and why? A national survey 10 years after initial certification. Ann Intern Med 2006;144:29–36. 18. Falcone JL, Hamad GG. The American Board of Surgery Certifying Examination: a retrospective study of the decreasing pass rates and performance for first-time examinees. J Surg Educ 2012;69:231–5. 19. Brennan TA, Horwitz RI, Duffy FD, et al. The role of physician specialty board certification status in the quality movement. JAMA 2004;292:1038–43. 20. Norcini JJ, Kimball HR, Lipner RS. Certification and specialization: do they matter in the outcome of acute myocardial infarction? Acad Med 2000;75:1193–8. 21. Hoangmai PH, Schrag D, Hargraves JL, et al. Delivery of preventive services to older adults by primary care physicians. JAMA 2005;294:473–81.
Volume 347, Number 1, January 2014
Changes in ABIM Pass Rates
22. Prystowsky JB, Bordage G, Feinglass JM. Patient outcomes for segmental colon resection according to surgeon’s training, certification, and experience. Surgery 2002;132:663–70. 23. Haas JS, Orav EJ, Goldman L. The relationship between physicians’ qualifications and experience and the adequacy of prenatal care and low birthweight. Am J Public Health 1995;85:1087–91. 24. American Board of Medical Specialties. Facts about the ABMS consumer survey; lifelong learning and other qualities in choosing a doctor [PDF]. Chicago, IL: ABMS; 2011. 25. American Board of Medical Specialties. Facts about the ABMS consumer survey: how Americans choose their doctor [PDF]. Chicago, IL: ABMS; 2010.
Ó 2013 Lippincott Williams & Wilkins
26. ABIM MOC program survey results. 2008–2011. Available at: http://www.abim.org/. Accessed October 2013. 27. Dale DC. Recertification in internal medicine—the American experience. Ann Acad Med Singapore 2007;36:894–7. 28. American Board of Internal Medicine (ABIM). Number of candidates certified annually by the American board of internal medicine. 1936–2011. Available at: http://www.abim.org. Accessed June 2013. 29. Hawkins RE, Weiss KB. Building the evidence base in support of the American board of medical specialties maintenance of certification program. Acad Med 2011;86:6–7. 30. Palmer I. Strategies to pass the internal medicine board exam. Philadelphia, PA: ACP-ASIM Observer; 2000.
77
ON
MEDICAL EDUCATION
Trends in the History of Certification and Recertification of the American Board of Internal Medicine Michael T. Flannery, MD
Abstract: This commentary reviews the trends of pass rates for certification and recertification in internal medicine. This is true for certification since the 1930s and recertification since 2000. Predictors of performance, such as program director ratings and the in-training examination, are discussed in addition to positive clinical outcomes in relation to recertification. Differences in examinations pass rates due to gender, geographic location and number of attempts are reviewed. Recent trends in internal medicine demonstrate a decline, which may be multifactorial in reasoning. This is not unique to internal medicine as declines in certification rates have been noted in general surgery as well. Methods of preparing for the examination are discussed to maximize performance on the examination. Key Indexing Terms: American Board of Internal Medicine; Certification; Recertification; Pass rates. [Am J Med Sci 2014;347(1):74–77.]
T
TRENDS
he American Board of Internal Medicine (ABIM) 1st began issuing certificates to candidates in 1936. By the end of the 1st decade, 2,197 certificates had been issued. One of the 1st reviews, on initial certification over the 1st 30 years, compared the differences between the examinations given in the 1950s versus the 1980s. In the early years, the examinations were considered voluntary personal achievements and less than 50% of internists took the examination.1 The other major change was that certification before 1976 required both written and oral components. The written examination in the 1950’s was described as approximately 150 questions with 32 patient management problems.1 By 1982, the written examination occurred over 2 days and had approximately 200 questions with several patient management problems. In 1952, 56% of the 1sttime takers passed the written and the prevailing pass rate on the orals for the 1st-time takers was 70% meaning that only 40% became certified on their 1st try.1 In 1982, 82% of the 1st-time takers who were U.S./Canadian candidates passed on their 1st try.1 The overall passing curves were quite similar due to the inclusion of the poor performance by U.S. and non-U.S. international medical graduates (IMGs). In another study, looking at certification examination rates between 1980 and 1985, pass rates correlated with program directors (PDs) ratings on ABIM tracking. ABIM ratings by the PDs strongly correlated with their subsequent passage of the certification examination.2 Findings ranged an average of 46% passing for those receiving a rating of 4% to 93% passing for those candidates receiving a rating of 9.2 In another study, the pass rates for different PDs rating levels roughly remained the same or dropped slightly during the 5-year period.3 There was From the Department of Internal Medicine, Morsani College of Medicine, University of South Florida, Tampa, Florida. The author has no financial or other conflicts of interest to disclose. Correspondence: Michael T. Flannery, MD, Department of Medicine, Morsani College of Medicine, University of South Florida, 12901 Bruce B Downs Boulevard, MDC Box 19, Room L1041, Tampa, FL 33612 (E-mail: mfl[email protected]).
74
also a review from 1983 to 1988 focusing on medical knowledge for certification, which showed a slight decrease in the performance of candidates from the United States during that time frame.4 When globally looking at the overall score scale, the decreases were roughly 1 to 3 question items on the examination.4 This occurred at a time when interest in internal medicine had declined by 37%.5 It is difficult to blame these slight changes on the change of interest in internal medicine. Again, this is especially true given the fact that the raw score difference between any 2 years was roughly 1 to 3 common items, which are very small. Also, the PD ratings from the ABIM, during that time frame, on clinical competence, had been increasing, suggesting that while medical students choosing internal medicine as a career could be getting worse at test taking but better at patient care. Applicants entering the certification examination in 1982 or 1983 were tracked in the ABIM database for 5 years. Ultimately, 85% of the residents were certified in internal medicine; the pass rates were 87% for men, 81% for women, 92% for U.S. medical graduates and 60% for IMGs.6 Rates increased minimally after the 2nd attempt. Most residents, 87% passed on their 1st attempt of the certifying examination in the year in which their training had been completed. Delaying the examination, the 1st year was associated with lower pass rates. The other 13% who waited to take the examination had a pass rate of 63%.6 Half of the candidates who had passed the internal medicine examination attempted subspecialty certification. At the end of the tracking period, over the 9 subspecialty examinations, the pass rate was 87%.6 The certification examination pass rates ranged from 81% in 1982 to nearly 83% in 1985 for an average over that time frame of 81.5% passing.3 Another review examined the certification pass rates in the early 1990s and found a decrease in the pass rate of 1st-time takers from 70% in 1989 to 63% in 1992.7 The reduction was seen regardless of the type of training program or the level of PD ratings. In addition, it was suggested that internal medicine may be accepting less able students to fill training spots. The number of 1st-year postgraduates who were IMGs more than doubled during this time frame from 11% to 21%.7 Likewise, the number of osteopathic medical graduates opting for allopathic training in internal medicine also increased. IMGs had a 12% decrease in their pass rate between 1991 and 1992. The number of osteopathic medical graduates, taking the examination, increased by 50% from 1989 to 1992.7 While their numbers were still small, their performance remained relatively constant.
PREDICTORS OF PERFORMANCE On reviewing the 3rd-year internal medicine residents by the type of medical school as reported by the ABIM, by 2011, the percentages of U.S. medical graduates and IMGs were roughly the same around 47%. The remaining 6% represented the osteopathic group, which held steady at 6% since 2002.8 One study demonstrated that internal medicine programs with a higher proportion of IMGs were associated with a shorter certification by the Residency Review Committee for Internal Medicine. In
The American Journal of the Medical Sciences
Volume 347, Number 1, January 2014
Changes in ABIM Pass Rates
2011, the higher proportion of IMGs was associated with 1.17 years shorter of Residency Review Committee for Internal Medicine accreditation.9 There are very little published data on initial certification pass rates for the ABIM in the 1990s. One study looking at the ABIM certification found that in 1994, 72% of U.S. medical graduates and 56% of IMGs passed the boards on the 1st attempt. From 1991 to 1994, the number of IMGs taking the examination for the 1st time increased from 31.3% to 44.4%.10 Selecting the examination data, assessing physician performance and profiling physicians became more important during this time frame as recertification examinations were to start in the year 2000. Another area reviewed in the early 1990s was that of the pass rates from a gender perspective. In the 1970s and 1980s, men outscored women, but by 1993, the pass rates for women and men taking the initial internal medicine examination differed by less than 1%.11 The thought was that similar performances on the National Board of Medical Examiners, part II examination and its closer timing to the certification examination were part of the reasons for the improved pass rates for women compared with men in internal medicine. Additional data examination reviewed the performance on the in-training examination (ITE) in relation to the initial certification pass rates for internal medicine. Results from that study demonstrated a statistically significant prediction of the ABIM pass or fail performance based on ITE scores.12 Therefore, the ITE was thought to be a useful tool in assessing the likelihood of a resident passing or failing the ABIM certification. This was particularly true at the extreme low or high values but more difficult in the mid level (ITE) score performance.12 In addition, comparison was made on those who completed the maintenance of certification (MOC) for their subspecialty and how many of those who maintain their certification in core internal medicine. This is particularly focusing on a group that certified in a subspecialty between 1990 and 2000. It was published by the ABIM in February 2012.13 Programs less likely to maintain their core internal medicine certification after subspecialty certification were cardiac electrophysiology, interventional cardiology and critical care medicine. The top 3 programs most likely to continue core certification in addition to specialty certification were geriatrics, pathology and general internal medicine. Between 2000 and 2005, the pass rate gradually increased from 86% to 92%. Between 2003 and 2005, the pass rate for 1st-time takers on the initial certification peaked in the year 2007 at 94%, again, with roughly 7,100 candidates. However, over the next 5 years, the pass rate dropped from 94% to 84%, reflecting a 2% reduction per year over the past 5-year period (Table 1).14 There are many theories of why the current pass rates on the initial certification have returned to pre-2000 levels. One of the most prevailing considerations was the importance of board certification in the 1990s to obtain a job in the candidate’s area of interest. One theory was that when looking at other primary care areas, students might turn to family medicine, which they thought had a mid-90% pass rate for many years. In reality, according to the American Board of Family Medicine, the pass rate for initial certification was in the mid-80% range between 2004 and 2012.15 This would be attractive for students interested in primary care who considered board performance as an important feature for job attainment. However, over the past 10 years, looking at the National Resident Matching Program data, interest in family medicine has leveled off with a narrowing between positions offered and filled.16 Therefore, there may be less concern about the overall passing percentage for the internal medicine certification examination. Students will still want to specialize through an internal medicine pathway. Ó 2013 Lippincott Williams & Wilkins
TABLE 1. First time pass rates for the American Board of Internal Medicine over various points of time Year Pass rate (%)a 1952b 1982 1989 1994 2000 2005 2007 2011 2012 a b
40 82 70 72 86 92 94 84 85
Pass rate on 1st attempt. Oral and written examinations required.
As many PDs know, to avoid an American College Graduate Medical Education citation, they must maintain a 3-year rolling pass rate of 80% or higher on the certification examination for candidates from their program. For our program, by the late 1990s, we were slightly below the national pass rate in internal medicine with an overall pass rate of 68%. To solve that problem, we initiated Board Review upon which data had already shown improvement in board scores in residents who participated in reviews at their program.17 While clearly attending a board review course may be beneficial, it is unknown how many programs have board reviews or how many of their residents take advantage of such. Subsequently, our program peaked similar to the national level to a 94% threeyear rolling pass rate in 2007. In our own program, the decline over the past 5 years has mirrored that of the national decline with a 10% decrease in our 3-year rolling pass rate. Looking at the ABIM data on the number of programs and percent passing, there has been a marked change as well over the past 10 years. Ten years ago, there were roughly 250 programs that had a 91% to 100% 3-year rolling pass rate with another near 150 with an 81% to 90% 3-year rolling pass rate. Now, roughly 175 programs have a 91% to 100% pass rate with 150 programs having an 81% to 90% pass rate. There has been a slight increase in the number of programs with a 3-year rolling pass rates between 71% and 80%, which would then generate a citation from the American College Graduate Medical Education.13
DIFFERENCES IN PASS RATES There is quite a difference looking at subspecialty certification 1st-time taker pass rates over the past 10 years. Initially in 2000, the range was a 68% pass rate in cardiac electrophysiology to a high of 89% in pulmonary medicine. By 2011, the pass rate was 77% in clinical cardiac physiology and 93% in pulmonary medicine.16 This was true for most areas in the subspecialties for 2011, the pass rate range was 65% in sleep medicine to 96% in infectious disease. Therefore, whereas for initial certification of core programs, the pass rates have gone down by approximately 10 percentage points in the past 5 years, in the 1st 10 years of recertification, the percentage points went up from 5 to 10 for most subspecialties.16 One might speculate that the candidates do not underestimate the recertification process and are better prepared for the examination compared with those who took the examination in 2000. Similar to internal medicine, general surgery has seen a 7% decrease in examination pass rates from 2006 to 2011. This also varied geographically with Puerto Rico and the southern states underperforming
75
Flannery
compared with pacific programs.18 Core internal medicine had a pass rate of 83% in 2007 and then 88% in 2011 for 1st-time takers. The 88% remains the pass rate for the 1st attempt, and the ultimate pass rate, up to 3 subsequent examinations, is 96%.16 Some of those candidates may not have needed 3 examinations to ultimately pass the recertification examination for internal medicine. There are many questions why physicians needed to go through the MOC, after 10 years, beginning in 1990. In 1 study, the American Board of Medical Specialties demonstrated that approximately 23% of general internist and 14% of subspecialist did not renew their certification.17 The complaint focused on the amount of time, money and a lack of useful need for employment. Other studies point out quality benefits before the MOC, including better outcomes,19 15% low mortality in myocardial infarction,20 17% higher rates of preventative services,21 40% lower colon resection mortality22 and 20% lower birth rate babies.23 There are a number of other quality studies associated with MOC and studies that demonstrate that the public expects MOC.24,25 In addition, studies have shown that physicians value the MOC process.26,27 There are a number of components for MOC, including practice improvement modules, the examination, and supervisory support letters. Obviously, the goal is to maintain and improve the quality of patient care and obtain increasing support by a number of organizations and physicians themselves.28 To support that concept, the number of candidates issued certificates in core medicine and the various subspecialties have remained relatively stable over the past decade.29 Of course, some physicians do not take the MOC examination. A study of U.S. internists demonstrated that although 91% of applicants were still working in the field, less so for general medicine (79%), approximately 1/2 report being required to recertify by their employer. However, after completion of the process, only 1/3 report this requirement by their employer. The most common reasons for noncompliance were too much time, expense and not required for employment.17 In building an evidence base support for recertification, there must be an appropriate focus on learning and improvement activities while building the necessary body of evidence to demonstrate the value of recertification.29
METHODS TO IMPROVE PREPARATION The keys to board certification have remained fairly constant over the years. Use all opportunities during residency to prepare morning report, grand rounds, noon conference didactics, patient loads, journal club, morbidity/mortality, simulation experiences. This author recommends that residents spend 2 hours per day for 100 days before the examination leaving the last day for rest. Use a program’s internal board review course and cross reference that learning with questions and material from an external reliable board review course. Examination content areas typically complained about year to year, from our own residents over many years, include gynecology/obstetrics, ophthalmology, vaccinations, urinary incontinence/geriatrics and statistics. Applicants should spend extra time in their areas of weakness and read the material provided on test taking skills. Given this amount of learning devotion, applicant confidence increases and such may improve examination score rates. The ABIM also provides a blueprint of the examination, which reviews the numbers and types of questions per content area, which is invaluable for review preparation but is likely rarely used. Residents and fellows in the various disciplines should review this content and recent trends on examination score pass rates to maximize their own performance.30
76
REFERENCES 1. Wallerstein RO. ABIM certification revisited—what a difference 30 years make. J Trans Am Clin Climatol Assoc 1983;94:182–8. 2. Norcini JJ, Webster GD, Grosso LJ, et al. Ratings of residents’ clinical competence and performance on certification examination. J Med Educ 1987;62:457–62. 3. Shea JA, Norcini JJ, Kimball HR. Relationships of ratings of clinical competence and ABIM scores to certification status. Acad Med 1993;68 (suppl 10):S22–4. 4. Norcini JJ, Shea JA, Benson JA. Changes in the medical knowledge of candidates for certification. Ann Intern Med 1991;114:33–5. 5. Norcini JJ, Maihoff NA, Day SC, et al. Trends in medical knowledge as assessed by the certifying examination in internal medicine. JAMA 1989;262:2402–4. 6. Shea JA, Norcini JJ, Day SC, et al. A longitudinal description of patterns of certification in internal medicine and the subspecialties. J Gen Intern Med 1991;6:553–7. 7. Norcini JJ, Kimball HR, Grosso LJ, et al. Certification in internal medicine: 1989-1992. J Gen Intern Med 1994;9:361–5. 8. American Board of Internal Medicine (ABIM). Figure 4—percentage of third year internal medicine residents by type of medical school attended. 2004–2013. Available at: http://www.abim.org. Accessed June 2013. 9. Sisson SD, Casagrande SS, Dalal D, et al. Associations between quality indicators of internal medicine residency training programs. BMC Med Educ 2011;11:30. 10. Kimball HR, Benson JA. The ABIM perspective: past, present, future. Internist 1995;36:11–4. 11. Norcini JJ. Examining the examinations for licensure and certification in medicine. JAMA 1994;272:713–4. 12. Rollins LK, Martindale JR, Edmond M, et al. Predicting pass rates on the American board of internal medicine certifying examination. J Gen Intern Med 1998;13:414–6. 13. American Board of Internal Medicine (ABIM). Maintenance of certification program completion rates. 2012. Available at: http://www.abim.org. Accessed June 2013. 14. American Board of Internal Medicine (ABIM). First-time taker pass rates—initial certification. 2007–2011. Available at: http://www.abim.org. Accessed June 2013. 15. American Board of Internal Medicine (ABIM). ACGME accredited programs with at least ten first-time takers over the three-year period. 2012. Available at: http://www.abim.org. Accessed June 2013. 16. American Board of Internal Medicine (ABIM). First-time taker pass rates-maintenance of certification. 1997–2011. Available at: http://www.abim.org. Accessed June 2013. 17. Lipner RS, Bylsma WH, Arnold GK, et al. Who is maintaining certification in internal medicine and why? A national survey 10 years after initial certification. Ann Intern Med 2006;144:29–36. 18. Falcone JL, Hamad GG. The American Board of Surgery Certifying Examination: a retrospective study of the decreasing pass rates and performance for first-time examinees. J Surg Educ 2012;69:231–5. 19. Brennan TA, Horwitz RI, Duffy FD, et al. The role of physician specialty board certification status in the quality movement. JAMA 2004;292:1038–43. 20. Norcini JJ, Kimball HR, Lipner RS. Certification and specialization: do they matter in the outcome of acute myocardial infarction? Acad Med 2000;75:1193–8. 21. Hoangmai PH, Schrag D, Hargraves JL, et al. Delivery of preventive services to older adults by primary care physicians. JAMA 2005;294:473–81.
Volume 347, Number 1, January 2014
Changes in ABIM Pass Rates
22. Prystowsky JB, Bordage G, Feinglass JM. Patient outcomes for segmental colon resection according to surgeon’s training, certification, and experience. Surgery 2002;132:663–70. 23. Haas JS, Orav EJ, Goldman L. The relationship between physicians’ qualifications and experience and the adequacy of prenatal care and low birthweight. Am J Public Health 1995;85:1087–91. 24. American Board of Medical Specialties. Facts about the ABMS consumer survey; lifelong learning and other qualities in choosing a doctor [PDF]. Chicago, IL: ABMS; 2011. 25. American Board of Medical Specialties. Facts about the ABMS consumer survey: how Americans choose their doctor [PDF]. Chicago, IL: ABMS; 2010.
Ó 2013 Lippincott Williams & Wilkins
26. ABIM MOC program survey results. 2008–2011. Available at: http://www.abim.org/. Accessed October 2013. 27. Dale DC. Recertification in internal medicine—the American experience. Ann Acad Med Singapore 2007;36:894–7. 28. American Board of Internal Medicine (ABIM). Number of candidates certified annually by the American board of internal medicine. 1936–2011. Available at: http://www.abim.org. Accessed June 2013. 29. Hawkins RE, Weiss KB. Building the evidence base in support of the American board of medical specialties maintenance of certification program. Acad Med 2011;86:6–7. 30. Palmer I. Strategies to pass the internal medicine board exam. Philadelphia, PA: ACP-ASIM Observer; 2000.
77
