Opinion
VIEWPOINT
David G. Nichols, MD, MBA The American Board of Pediatrics, Chapel Hill, North Carolina.
The Future of Board Certification Learning Is Competency Although its wording has evolved, the essence of the mission of the American Board of Pediatrics (ABP) has not changed: to certify “pediatricians based on standards of excellence that lead to high-quality health care during infancy, childhood, adolescence, and the transition to adulthood.” The ABP needs assessments embedded in clinical practice and improved patient outcomes as the explicit metric for excellence, which will require a commitment to continuous learning. This Viewpoint proposes ways for a certifying board to underscore learning as the most important competency for the physician while acknowledging that any new approach must be pilot tested and subject to rigorous evaluation.
The Learning Environment
Corresponding Author: David G. Nichols, MD, MBA, The American Board of Pediatrics, 111 Silver Cedar Court, Chapel Hill, NC 27514-1513 ([email protected]). jamapediatrics.com
The important first step toward credentialing future ABP diplomates involves setting standards for their learning during residency and fellowship. Pediatric training programs have embraced flexibility, innovation, and critical thinking so their graduates appear comfortable caring for a wide array of children, including those with chronic illnesses.1 The notable exception to this trend involves mental health disorders where residency graduates feel unprepared for growing demands.1 This example underscores the dynamic nature of disease patterns to which the board certification process must adapt. The ABP’s collaborations in developing training guidelines and assessments need to include behavioral and mental health disorders. Greater emphasis on mental health content in the general pediatrics examination and m a i n t e n a n c e o f c e r t i f i c a t i o n ( M O C ) q u a l i tyimprovement projects (MOC part 4) need to be designed to enhance mental health care competencies. Mobile devices and applications could also incorporate life-long learning and self-assessment (MOC part 2) opportunities around mental health care into routine daily practice. Competencies beyond medical knowledge are essential for effective patient care, illustrating the need for partnerships (systems-based practice) with schools and other providers such as nurse practitioners, child psychiatrists, and social workers to properly care for these children. Asch and colleagues2 demonstrated an association between maternal complications and the learning environment from which the treating obstetrician graduated. Assuming this association between residency program quality and future patient complications applies to all specialties, faculty, accrediting agencies, and certifying boards must collaborate so highly competent physicians can emerge from highly competent learning environments focused on safe, effective, and patientcentered care.
Assessment of and for Learning Although the ABP’s responsibility to the public mandates assessment of the diplomate’s learning, we have an opportunity to use assessment for the purpose of catalyzing learning to improve patient care and outcomes. Assessment in the Certifying Examination
In the same way that systematic reviews grade evidence, certifying examination questions should list the evidence grade underlying the question even if highgrade evidence does not exist in certain areas of practice. Numerous studies have pointed out the lag in applying best evidence to patient care, resulting in unexplained variation in care. Certifying boards could help shorten this lag and reduce the variation if the design of the certifying examination encouraged examinees to learn and stay current with the best evidence. The specific content of best evidence used in the examination would need to consider the prevalence, risk profiles, phenotypic variation, and economic cost of everchanging disease patterns for children. The best evidence that will improve children’s health should be reflected on board examinations and should be widely publicized in advance. Pediatricians need a broad knowledge base of pediatric health. They also routinely use mobile devices to look up the rapidly changing details of care. Because there is not time to look up everything, the real skill occurs in integrating an individual’s knowledge base with large amounts of data, analyzing those data, and arriving at the best decision for the patient. If the technological obstacles can be overcome, access to online resources during the examination will better align knowledge assessment with authentic practice. Assessment in the Clinic
Educators have long strived to tie assessments to educational outcomes. This year, the Accreditation Council for Graduate Medical Education will require pediatric training programs to report resident performance based on the Pediatrics Milestones Project. This project uses brief narratives of behavior within a competency to describe performance development from novice to expert.3 Preliminary data suggest the value of the Pediatric Milestones in providing resident feedback.4 Because feedback catalyzes learning and may change behavior, the assessment process should drive learning to improve patient care. Parents look for an integrated whole of knowledge, skill, compassion, and ability to communicate in a great pediatrician and so must the ABP. Entrustable professional activities hold promise in this regard as observJAMA Pediatrics September 2014 Volume 168, Number 9
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a BYU Harold B Lee Library User on 05/19/2015
789
Opinion Viewpoint
able and measurable professional activities (eg, providing consultation) that define a discipline in the aggregate.5 Entrustment—safely and effectively performing the entrustable professional activities without supervision—is the essential criterion for assessment. The duration of training needed to gain entrustment may vary by subspecialty (or by individual). The recommendations from the ABP Subspecialty Clinical Training and Certification Initiative suggest the possibility of departing from the uniform 3-year fellowship duration to a fellowship that varies by subspecialty based on entrustable professional activities.6 The ABP’s responsibility to the public is then to verify the graduate and future diplomate has met a high standard of competency.
Learning in the Health Care System The ABP supports the Institute of Medicine’s call for alignment of science, informatics, and incentives to improve care through a learning health care system.7 As data captured by the electronic health record for quality improvement and patient outcomes become embedded in practice, deidentified data showing meaningful and sustained quality improvement obtained in the course of practice could result in an MOC part 4 credit. Passage of the written MOC examination (part 3) is a proxy for the correct application of patient-care knowledge. At some point, if the electronic health record allows individual pediatricians to document case mix and risk-adjusted superior outcomes for all patient ARTICLE INFORMATION Published Online: July 7, 2014. doi:10.1001/jamapediatrics.2014.943. Conflict of Interest Disclosures: None reported. Additional Contributions: I thank Linda A. Althouse, PhD, The American Board of Pediatrics; William Ballistreri, MD, University of Cincinnati; Laura Brooks, MD, F. Read Hopkins Pediatric Associates; Carol Carracio, MD, The American Board of Pediatrics; M. Douglas Jones Jr, MD, University of Colorado; Marshall Land, MD, University of Vermont; Laurel K. Leslie, MD, Tufts University; Gail A. McGuinness, The American Board of Pediatrics; and Virginia A. Moyer, MD, The American Board of Pediatrics, for review and suggested revisions to the manuscript. I thank Erik Meyer, BS, the American Board of Pediatrics, for editorial assistance. These individuals did not receive financial compensation. REFERENCES 1. Freed GL, Dunham KM, Switalski KE, Jones MD Jr, McGuinness GA; Research Advisory Committee
790
groups in their practice, reliance on the examination for MOC will decrease. The enhanced sample size of a national Medicaid claims database similar to the Medicare database would greatly facilitate research on the relationship between pediatrician competency, quality-improvement efforts, and child health outcomes, as has been demonstrated for adult patients using claims data.8,9
Learning Professionalism Currently, all certifying boards require diplomates to have an unrestricted license as a marker of professionalism. During the past decade, less than 1% of ABP diplomates have had their diplomate status revoked for grave offenses. Several writers have attempted to lift the concept of professionalism beyond the good doctor–bad doctor dichotomy and reframe professionalism as behavior affecting concentric circles of engagement, beginning with the individual patient and extending outwards to the care by teams, practice settings, and the entire medical system.10 In this context, professionalism is a behavior that can be learned and improved by all of us.
Conclusions The mission of the ABP endures and is meant to provide assurance to the public that the pediatrician meets high standards of competency. The future focus of this paradigm should be on learning so that patient outcomes serve as the important metric for diplomate status.
of the American Board of Pediatrics. Recently trained general pediatricians: perspectives on residency training and scope of practice. Pediatrics. 2009;123(suppl 1):S38-S43. 2. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009; 302(12):1277-1283.
and Certification (SCTC): background and recommendations. Pediatrics. 2014;133:S51-S52. doi: 10.1542/peds.2013-3861C. 7. Smith M, Saunders R, Stuckhardt L, McGinnis JM. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2013.
3. Carraccio C, Englander R. A milestone for the pediatrics milestones. Acad Pediatr. 2014;14(2) (suppl):1-3.
8. Reid RO, Friedberg MW, Adams JL, McGlynn EA, Mehrotra A. Associations between physician characteristics and quality of care. Arch Intern Med. 2010;170(16):1442-1449.
4. Poynter S, Turner T, Chaffinch C, Hicks PJ. Learner feedback on the Pediatrics Milestones Assessment Project. Paper presented at: ACGME Annual Education Conference. Orlando, FL. March 1-3, 2013.
9. Holmboe ES, Wang Y, Meehan TP, et al. Association between maintenance of certification examination scores and quality of care for medicare beneficiaries. Arch Intern Med. 2008;168(13):13961403.
5. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542-547.
10. Lesser CS, Lucey CR, Egener B, Braddock CH III, Linas SL, Levinson W. A behavioral and systems view of professionalism. JAMA. 2010;304(24): 2732-2737.
6. Stevenson DK, McGuinness GA, Bancroft JD, et al. The initiative on Subspecialty Clinical Training
JAMA Pediatrics September 2014 Volume 168, Number 9
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a BYU Harold B Lee Library User on 05/19/2015
jamapediatrics.com
VIEWPOINT
David G. Nichols, MD, MBA The American Board of Pediatrics, Chapel Hill, North Carolina.
The Future of Board Certification Learning Is Competency Although its wording has evolved, the essence of the mission of the American Board of Pediatrics (ABP) has not changed: to certify “pediatricians based on standards of excellence that lead to high-quality health care during infancy, childhood, adolescence, and the transition to adulthood.” The ABP needs assessments embedded in clinical practice and improved patient outcomes as the explicit metric for excellence, which will require a commitment to continuous learning. This Viewpoint proposes ways for a certifying board to underscore learning as the most important competency for the physician while acknowledging that any new approach must be pilot tested and subject to rigorous evaluation.
The Learning Environment
Corresponding Author: David G. Nichols, MD, MBA, The American Board of Pediatrics, 111 Silver Cedar Court, Chapel Hill, NC 27514-1513 ([email protected]). jamapediatrics.com
The important first step toward credentialing future ABP diplomates involves setting standards for their learning during residency and fellowship. Pediatric training programs have embraced flexibility, innovation, and critical thinking so their graduates appear comfortable caring for a wide array of children, including those with chronic illnesses.1 The notable exception to this trend involves mental health disorders where residency graduates feel unprepared for growing demands.1 This example underscores the dynamic nature of disease patterns to which the board certification process must adapt. The ABP’s collaborations in developing training guidelines and assessments need to include behavioral and mental health disorders. Greater emphasis on mental health content in the general pediatrics examination and m a i n t e n a n c e o f c e r t i f i c a t i o n ( M O C ) q u a l i tyimprovement projects (MOC part 4) need to be designed to enhance mental health care competencies. Mobile devices and applications could also incorporate life-long learning and self-assessment (MOC part 2) opportunities around mental health care into routine daily practice. Competencies beyond medical knowledge are essential for effective patient care, illustrating the need for partnerships (systems-based practice) with schools and other providers such as nurse practitioners, child psychiatrists, and social workers to properly care for these children. Asch and colleagues2 demonstrated an association between maternal complications and the learning environment from which the treating obstetrician graduated. Assuming this association between residency program quality and future patient complications applies to all specialties, faculty, accrediting agencies, and certifying boards must collaborate so highly competent physicians can emerge from highly competent learning environments focused on safe, effective, and patientcentered care.
Assessment of and for Learning Although the ABP’s responsibility to the public mandates assessment of the diplomate’s learning, we have an opportunity to use assessment for the purpose of catalyzing learning to improve patient care and outcomes. Assessment in the Certifying Examination
In the same way that systematic reviews grade evidence, certifying examination questions should list the evidence grade underlying the question even if highgrade evidence does not exist in certain areas of practice. Numerous studies have pointed out the lag in applying best evidence to patient care, resulting in unexplained variation in care. Certifying boards could help shorten this lag and reduce the variation if the design of the certifying examination encouraged examinees to learn and stay current with the best evidence. The specific content of best evidence used in the examination would need to consider the prevalence, risk profiles, phenotypic variation, and economic cost of everchanging disease patterns for children. The best evidence that will improve children’s health should be reflected on board examinations and should be widely publicized in advance. Pediatricians need a broad knowledge base of pediatric health. They also routinely use mobile devices to look up the rapidly changing details of care. Because there is not time to look up everything, the real skill occurs in integrating an individual’s knowledge base with large amounts of data, analyzing those data, and arriving at the best decision for the patient. If the technological obstacles can be overcome, access to online resources during the examination will better align knowledge assessment with authentic practice. Assessment in the Clinic
Educators have long strived to tie assessments to educational outcomes. This year, the Accreditation Council for Graduate Medical Education will require pediatric training programs to report resident performance based on the Pediatrics Milestones Project. This project uses brief narratives of behavior within a competency to describe performance development from novice to expert.3 Preliminary data suggest the value of the Pediatric Milestones in providing resident feedback.4 Because feedback catalyzes learning and may change behavior, the assessment process should drive learning to improve patient care. Parents look for an integrated whole of knowledge, skill, compassion, and ability to communicate in a great pediatrician and so must the ABP. Entrustable professional activities hold promise in this regard as observJAMA Pediatrics September 2014 Volume 168, Number 9
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a BYU Harold B Lee Library User on 05/19/2015
789
Opinion Viewpoint
able and measurable professional activities (eg, providing consultation) that define a discipline in the aggregate.5 Entrustment—safely and effectively performing the entrustable professional activities without supervision—is the essential criterion for assessment. The duration of training needed to gain entrustment may vary by subspecialty (or by individual). The recommendations from the ABP Subspecialty Clinical Training and Certification Initiative suggest the possibility of departing from the uniform 3-year fellowship duration to a fellowship that varies by subspecialty based on entrustable professional activities.6 The ABP’s responsibility to the public is then to verify the graduate and future diplomate has met a high standard of competency.
Learning in the Health Care System The ABP supports the Institute of Medicine’s call for alignment of science, informatics, and incentives to improve care through a learning health care system.7 As data captured by the electronic health record for quality improvement and patient outcomes become embedded in practice, deidentified data showing meaningful and sustained quality improvement obtained in the course of practice could result in an MOC part 4 credit. Passage of the written MOC examination (part 3) is a proxy for the correct application of patient-care knowledge. At some point, if the electronic health record allows individual pediatricians to document case mix and risk-adjusted superior outcomes for all patient ARTICLE INFORMATION Published Online: July 7, 2014. doi:10.1001/jamapediatrics.2014.943. Conflict of Interest Disclosures: None reported. Additional Contributions: I thank Linda A. Althouse, PhD, The American Board of Pediatrics; William Ballistreri, MD, University of Cincinnati; Laura Brooks, MD, F. Read Hopkins Pediatric Associates; Carol Carracio, MD, The American Board of Pediatrics; M. Douglas Jones Jr, MD, University of Colorado; Marshall Land, MD, University of Vermont; Laurel K. Leslie, MD, Tufts University; Gail A. McGuinness, The American Board of Pediatrics; and Virginia A. Moyer, MD, The American Board of Pediatrics, for review and suggested revisions to the manuscript. I thank Erik Meyer, BS, the American Board of Pediatrics, for editorial assistance. These individuals did not receive financial compensation. REFERENCES 1. Freed GL, Dunham KM, Switalski KE, Jones MD Jr, McGuinness GA; Research Advisory Committee
790
groups in their practice, reliance on the examination for MOC will decrease. The enhanced sample size of a national Medicaid claims database similar to the Medicare database would greatly facilitate research on the relationship between pediatrician competency, quality-improvement efforts, and child health outcomes, as has been demonstrated for adult patients using claims data.8,9
Learning Professionalism Currently, all certifying boards require diplomates to have an unrestricted license as a marker of professionalism. During the past decade, less than 1% of ABP diplomates have had their diplomate status revoked for grave offenses. Several writers have attempted to lift the concept of professionalism beyond the good doctor–bad doctor dichotomy and reframe professionalism as behavior affecting concentric circles of engagement, beginning with the individual patient and extending outwards to the care by teams, practice settings, and the entire medical system.10 In this context, professionalism is a behavior that can be learned and improved by all of us.
Conclusions The mission of the ABP endures and is meant to provide assurance to the public that the pediatrician meets high standards of competency. The future focus of this paradigm should be on learning so that patient outcomes serve as the important metric for diplomate status.
of the American Board of Pediatrics. Recently trained general pediatricians: perspectives on residency training and scope of practice. Pediatrics. 2009;123(suppl 1):S38-S43. 2. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009; 302(12):1277-1283.
and Certification (SCTC): background and recommendations. Pediatrics. 2014;133:S51-S52. doi: 10.1542/peds.2013-3861C. 7. Smith M, Saunders R, Stuckhardt L, McGinnis JM. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2013.
3. Carraccio C, Englander R. A milestone for the pediatrics milestones. Acad Pediatr. 2014;14(2) (suppl):1-3.
8. Reid RO, Friedberg MW, Adams JL, McGlynn EA, Mehrotra A. Associations between physician characteristics and quality of care. Arch Intern Med. 2010;170(16):1442-1449.
4. Poynter S, Turner T, Chaffinch C, Hicks PJ. Learner feedback on the Pediatrics Milestones Assessment Project. Paper presented at: ACGME Annual Education Conference. Orlando, FL. March 1-3, 2013.
9. Holmboe ES, Wang Y, Meehan TP, et al. Association between maintenance of certification examination scores and quality of care for medicare beneficiaries. Arch Intern Med. 2008;168(13):13961403.
5. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542-547.
10. Lesser CS, Lucey CR, Egener B, Braddock CH III, Linas SL, Levinson W. A behavioral and systems view of professionalism. JAMA. 2010;304(24): 2732-2737.
6. Stevenson DK, McGuinness GA, Bancroft JD, et al. The initiative on Subspecialty Clinical Training
JAMA Pediatrics September 2014 Volume 168, Number 9
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a BYU Harold B Lee Library User on 05/19/2015
jamapediatrics.com
