Acad Psychiatry (2014) 38:303–304 DOI 10.1007/s40596-014-0094-y
FEATURE: PERSPECTIVE
Opportunities and Challenges: Residents’ Perspectives on the Next Accreditation System in Psychiatry Alik S. Widge & Heather E. Schultz
Received: 30 November 2013 / Accepted: 5 March 2014 / Published online: 25 March 2014 # Academic Psychiatry 2014
Keywords Accreditation . Graduate medical education . Internship and residency
The unfolding of the Accreditation Council on Graduate Medical Education (ACGME)’s Next Accreditation System (NAS) and its corresponding Milestones introduce new data elements into resident evaluation. In theory, these new regulations should be welcomed and freeing for programs and trainees alike. The switch to a self-study model will reduce site visits and may relieve the associated cycles of panic. Standardized evaluations across programs should allow us to more directly compare graduates of different residencies. The grand vision is that these comparative data will act in a virtuous cycle, encouraging programs and individual trainees to try harder and innovate more [1]. NAS is intended as a step towards the utopia of “value-based health care” that was taught to us in medical school, but of which we have seen far too little on the wards [2]. That perspective has much to recommend it. First, although the Milestones were developed by committee and contain much that is subjective, they are grounded in both evidence and the collective experience of multiple respected educators. Residents can take comfort that we will be measured by a yardstick that is fair, rigorous, and well-aligned to the needed skill-set of a 21st century psychiatrist. This is doubly true given that the Psychiatry Milestones come with suggested assessment tools and anchors. No longer will an entire year
A. S. Widge (*) Massachusetts General Hospital, Charlestown, MA, USA e-mail: [email protected] H. E. Schultz University of Michigan Health System, Ann Arbor, MI, USA
be summarized by a stack of Likert scales filled with average scores and empty comment boxes. Second, a key value of the Milestones is identification of our individual areas of distinction and deficit. We and our faculty preceptors will know where we need help, and we can seek out mentors to remedy our weaknesses. Where we are exceptional, we will have standardized data to prove it. Those data could be the door to greater autonomy at an earlier stage, opportunities for teaching and scholarship, and leadership roles earlier than the chief year. All those things happen now, as strengths are organically recognized by faculty, but the Milestones could easily expedite the process. They will also galvanize new teaching, as neuroscience, system-based practice, and formal self-improvement take on the same stature in education that they are rapidly gaining in psychiatric practice. As noted elsewhere in this issue, the Milestones codify expectations in competencies that have long been neglected, but that will be critical in coming decades. Beyond the immediate horizon, Milestones might be the bridge to a portfolio-model residency. The ongoing crisis in Federal Graduate Medical Education (GME) funding, has fueled a growing call for shorter and more flexible training [3]. Although the current framework is not explicitly designed to support this, it is easy to imagine a future in which rapid progression through Milestones qualifies a resident for early graduation, perhaps even earlier than is achievable under the current “fast tracking” system. Although there would be much to resolve in terms of funding, such a scenario could benefit both trainees (by bringing us more quickly to attending salaries) and payors (by maximally leveraging GME dollars). NAS’ designers hope that all programs are (or want to be) above average, and thus, their residents also will be. Unfortunately, all of us, from the newest intern to the most senior faculty, know this to be untrue. Problem residents have existed as long as there have been residencies, and the objectivity of the Milestones will make their deficiencies blatant and un-
304
ignorable. For every trainee who excels under the Milestones, another will demonstrate clearly unsatisfactory progress. Will that resident graduate, or must his/her training be extended? How will a stipend be guaranteed during that extended time? How can early fellowship matches operate if there is no certainty of Milestone achievement before fellowship matriculation? We have no answers—and neither, it seems, does anyone else. The only certainties are that some of our colleagues will see their career plans deviate sharply once NAS goes into effect and that next years’ chief residents will face exceptionally difficult decisions. Neither is every psychiatric training program truly above average. Both of us have served as resident representatives to ACGME’s Psychiatry Residency Review Committee, and we have seen site visits reveal problems that no other instrument uncovers. Under NAS, the only site visits will be self studies, conducted decennially [1]. How will we find the places where basic supervisory responsibilities are being neglected? Where resident or patient safety takes a back seat to financial concerns? The ACGME Resident Survey cannot carry that load; the survey is feared and mis-answered by program directors and trainees alike [4–6]. In theory, trainees in particularly egregious programs will not progress at the expected rate, and ACGME data collection will flag those outliers. In practice, every instrument is subject to user error, and there is no guarantee of accurate data collection. The Milestones and their rating forms are clear, but they depend on the motivation of both teachers and learners. It would be easy enough for an evaluator to simply score every resident at his/her level of training, or to leave key items blank due to inadequate observation. Perhaps national-level statistics can identify such “faulty sensors” and troubled programs; only the experience of the next few years will tell. Finally, we wonder: what about the many aspects of psychiatry that cannot be quantified? In psychiatry, more than any other specialty, the “art” of medicine persists. It can be found, for instance, in the integration of psychopharmacology with therapy while understanding the dual backdrop of neuroscience and psychodynamics. Becoming a psychiatrist is more than linear progression along a path of skills and knowledge; it is assimilation into a professional community, and it often involves developing the first glimmers of personal insight and wisdom. Boundary crossings can be observed and rated by supervisors; it is much harder to rate whether a resident truly understands countertransference or is developing a sense of professional identity. These topics are covered in the Milestones, and it is essential that they be considered, but there is richness that cannot be captured on even an expertly crafted form. Quantitative semiannual Milestone data will meet
Acad Psychiatry (2014) 38:303–304
ACGME mandates, and may well improve psychiatric training, but when promotion or hiring decisions need to be made, data will be no substitute for an open and honest discussion with a resident’s peers or supervisors. What we can say for certain is that the NAS will generate voluminous and detailed data. For now, those data are protected. ACGME is considered a peer review organization, and it would be profoundly difficult to compel release of Milestone assessments for any public or legal purpose. The American Board of Psychiatry and Neurology has stated no plans to use Milestone data, and will continue to rely on the program director’s judgment that a resident is prepared to graduate and sit for the board exam. On the other hand, 10 years ago, the NAS itself would have been infeasible. If private or public payors do agree to greater financial support of GME, there will be strings attached [3, 7]. That demand for accountability is a key motivator behind NAS [1].Once created, data are not easily kept bottled, and we may be at the beginning of a much greater experiment in “physician quantification”. We remain optimistic that the Milestones will be a force for positive change in our profession, but that optimism is tempered with caution. Conflicts of interest Both authors have received expense reimbursement from ACGME for participation in Residency Review Committee activities and associated meetings. We otherwise declare no financial or other conflicts of interest related to this work.
References 1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system — rationale and benefits. N Engl J Med. 2012;366(11): 1051–6. 2. Porter ME. A strategy for health care reform — toward a value-based system. N Engl J Med. 2009;361(2):109–12. 3. Debra Weinstein, Ensuring an effective physician workforce for the United States: the content and format. New York, NY: Josiah Macy Jr. Foundation; 2011 Nov. 4. Fahy BN, Todd SR, Paukert JL, Johnson ML, Bass BL. How accurate is the Accreditation Council for Graduate Medical Education (ACGME) Resident Survey? Comparison between ACGME and inhouse GME survey. J Surg Educ. 2010;67(6):387–92. 5. Sticca RP, MacGregor JM, Szlabick RE. Is the Accreditation Council for Graduate Medical Education (ACGME) Resident/Fellow Survey a valid tool to assess general surgery residency programs compliance with work hour regulations? J Surg Educ. 2010;67(6):406–11. 6. Balon R. The unspoken tyranny of regulatory agencies: a commentary on the ACGME Resident Survey. Acad Psychiatry. 2012;36(5):351–2. 7. Council on Graduate Medical Education. Twenty-first report: improving value in Graduate Medical Education. Washington, DC; 2013 Aug. Available from: http://www.hrsa.gov/advisorycommittees/ bhpradvisory/cogme/Reports/twentyfirstreport.pdf. Accessed 14 Dec 2013.
FEATURE: PERSPECTIVE
Opportunities and Challenges: Residents’ Perspectives on the Next Accreditation System in Psychiatry Alik S. Widge & Heather E. Schultz
Received: 30 November 2013 / Accepted: 5 March 2014 / Published online: 25 March 2014 # Academic Psychiatry 2014
Keywords Accreditation . Graduate medical education . Internship and residency
The unfolding of the Accreditation Council on Graduate Medical Education (ACGME)’s Next Accreditation System (NAS) and its corresponding Milestones introduce new data elements into resident evaluation. In theory, these new regulations should be welcomed and freeing for programs and trainees alike. The switch to a self-study model will reduce site visits and may relieve the associated cycles of panic. Standardized evaluations across programs should allow us to more directly compare graduates of different residencies. The grand vision is that these comparative data will act in a virtuous cycle, encouraging programs and individual trainees to try harder and innovate more [1]. NAS is intended as a step towards the utopia of “value-based health care” that was taught to us in medical school, but of which we have seen far too little on the wards [2]. That perspective has much to recommend it. First, although the Milestones were developed by committee and contain much that is subjective, they are grounded in both evidence and the collective experience of multiple respected educators. Residents can take comfort that we will be measured by a yardstick that is fair, rigorous, and well-aligned to the needed skill-set of a 21st century psychiatrist. This is doubly true given that the Psychiatry Milestones come with suggested assessment tools and anchors. No longer will an entire year
A. S. Widge (*) Massachusetts General Hospital, Charlestown, MA, USA e-mail: [email protected] H. E. Schultz University of Michigan Health System, Ann Arbor, MI, USA
be summarized by a stack of Likert scales filled with average scores and empty comment boxes. Second, a key value of the Milestones is identification of our individual areas of distinction and deficit. We and our faculty preceptors will know where we need help, and we can seek out mentors to remedy our weaknesses. Where we are exceptional, we will have standardized data to prove it. Those data could be the door to greater autonomy at an earlier stage, opportunities for teaching and scholarship, and leadership roles earlier than the chief year. All those things happen now, as strengths are organically recognized by faculty, but the Milestones could easily expedite the process. They will also galvanize new teaching, as neuroscience, system-based practice, and formal self-improvement take on the same stature in education that they are rapidly gaining in psychiatric practice. As noted elsewhere in this issue, the Milestones codify expectations in competencies that have long been neglected, but that will be critical in coming decades. Beyond the immediate horizon, Milestones might be the bridge to a portfolio-model residency. The ongoing crisis in Federal Graduate Medical Education (GME) funding, has fueled a growing call for shorter and more flexible training [3]. Although the current framework is not explicitly designed to support this, it is easy to imagine a future in which rapid progression through Milestones qualifies a resident for early graduation, perhaps even earlier than is achievable under the current “fast tracking” system. Although there would be much to resolve in terms of funding, such a scenario could benefit both trainees (by bringing us more quickly to attending salaries) and payors (by maximally leveraging GME dollars). NAS’ designers hope that all programs are (or want to be) above average, and thus, their residents also will be. Unfortunately, all of us, from the newest intern to the most senior faculty, know this to be untrue. Problem residents have existed as long as there have been residencies, and the objectivity of the Milestones will make their deficiencies blatant and un-
304
ignorable. For every trainee who excels under the Milestones, another will demonstrate clearly unsatisfactory progress. Will that resident graduate, or must his/her training be extended? How will a stipend be guaranteed during that extended time? How can early fellowship matches operate if there is no certainty of Milestone achievement before fellowship matriculation? We have no answers—and neither, it seems, does anyone else. The only certainties are that some of our colleagues will see their career plans deviate sharply once NAS goes into effect and that next years’ chief residents will face exceptionally difficult decisions. Neither is every psychiatric training program truly above average. Both of us have served as resident representatives to ACGME’s Psychiatry Residency Review Committee, and we have seen site visits reveal problems that no other instrument uncovers. Under NAS, the only site visits will be self studies, conducted decennially [1]. How will we find the places where basic supervisory responsibilities are being neglected? Where resident or patient safety takes a back seat to financial concerns? The ACGME Resident Survey cannot carry that load; the survey is feared and mis-answered by program directors and trainees alike [4–6]. In theory, trainees in particularly egregious programs will not progress at the expected rate, and ACGME data collection will flag those outliers. In practice, every instrument is subject to user error, and there is no guarantee of accurate data collection. The Milestones and their rating forms are clear, but they depend on the motivation of both teachers and learners. It would be easy enough for an evaluator to simply score every resident at his/her level of training, or to leave key items blank due to inadequate observation. Perhaps national-level statistics can identify such “faulty sensors” and troubled programs; only the experience of the next few years will tell. Finally, we wonder: what about the many aspects of psychiatry that cannot be quantified? In psychiatry, more than any other specialty, the “art” of medicine persists. It can be found, for instance, in the integration of psychopharmacology with therapy while understanding the dual backdrop of neuroscience and psychodynamics. Becoming a psychiatrist is more than linear progression along a path of skills and knowledge; it is assimilation into a professional community, and it often involves developing the first glimmers of personal insight and wisdom. Boundary crossings can be observed and rated by supervisors; it is much harder to rate whether a resident truly understands countertransference or is developing a sense of professional identity. These topics are covered in the Milestones, and it is essential that they be considered, but there is richness that cannot be captured on even an expertly crafted form. Quantitative semiannual Milestone data will meet
Acad Psychiatry (2014) 38:303–304
ACGME mandates, and may well improve psychiatric training, but when promotion or hiring decisions need to be made, data will be no substitute for an open and honest discussion with a resident’s peers or supervisors. What we can say for certain is that the NAS will generate voluminous and detailed data. For now, those data are protected. ACGME is considered a peer review organization, and it would be profoundly difficult to compel release of Milestone assessments for any public or legal purpose. The American Board of Psychiatry and Neurology has stated no plans to use Milestone data, and will continue to rely on the program director’s judgment that a resident is prepared to graduate and sit for the board exam. On the other hand, 10 years ago, the NAS itself would have been infeasible. If private or public payors do agree to greater financial support of GME, there will be strings attached [3, 7]. That demand for accountability is a key motivator behind NAS [1].Once created, data are not easily kept bottled, and we may be at the beginning of a much greater experiment in “physician quantification”. We remain optimistic that the Milestones will be a force for positive change in our profession, but that optimism is tempered with caution. Conflicts of interest Both authors have received expense reimbursement from ACGME for participation in Residency Review Committee activities and associated meetings. We otherwise declare no financial or other conflicts of interest related to this work.
References 1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system — rationale and benefits. N Engl J Med. 2012;366(11): 1051–6. 2. Porter ME. A strategy for health care reform — toward a value-based system. N Engl J Med. 2009;361(2):109–12. 3. Debra Weinstein, Ensuring an effective physician workforce for the United States: the content and format. New York, NY: Josiah Macy Jr. Foundation; 2011 Nov. 4. Fahy BN, Todd SR, Paukert JL, Johnson ML, Bass BL. How accurate is the Accreditation Council for Graduate Medical Education (ACGME) Resident Survey? Comparison between ACGME and inhouse GME survey. J Surg Educ. 2010;67(6):387–92. 5. Sticca RP, MacGregor JM, Szlabick RE. Is the Accreditation Council for Graduate Medical Education (ACGME) Resident/Fellow Survey a valid tool to assess general surgery residency programs compliance with work hour regulations? J Surg Educ. 2010;67(6):406–11. 6. Balon R. The unspoken tyranny of regulatory agencies: a commentary on the ACGME Resident Survey. Acad Psychiatry. 2012;36(5):351–2. 7. Council on Graduate Medical Education. Twenty-first report: improving value in Graduate Medical Education. Washington, DC; 2013 Aug. Available from: http://www.hrsa.gov/advisorycommittees/ bhpradvisory/cogme/Reports/twentyfirstreport.pdf. Accessed 14 Dec 2013.
