Opinion
Editorials represent the opinions of the authors and JAMA and not those of the American Medical Association.
EDITORIAL
Containing Multitudes Medical Education 2013 Robert M. Golub, MD
An experiment: without any deliberation, write down what immediately comes to mind when you hear the phrase “medical education.” Ask a number of colleagues to do the same and compare answers. The results are likely to be very disparate, depending perhaps on career stage, or even more on the types of teaching activities in which each person is engaged. This is one of the challenges of medical education research: it is extremely diverse. The major contributor to this diversity is that the research is focused on a variety of levels of training, meaningful outcomes, and perspectives. For example, the objectives and techniques of education are often substantively different for medical students, residents, fellows, and practicing physicians. Outcomes examined may be approaches to delivering knowledge or facilitating learning, but may also be ways to improve assessment of learning or the effects of learning on patient care. Studies may address understanding and improving medical school admission or may instead look at how an admissions process can lead to changes in the composition of the physician workforce. While most research deals with issues related to learners, there is an increasing interest in investigating the rewards and competing stresses of faculty. This heterogeneity requires a breadth of research methods as well as investigator expertise that spans areas from statistical methods in examination development to cognitive psychology, from expertise in qualitative research to expertise in randomized trials. This range of educational levels, components, and study methods is illustrated by the spectrum of articles in the 2013 JAMA Medical Education theme issue. For any educator who has noticed with dismay dwindling attendance at lectures, the study by Deane and Murphy1 will resonate. Does it matter (other than to ego) if students are present? The authors addressed the question of whether medical student attendance at recommended activities during an obstetrics/gynecology clerkship was associated with academic performance. Previous studies have found positive associations between classroom-based lectures and academic performance, but the novelty of this study was to look at broader educational activities, including clinical teaching and observation. The authors found a positive association between attendance at each type of activity and comprehensive multimodal clerkship evaluations, but the observational design limits the ability to draw causal inferences. While the approach to student evaluation at the authors’ school in Ireland is different from most US schools, raising questions of whether the quantitative results can translate to other institutions, it
provides a blueprint for how other schools could assess similar relationships with their own students and perhaps conduct a randomized trial. Also evaluating medical students, the study by Curtis and colleagues2 used a randomized trial design to assess outcomes of a communication skills curriculum. Such curricula, which use standardized patients or role play as pedagogical tools, are common and many studies have evaluated outcomes using standardized patients combined with written examinations. However, what is particularly notable about this study is that it made the critical leap to assessing actual patient- and family-reported outcomes. In medical education research, outcome assessment directly evaluating the effect of interventions on quality of patient care is very challenging to conduct, and hence rare. Although the authors previously reported evidence using standardized patients supporting that these students had gained the desired skills for end-of-life communications, no effect was found when patients and their families did the assessment. These findings suggest a disconnect between the quality of communication skills as assessed by standardized patients and as assessed by true patients, with many possible explanations for this discrepancy that are explored in the accompanying Editorial by Chi and Verghese.3 Over the last 8 years, JAMA has published a number of studies related to the effects of resident duty hour restrictions on patient care and the quality of education.4,5 One of the consequences of these changes to the residency structure has been an increase in the frequency of patient handoffs, with an inherent potential for medical errors. Starmer and colleagues6 addressed this issue of patient safety by performing a prospective pre-post intervention study to assess whether implementation of a multifaceted handoff program would be associated with a decrease in errors and preventable adverse events. They found a statistically significant and clinically important decrease in overall errors from 33.8 to 18.3 per 100 admissions and in the more serious preventable adverse events from 3.3 to 1.5 per 100 admissions. An accompanying Editorial by Horwitz7 provides background on this issue, discusses some of the limitations in interpreting the study, and reflects on the Viewpoint by Dhaliwal and Hauer,8 which proposes a modification of the classic form of oral patient presentation to take advantage of some of the changes in admitting patterns that have resulted in part from duty hour restrictions. Along with patient safety, studies in past issues of JAMA have dealt with student and resident safety and health.9-11 In
jama.com
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
2259
Opinion Editorial
this issue, Warner and colleagues12 provide what is likely to be the most comprehensive look at substance use disorder among anesthesiology residents, studying 44 612 trainees. Although it has generally been assumed that the risk of substance use disorder in this specialty is relatively high, data have been based on limited surveys. This study not only provides higherquality estimates of the incidence of substance use during residency, but goes further with longitudinal analyses estimating the risk of relapse and of deaths related to substance use disorder over the ensuing decades of practice. The rates were very concerning: 43% risk of relapse over 30 years and at least 11% risk of death. Spanning issues of student health and academic performance is a Research Letter by Teherani and Papadakis.13 The authors compared clinical performance and graduation rates among students with vs without mental and physical disabilities. The study was limited to a single institution and hence had a small study sample, both of which may limit generalizability. The authors found a decreased graduation rate and worse academic performance on some metrics for those with disabilities, but the differences were not large and most students performed well, graduated, and matched into residency programs. This novel study should help spur more research into this area of importance to medical schools, which have had limited information about the effect of protected disabilities on medical students. Diversity in medical education is critical to produce a diverse physician workforce.14,15 Similarly, it is important to enhance diversity in medical school faculty. One approach to try to achieve this goal has been the introduction of minority faculty development programs, which have varied in their intensity and their funding. Whether such programs have been associated with improvements in the percentage of underrepresented minority faculty in US medical schools was investigated by Guevara and colleagues.16 Between 2000 and 2010, the authors found a modest increase in underrepresented minority faculty prevalence, from 6.8% to 8.0%. However, there was no significant association between these programs and increases in minority faculty representation, raising a question of whether such programs are accomplishing their goals. Because of the heterogeneity in program characteristics, further research is needed to know if there are some types of programs that may be having greater success. For example, the findings suggest greater increases among programs of longer duration and greater intensity. Diversity in the physician workforce is a downstream reflection of the medical school admissions process. The skills and knowledge in that workforce are in part shaped by licensing examinations. Given the importance of these factors, changes in the Medical College Admission Test (MCAT) and US Medical Licensing Examination (USMLE)—some of
ARTICLE INFORMATION Author Affiliation: Dr Golub is Deputy Editor, JAMA. Corresponding Author: Robert M. Golub, MD, Deputy Editor, JAMA, 330 N Wabash, Chicago, IL 60611 ([email protected]). 2260
which have already been instituted, others to occur in the near future—are likely to play a major role in the characteristics of the future physician corps. Viewpoints in this issue present summaries of the changes in the MCAT 1 7 and USMLE.18 In his accompanying Editorial, Cohen19 comments on the value that these modifications may have, but also notes that neither examination will capture some important facets of what will be needed in future physicians. Both the MCAT and USMLE focus on achievement of competencies, and the Viewpoint by Pageler and colleagues20 discusses how the electronic medical record (EMR) can be used to help assess competencies and proposes a set of EMR-related competencies the authors believe should be added to assessments. One of the major goals of the yearly JAMA Medical Education theme issue has been to highlight new research in medical education with the hope of moving the quality of such research forward. Related to this, past issues have presented the development and validation of a tool for assessing the quality of medical education research and an investigation into the relationship between funding and medical education research quality.21 The question of how much of this research gets published is raised in the Research Letter by Walsh and colleagues,22 who examined the rate and time course of peer-reviewed publication of studies presented as abstracts at major medical education conferences. They estimated that only 35% of these studies ultimately get published, lower than the 44% rate for biomedical research. Barriers to publication need to be identified and addressed. Collectively, the articles in this issue involve topics that include pedagogy, assessment of clinical skills, oral presentation, patient safety, trainee health and disability, faculty diversity, examinations affecting matriculation and licensing, recording patient data, and the fate of research. Study designs include randomized trial, pre-post intervention, prospective and retrospective cohort, and literature review. Outcomes assessed were medical errors, trainee substance use and death, ability to effectively communicate about end-of-life issues, academic performance, underrepresented minority faculty prevalence, and rate of publication. It is unusual for one discipline to have such an expansive embrace. “I am large, I contain multitudes,” Walt Whitman wrote in “Song of Myself,” his poetic celebration of the personal and the universal. Because the profession and the practice of medicine require skills, knowledge, and humanism that span the personal and universal, it is natural that its education must similarly span extraordinarily diverse disciplines. It is therefore not surprising that research covering such broad territory is challenging, and each study can only incrementally illuminate the entire picture. But each year, piece by piece, the picture continues to become more complete.
Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
REFERENCES 1. Deane RP, Murphy DJ. Student attendance and academic performance in undergraduate obstetrics/gynecology clinical rotations. JAMA. doi:10.1001/jama.2013.282228.
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
jama.com
Editorial Opinion
2. Curtis JR, Back AL, Ford DW, et al Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: a randomized trial. JAMA. doi:10.1001/jama.2013.282081. 3. Chi J, Verghese A. Improving communication with patients: learning by doing. JAMA. doi:10.1001/jama.2013.281828. 4. Fletcher KE, Underwood W III, Davis SQ, Mangrulkar RS, McMahon LF Jr, Saint S. Effects of work hour reduction on residents’ lives: a systematic review. JAMA. 2005;294(9):1088-1100. 5. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):975-983. 6. Starmer AJ, Sectish TC, Simon DW, et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA. doi:10.1001/jama.2013.281961. 7. Horwitz LI. Does improving handoffs reduce medical error rates? JAMA. doi:10.1001 /jama.2013.281827. 8. Dhaliwal G, Hauer KE. The oral patient presentation in the era of night float admissions: credit and critique. JAMA. doi:10.1001 /jama.2013.282322.
9. Ayas NT, Barger LK, Cade BE, et al. Extended work duration and the risk of self-reported percutaneous injuries in interns. JAMA. 2006;296(9):1055-1062. 10. Dyrbye LN, Massie FS Jr, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173-1180. 11. Schwenk TL, Davis L, Wimsatt LA. Depression, stigma, and suicidal ideation in medical students. JAMA. 2010;304(11):1181-1190. 12. Warner DO, Berge K, Sun H, Harman A, Hanson A, Schroeder DR. Substance use disorder among anesthesiology residents, 1975-2009. JAMA. doi:10.1001/jama.2013.281954. 13. Teherani A, Papadakis MA. Clinical performance of medical students with protected disabilities. JAMA. doi:10.1001/jama.2013.283198. 14. Grumbach K, Chen E. Effectiveness of University of California postbaccalaureate premedical programs in increasing medical school matriculation for minority and disadvantaged students. JAMA. 2006;296(9):1079-1085. 15. Saha S, Guiton G, Wimmers PF, Wilkerson L. Student body racial and ethnic composition and diversity-related outcomes in US medical schools. JAMA. 2008;300(10):1135-1145.
underrepresented minority faculty representation at US medical schools. JAMA. doi:10.1001 /jama.2013.282116. 17. Kirch DG, Mitchell K, Ast C. The new 2015 MCAT: testing competencies. JAMA. doi:10.1001/jama.2013.282093. 18. Haist SA, Katsufrakis PJ, Dillon GF. The evolution of the United States Medical Licensing Examination (USMLE): enhancing assessment of practice-related competencies. JAMA. doi:10.1001/jama.2013.282328. 19. Cohen JJ. Will changes in the MCAT and USMLE ensure that future physicians have what it takes? JAMA. doi:10.1001/jama.2013.283389. 20. Pageler NM, Friedman CP, Longhurst CA. Refocusing medical education in the EMR era. JAMA. doi:10.1001/jama.2013.282326. 21. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-1009. 22. Walsh CM, Fung M, Ginsburg S. Publication of results of abstracts presented at medical education conferences. JAMA. doi:10.1001/jama.2013.281671.
16. Guevara JP, Adanga E, Avakame E, Carthon MB. Minority faculty development programs and
jama.com
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
2261
Editorials represent the opinions of the authors and JAMA and not those of the American Medical Association.
EDITORIAL
Containing Multitudes Medical Education 2013 Robert M. Golub, MD
An experiment: without any deliberation, write down what immediately comes to mind when you hear the phrase “medical education.” Ask a number of colleagues to do the same and compare answers. The results are likely to be very disparate, depending perhaps on career stage, or even more on the types of teaching activities in which each person is engaged. This is one of the challenges of medical education research: it is extremely diverse. The major contributor to this diversity is that the research is focused on a variety of levels of training, meaningful outcomes, and perspectives. For example, the objectives and techniques of education are often substantively different for medical students, residents, fellows, and practicing physicians. Outcomes examined may be approaches to delivering knowledge or facilitating learning, but may also be ways to improve assessment of learning or the effects of learning on patient care. Studies may address understanding and improving medical school admission or may instead look at how an admissions process can lead to changes in the composition of the physician workforce. While most research deals with issues related to learners, there is an increasing interest in investigating the rewards and competing stresses of faculty. This heterogeneity requires a breadth of research methods as well as investigator expertise that spans areas from statistical methods in examination development to cognitive psychology, from expertise in qualitative research to expertise in randomized trials. This range of educational levels, components, and study methods is illustrated by the spectrum of articles in the 2013 JAMA Medical Education theme issue. For any educator who has noticed with dismay dwindling attendance at lectures, the study by Deane and Murphy1 will resonate. Does it matter (other than to ego) if students are present? The authors addressed the question of whether medical student attendance at recommended activities during an obstetrics/gynecology clerkship was associated with academic performance. Previous studies have found positive associations between classroom-based lectures and academic performance, but the novelty of this study was to look at broader educational activities, including clinical teaching and observation. The authors found a positive association between attendance at each type of activity and comprehensive multimodal clerkship evaluations, but the observational design limits the ability to draw causal inferences. While the approach to student evaluation at the authors’ school in Ireland is different from most US schools, raising questions of whether the quantitative results can translate to other institutions, it
provides a blueprint for how other schools could assess similar relationships with their own students and perhaps conduct a randomized trial. Also evaluating medical students, the study by Curtis and colleagues2 used a randomized trial design to assess outcomes of a communication skills curriculum. Such curricula, which use standardized patients or role play as pedagogical tools, are common and many studies have evaluated outcomes using standardized patients combined with written examinations. However, what is particularly notable about this study is that it made the critical leap to assessing actual patient- and family-reported outcomes. In medical education research, outcome assessment directly evaluating the effect of interventions on quality of patient care is very challenging to conduct, and hence rare. Although the authors previously reported evidence using standardized patients supporting that these students had gained the desired skills for end-of-life communications, no effect was found when patients and their families did the assessment. These findings suggest a disconnect between the quality of communication skills as assessed by standardized patients and as assessed by true patients, with many possible explanations for this discrepancy that are explored in the accompanying Editorial by Chi and Verghese.3 Over the last 8 years, JAMA has published a number of studies related to the effects of resident duty hour restrictions on patient care and the quality of education.4,5 One of the consequences of these changes to the residency structure has been an increase in the frequency of patient handoffs, with an inherent potential for medical errors. Starmer and colleagues6 addressed this issue of patient safety by performing a prospective pre-post intervention study to assess whether implementation of a multifaceted handoff program would be associated with a decrease in errors and preventable adverse events. They found a statistically significant and clinically important decrease in overall errors from 33.8 to 18.3 per 100 admissions and in the more serious preventable adverse events from 3.3 to 1.5 per 100 admissions. An accompanying Editorial by Horwitz7 provides background on this issue, discusses some of the limitations in interpreting the study, and reflects on the Viewpoint by Dhaliwal and Hauer,8 which proposes a modification of the classic form of oral patient presentation to take advantage of some of the changes in admitting patterns that have resulted in part from duty hour restrictions. Along with patient safety, studies in past issues of JAMA have dealt with student and resident safety and health.9-11 In
jama.com
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
2259
Opinion Editorial
this issue, Warner and colleagues12 provide what is likely to be the most comprehensive look at substance use disorder among anesthesiology residents, studying 44 612 trainees. Although it has generally been assumed that the risk of substance use disorder in this specialty is relatively high, data have been based on limited surveys. This study not only provides higherquality estimates of the incidence of substance use during residency, but goes further with longitudinal analyses estimating the risk of relapse and of deaths related to substance use disorder over the ensuing decades of practice. The rates were very concerning: 43% risk of relapse over 30 years and at least 11% risk of death. Spanning issues of student health and academic performance is a Research Letter by Teherani and Papadakis.13 The authors compared clinical performance and graduation rates among students with vs without mental and physical disabilities. The study was limited to a single institution and hence had a small study sample, both of which may limit generalizability. The authors found a decreased graduation rate and worse academic performance on some metrics for those with disabilities, but the differences were not large and most students performed well, graduated, and matched into residency programs. This novel study should help spur more research into this area of importance to medical schools, which have had limited information about the effect of protected disabilities on medical students. Diversity in medical education is critical to produce a diverse physician workforce.14,15 Similarly, it is important to enhance diversity in medical school faculty. One approach to try to achieve this goal has been the introduction of minority faculty development programs, which have varied in their intensity and their funding. Whether such programs have been associated with improvements in the percentage of underrepresented minority faculty in US medical schools was investigated by Guevara and colleagues.16 Between 2000 and 2010, the authors found a modest increase in underrepresented minority faculty prevalence, from 6.8% to 8.0%. However, there was no significant association between these programs and increases in minority faculty representation, raising a question of whether such programs are accomplishing their goals. Because of the heterogeneity in program characteristics, further research is needed to know if there are some types of programs that may be having greater success. For example, the findings suggest greater increases among programs of longer duration and greater intensity. Diversity in the physician workforce is a downstream reflection of the medical school admissions process. The skills and knowledge in that workforce are in part shaped by licensing examinations. Given the importance of these factors, changes in the Medical College Admission Test (MCAT) and US Medical Licensing Examination (USMLE)—some of
ARTICLE INFORMATION Author Affiliation: Dr Golub is Deputy Editor, JAMA. Corresponding Author: Robert M. Golub, MD, Deputy Editor, JAMA, 330 N Wabash, Chicago, IL 60611 ([email protected]). 2260
which have already been instituted, others to occur in the near future—are likely to play a major role in the characteristics of the future physician corps. Viewpoints in this issue present summaries of the changes in the MCAT 1 7 and USMLE.18 In his accompanying Editorial, Cohen19 comments on the value that these modifications may have, but also notes that neither examination will capture some important facets of what will be needed in future physicians. Both the MCAT and USMLE focus on achievement of competencies, and the Viewpoint by Pageler and colleagues20 discusses how the electronic medical record (EMR) can be used to help assess competencies and proposes a set of EMR-related competencies the authors believe should be added to assessments. One of the major goals of the yearly JAMA Medical Education theme issue has been to highlight new research in medical education with the hope of moving the quality of such research forward. Related to this, past issues have presented the development and validation of a tool for assessing the quality of medical education research and an investigation into the relationship between funding and medical education research quality.21 The question of how much of this research gets published is raised in the Research Letter by Walsh and colleagues,22 who examined the rate and time course of peer-reviewed publication of studies presented as abstracts at major medical education conferences. They estimated that only 35% of these studies ultimately get published, lower than the 44% rate for biomedical research. Barriers to publication need to be identified and addressed. Collectively, the articles in this issue involve topics that include pedagogy, assessment of clinical skills, oral presentation, patient safety, trainee health and disability, faculty diversity, examinations affecting matriculation and licensing, recording patient data, and the fate of research. Study designs include randomized trial, pre-post intervention, prospective and retrospective cohort, and literature review. Outcomes assessed were medical errors, trainee substance use and death, ability to effectively communicate about end-of-life issues, academic performance, underrepresented minority faculty prevalence, and rate of publication. It is unusual for one discipline to have such an expansive embrace. “I am large, I contain multitudes,” Walt Whitman wrote in “Song of Myself,” his poetic celebration of the personal and the universal. Because the profession and the practice of medicine require skills, knowledge, and humanism that span the personal and universal, it is natural that its education must similarly span extraordinarily diverse disciplines. It is therefore not surprising that research covering such broad territory is challenging, and each study can only incrementally illuminate the entire picture. But each year, piece by piece, the picture continues to become more complete.
Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
REFERENCES 1. Deane RP, Murphy DJ. Student attendance and academic performance in undergraduate obstetrics/gynecology clinical rotations. JAMA. doi:10.1001/jama.2013.282228.
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
jama.com
Editorial Opinion
2. Curtis JR, Back AL, Ford DW, et al Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: a randomized trial. JAMA. doi:10.1001/jama.2013.282081. 3. Chi J, Verghese A. Improving communication with patients: learning by doing. JAMA. doi:10.1001/jama.2013.281828. 4. Fletcher KE, Underwood W III, Davis SQ, Mangrulkar RS, McMahon LF Jr, Saint S. Effects of work hour reduction on residents’ lives: a systematic review. JAMA. 2005;294(9):1088-1100. 5. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):975-983. 6. Starmer AJ, Sectish TC, Simon DW, et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA. doi:10.1001/jama.2013.281961. 7. Horwitz LI. Does improving handoffs reduce medical error rates? JAMA. doi:10.1001 /jama.2013.281827. 8. Dhaliwal G, Hauer KE. The oral patient presentation in the era of night float admissions: credit and critique. JAMA. doi:10.1001 /jama.2013.282322.
9. Ayas NT, Barger LK, Cade BE, et al. Extended work duration and the risk of self-reported percutaneous injuries in interns. JAMA. 2006;296(9):1055-1062. 10. Dyrbye LN, Massie FS Jr, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173-1180. 11. Schwenk TL, Davis L, Wimsatt LA. Depression, stigma, and suicidal ideation in medical students. JAMA. 2010;304(11):1181-1190. 12. Warner DO, Berge K, Sun H, Harman A, Hanson A, Schroeder DR. Substance use disorder among anesthesiology residents, 1975-2009. JAMA. doi:10.1001/jama.2013.281954. 13. Teherani A, Papadakis MA. Clinical performance of medical students with protected disabilities. JAMA. doi:10.1001/jama.2013.283198. 14. Grumbach K, Chen E. Effectiveness of University of California postbaccalaureate premedical programs in increasing medical school matriculation for minority and disadvantaged students. JAMA. 2006;296(9):1079-1085. 15. Saha S, Guiton G, Wimmers PF, Wilkerson L. Student body racial and ethnic composition and diversity-related outcomes in US medical schools. JAMA. 2008;300(10):1135-1145.
underrepresented minority faculty representation at US medical schools. JAMA. doi:10.1001 /jama.2013.282116. 17. Kirch DG, Mitchell K, Ast C. The new 2015 MCAT: testing competencies. JAMA. doi:10.1001/jama.2013.282093. 18. Haist SA, Katsufrakis PJ, Dillon GF. The evolution of the United States Medical Licensing Examination (USMLE): enhancing assessment of practice-related competencies. JAMA. doi:10.1001/jama.2013.282328. 19. Cohen JJ. Will changes in the MCAT and USMLE ensure that future physicians have what it takes? JAMA. doi:10.1001/jama.2013.283389. 20. Pageler NM, Friedman CP, Longhurst CA. Refocusing medical education in the EMR era. JAMA. doi:10.1001/jama.2013.282326. 21. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-1009. 22. Walsh CM, Fung M, Ginsburg S. Publication of results of abstracts presented at medical education conferences. JAMA. doi:10.1001/jama.2013.281671.
16. Guevara JP, Adanga E, Avakame E, Carthon MB. Minority faculty development programs and
jama.com
JAMA December 4, 2013 Volume 310, Number 21
Copyright 2013 American Medical Association. All rights reserved.
Downloaded From: http://jama.jamanetwork.com/ by a Ndsu Library Periodicals User on 05/23/2015
2261
