Radiologic Education

Predictors of an Academic Career on Radiology Residency Applications Lars J. Grimm, MD, MHS, Lauren M. Shapiro, BA, Terry Singhapricha, BS, Maciej A. Mazurowski, PhD, Terry S. Desser, MD, Charles M. Maxfield, MD Rationale and Objectives: To evaluate radiology residency applications to determine if any variables are predictive of a future academic radiology career. Materials and Methods: Application materials from 336 radiology residency graduates between 1993 and 2010 from the Department of Radiology, Duke University and between 1990 and 2010 from the Department of Radiology, Stanford University were retrospectively reviewed. The institutional review boards approved this Health Insurance Portability and Accountability Act–compliant study with a waiver of informed consent. Biographical (gender, age at application, advanced degrees, prior career), undergraduate school (school, degree, research experience, publications), and medical school (school, research experience, manuscript publications, Alpha Omega Alpha membership, clerkship grades, United States Medical Licensing Examination Step 1 and 2 scores, personal statement and letter of recommendation reference to academics, couples match status) data were recorded. Listing in the Association of American Medical Colleges Faculty Online Directory and postgraduation publications were used to determine academic status. Results: There were 72 (21%) radiologists in an academic career and 264 (79%) in a nonacademic career. Variables associated with an academic career were elite undergraduate school (P = .003), undergraduate school publications (P = .018), additional advanced degrees (P = .027), elite medical school (P = .006), a research year in medical school (P < .001), and medical school publications (P < .001). A multivariate cross-validation analysis showed that these variables are jointly predictive of an academic career (P < .001). Conclusions: Undergraduate and medical school rankings and publications, as well as a medical school research year and an additional advanced degree, are associated with an academic career. Radiology residency selection committees should consider these factors in the context of the residency application if they wish to recruit future academic radiologists. Key Words: Academic career; residency applications; predictors of academics; residency graduates; career choice. ªAUR, 2014

M

any academic radiology residency programs consider it a part of their core mission to produce future academic and thought leaders (1). For these programs, an applicant’s potential for an academic career is a consideration in the residency selection process (2), and yet it is not clear whether there are variables on the residency application which reliably allow identification of those likely to pursue an academic career. This question has been asked by multiple investigators at different institutions over the past two decades, studying a broad range of demographic and educational variables across subspecialties. However, these results have been inconsistent and at times conflicting (Table 1) (3–10). The variable that has generally shown the greatest association with an academic career is medical school publications.

The factors that help influence individuals to pursue a career in academics may be distinct for each specialty; thus, the results from any one specialty may not be generalizable across all specialties. We are aware of only two radiology specific studies that looked at predictors of a future career in academics (5,7). Both rely on retrospective surveys of academic radiologists rather than relying on information available to resident selection committees at the time of applications; furthermore, both were published more than 20 years ago. The goal of our study is to evaluate a diverse spectrum of variables from radiology residency applications among a large cohort from two institutions over 20 years to determine if any variables are predictive of a future academic radiology career. We hope to provide some guidance for radiology residency program directors and selection committees who are interested in recruiting applicants likely to pursue academic careers and developing those residents with academic potential.

Acad Radiol 2014; 21:685–690 From the Department of Radiology, Duke University Hospital, Box 3808, Durham, NC 27710 (L.J.G., T.S., C.M.M.); Department of Radiology, Stanford University Medical Center, Stanford, CA (L.M.S., T.S.D.); and Carl E. Ravin Advanced Imaging Laboratories, Duke University Hospital, Durham, NC (M.A.M.). Received September 14, 2013; accepted October 21, 2013. Funding: The authors have no funding information to disclose. Address correspondence to: L.J.G. e-mail: [email protected] ªAUR, 2014 http://dx.doi.org/10.1016/j.acra.2013.10.019

MATERIALS AND METHODS The residency files and application materials from radiology residents who graduated between 1993 and 2010 from the Department of Radiology, Duke University and between 1990 and 2010 from the Department of Radiology, Stanford 685

GRIMM ET AL

Academic Radiology, Vol 21, No 5, May 2014

TABLE 1. Summary of Findings from Prior Work Assessing the Likelihood of a Future Academic Career Predictive Variable Female gender

Association

No Association

Hillman et al., 1990* Dorsey et al., 2006 Andriole et al., 2012 McClelland et al., 2010 Andriole et al., 2010 Markert et al., 1998 Lawton et al., 2007

Age

Markert et al., 1998 Lawton et al., 2007

Undergraduate Dorsey et al., 2006 school rankings Undergraduate Andriole et al., 2012 school research Additional Dorsey et al., 2006 advanced degree Markert et al., 1998 Lawton et al., 2007 Medical school research

Bilbey et al., 1992*

Andriole et al., 2012

Medical school ranking

Hillman et al., 1990* Dorsey et al., 2006

USMLE Step 1 scores

Andriole et al., 2010 Markert et al., 1998

AOA membership Medical school publications

Lawton et al., 2007 Hillman et al., 1990* Dorsey et al., 2006

Markert et al., 1998

Andriole et al., 2012 Andriole et al., 2010

Andriole et al., 2012

Bilbey et al., 1992*

Lawton et al., 2007

McClelland et al., 2010 Andriole et al., 2012 Medical school honors grades

Dorsey et al., 2006

Medical school class rank

Bilbey et al., 1992*

Lawton et al., 2007

AOA, Alpha Omega Alpha membership; USMLE, United States Medical Licensing Examination. *Denotes radiology specific study.

University were reviewed. Residents were grouped for analysis with their entering class, even if they took additional time or did not complete training at the program. Biographical, undergraduate school, and medical school data were recorded from original application materials. Biographical data included gender, age at the time of application, advanced degrees in addition to Medical Doctor (MD), and any prior career. A prior career was defined as work not designed to get research or clinical experience with the goal of attending medical school. For example, working as a laboratory assistant for a year would not qualify, but working as an elementary school science teacher would qualify. 686

Undergraduate school data included undergraduate school attended, degree obtained, research experience, and manuscript publications. Research experience was self-reported, but publications were confirmed via searches on PubMed and Google Scholar. Undergraduate schools were classified as ‘‘elite’’ if the school appeared in the top 10 list of US News and World Report Rankings for best national liberal arts colleges or national universities during the years (1993– 2010) of the study (Table 2) (11). Manuscripts published after undergraduate school but before medical school matriculation were classified as undergraduate school publications. Manuscripts published by the undergraduate institution were counted as undergraduate school publications even if the date of publication occurred after the applicant entered medical school. Medical school data included medical school attended, research experience, dedicated year of research, manuscript publications, Alpha Omega Alpha membership (AOA), clerkship grades, United States Medical Licensing Examination (USMLE) Step 1 score, USMLE Step 2 score, personal statement references to a career in academics, letters of recommendation references to a potential career in academics, and participation in the couples matching. Clerkship grades of Honors or A were considered synonymous. Medical schools were classified as elite if they appeared in the top 10 list of the US News and World Report Rankings during the years of study (Table 2) (11). Residency graduates were classified into ‘‘academic’’ or ‘‘not academic’’ categories. To determine academic status, graduates were searched via the Association of American Medical Colleges Faculty Online Directory (12) and via PubMed for postgraduation publications. Graduates who were listed in the directory and continued to publish articles after residency and fellowship were considered academic. Graduates who were not listed and who had not published articles since graduation were considered not academic. Discrepancies between the two sources were clarified by reviewing the graduates’ job description via departmental website or direct contact with the graduate. Graduates were considered ‘‘academic’’ if at any point in their career they qualified via the above criteria. To evaluate the relationship between the individual variables considered in this study, we constructed a univariate logistic regression model for each of the variables. The glm function in the R statistical software was used for this purpose (R Development Core Team). Furthermore, to determine whether the variables considered in this study can predict the academic career status, we constructed and validated a multivariate classifier based on these variables. Specifically, we used the random forest classifier (13) for prediction with 1000 individual decision trees. A MATLAB implementation of random forest was used in the function treebagger (MathWorks, Natick, MA). The USMLE Step 1 and 2 scores were excluded from the multivariate analysis because of a low number of observations. For validation, a popular leave-one-out cross-validation method was used to eliminate the potential

Academic Radiology, Vol 21, No 5, May 2014

RESIDENCY APPLICATION CAREER PREDICTORS

TABLE 2. List of Elite Undergraduate Schools and Elite Medical Schools Elite Undergraduate Schools Brown University California Institute of Technology Columbia University Cornell University Dartmouth University Duke University Emory University Harvard University Johns Hopkins University Massachusetts Institute of Technology Northwestern University Princeton University Stanford University University of Chicago University of Pennsylvania Yale University

Elite Medical Schools Amherst College Bowdoin College Bryn Mawr College Carleton College Claremont McKenna College Davidson College Haverford College Middlebury College Pomona College Swarthmore College Vassar College Wellesley College Wesleyan College Williams College

positive bias. To measure performance of this multivariate classifier, we used the receiver operating characteristic (ROC) analysis and specifically the area under the ROC curve (AUC). To estimate the AUC, we used the roc function in R’s pROC package, and to estimate the confidence interval for the AUC and the P value for the hypothesis that the AUC is larger than 0.5 (i.e., better than for a ‘‘flip of a coin’’) we used the ci.auc function in the same package. The confidence intervals were calculated using the DeLong method (14). A P value of less than .05 was considered statistically significant. The institutional review boards at both institutions approved this retrospective Health Insurance Portability and Accountability Act–compliant study with a waiver of informed consent.

Columbia University Duke University Harvard University Johns Hopkins University Stanford University University of California Los Angeles University of California San Francisco University of Chicago University of Michigan University of Pennsylvania University of Washington Washington University in St. Louis Yale University

(P = .577), undergraduate research (P = .670), medical school research (P = .059), a letter of recommendation reference to academics (P = .099), a personal statement reference to academics (P = .874), USMLE Step 1 scores (P = .440), USMLE Step 2 scores (0.341), clerkship grades (0.980), couples match status (P = .439), or AOA status (0.134). Furthermore, multivariate cross-validation analysis showed that the 19 analyzed variables could be collectively used to predict whether an applicant will pursue an academic career. The AUC was 0.631 (95% confidence interval: 0.561–0.701, P < .001), which means that the model predictions were significantly better than chance.

DISCUSSION RESULTS A total of 336 radiologists were included in the analysis, of whom 231 (68%) were men and 105 (32%) were women. Of this group, 72 (21%) were classified as academic and 264 (79%) were not academic. Table 3 shows the information available for each characteristic with a breakdown among all applicants and those with academic or not academic classifications. The univariate analysis showed a positive association between an academic career with attending an elite undergraduate school (P = .003), publishing a peer-reviewed manuscript during undergraduate school (P = .018), receiving an additional advanced degree (P = .027), attending an elite medical school (P = .006), pursuing a dedicated year of research in medical school (P < .001), and publishing a peer-reviewed manuscript in medical school (P < .001). An academic career was also positively associated with the quantity of publications during medical school (P = .009) but not during undergraduate school (P = .242). There was no statistically significant association between an academic career with gender (P = .191), age (P = .805), undergraduate degree (P = .619), a prior career

This study analyzes the largest number of potentially predictive criteria among the published literature to date and provides important additional insights into the predictive influences of residency applications, specifically for the field of radiology. Our data confirm that medical school publications are the most consistent predictive indicator of a future career in academics that is available to selection committees at the time of the residency application. There was a significant difference in both the act of publishing (P < .001) and the number of publications (P = .009) produced during medical school. This is in line with the findings of multiple other studies in radiology and other specialties (3,5,7,10). The consistent nature of this finding indicates that programs directors and residency committees should place great emphasis on medical school publications if they wish to enroll applicants with a high likelihood of pursuing a career in academics. A dedicated year of research during medical school was significantly (P < .001) associated with a career in academics, with research time during medical school borderline significant (P = .0587). Prior studies have demonstrated a mixed 687

GRIMM ET AL

Academic Radiology, Vol 21, No 5, May 2014

TABLE 3. Distribution of Applicant Characteristics by Total, Academics, and Not Academics

Characteristic Gender Male Female Age (y) Undergrad degree BA BS Elite undergraduate school No Yes Prior career No Yes Undergraduate research No Yes Undergraduate publications No Yes Undergraduate publication quantity Advanced degree No Yes Elite medical school No Yes Medical school research No Yes Medical school research year No Yes Medical school publications No Yes Medical school publication quantity Letter of recommendation: academics No Yes Personal statement: academics No Yes USMLE Step 1 USMLE Step 2 Clerkship honors No Yes Couples match No Yes AOA No Yes

Data Total, n (Column %) Academics, n (Row %) Not Academics, Available, n (%) or Mean (SD) or Mean (SD) n (Row %) or Mean (SD) P value 336 (100)

316 (94) 319 (95)

.191 231 (68%) 105 (32%) 27.9 (2.6)

45 (19%) 27 (26%) 27.8 (2.2)

186 (81%) 78 (74%) 27.9 (2.7)

146 (46%) 168 (54%)

27 (19%) 36 (21%)

119 (81%) 137 (79%)

217 (68%) 103 (32%)

33 (15%) 29 (28%)

184 (85%) 74 (72%)

279 (89%) 34 (11%)

57 (20%) 5 (15%)

222 (80%) 29 (85%)

96 (30%) 220 (70%)

18 (19%) 44 (20%)

78 (81%) 176 (80%)

230 (74%) 79 (26%) 0.6 (1.8)

39 (17%) 22 (28%) 0.8 (1.7)

191 (83%) 57 (72%) 0.6 (1.7)

256 (81%) 60 (19%)

46 (18%) 18 (30%)

210 (82%) 42 (70%)

192 (57%) 143 (43%)

31 (16%) 41 (29%)

161 (84%) 102 (71%)

37 (12%) 279 (88%)

3 (8%) 59 (21%)

34 (92%) 220 (79%)

320 (95)

.003*

313 (93)

.577

316 (94)

.670

309 (92)

309 (92) 316 (94)

.018*

335 (99)

.059

Predictors of an academic career on radiology residency applications.

To evaluate radiology residency applications to determine if any variables are predictive of a future academic radiology career...
92KB Sizes 0 Downloads 3 Views