Foreword
Foreword: The More Things Change, the More They Stay the Same
On behalf of the Research in Medical
Education (RIME) Program Planning Committee (PPC), we are delighted to share with you the content of the 54th annual Association of American Medical Colleges (AAMC) RIME program. For more than five decades, the RIME program has provided a snapshot of some of the best scholarship in medical education from across the globe, and this year is no exception. We had a total of 100 submissions in response to the call for papers in 2015: 87 research papers and 13 review articles. As a committee and with the help of 120 reviewers, we accepted 13 papers: 10 research papers and 3 review articles. The depth and breadth of the topics examined in this work is quite impressive, with diverse studies that span the medical education continuum. In addition, the submissions for 2015 were methodologically varied: a little over half of the research articles used quantitative methods, some used qualitative methods, and one featured a mixture of both. Finally, we were fortunate to have submissions from around the world, including the United States, Canada, the United Kingdom, Singapore, and Taiwan. Altogether, the papers accepted for this year’s RIME program represent a heterogeneous group of high-quality investigations in medical education. These studies ask and address many important questions and, like all good research, have generated additional questions and future directions for the field. As the AAMC continues to revise and reimagine how we disseminate, discuss, and review medical education scholarship—with the evolution of the annual Medical Education Meeting— the RIME PPC has maintained several important processes. At the same time, we have made a number of significant changes to how we review and disseminate RIME submissions. In the remainder of this foreword, we outline that which has remained the same and that which has changed since last year.
Acad Med. 2015;90:Si–Siii. doi: 10.1097/ACM.0000000000000899
What Has Remained the Same?
In 2003, Dr. Louis Pangaro1 outlined the major processes of the RIME PPC, including the solicitation of submissions, selection of papers, and steps leading to dissemination. Many of these processes are largely unchanged. We still solicit both research and review papers and attempt to select the best research using both external peer reviewers and internal review by members of the RIME committee. Even now, reviewing and selecting the best papers is our most challenging work. As in past years, decisions about individual submissions must take into account the relatively short timeline available for RIME manuscript revisions. Each year we have excellent papers that require major revisions and thus have to be rejected in light of our strict timeline, which allows for publication in Academic Medicine before the annual Medical Education Meeting. With this in mind, and in keeping with the RIME tradition of providing peer mentorship for our fellow scholars, we provide detailed reviews and recommend revisions on rejected papers so that authors can revise their work for submission elsewhere or for next year’s RIME program. Also unchanged from previous years are the typical reasons why we accept or reject RIME submissions. To summarize the good and the bad from 2015, we borrowed (and slightly modified) Dr. July Shea’s2 method of asking our RIME committee members to reflect on the research papers they reviewed. In particular, we asked them to provide two short lists: three features of the papers they accepted this year, and three features of the papers they rejected. The results are remarkably similar to previous findings.2,3 Starting with the good, Table 1 provides an overview of the features of RIME research articles that one might say “always impress.”2 Superior research papers in 2015 were well written, addressed relevant problems, were grounded in the relevant literature, used a conceptual framework, and employed appropriate research methods and statistical analyses. For a more in-depth review of the accepted RIME papers,
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
please see the Commentary written by three PPC members, included in this supplement.4 Conversely, rejected papers contained a number of issues, some of which were less about study design, per se, and more about writing quality and study framing (see Table 2). Many of the rejected papers did not present an adequate review of the literature, nor did they specifically identify and justify the gap in the literature being filled by their study. What’s more, many of the rejected articles failed to clearly articulate a conceptual or theoretical framework to guide the study. In addition, several were focused on issues that seemed applicable only to the institution in which they were performed and, therefore, were insufficiently generalizable. RIME committee members also noted that some rejected articles were not well written, used inappropriate statistical analyses, and overinterpreted their findings. And so, while our community and the quality of our research have certainly come a long way in the last two decades, some things are as true today as they were back in 2001 when Dr. George Bordage3 noted, “Scientific writing demands both conducting good science and writing good manuscripts.” What Has Changed?
Probably the most significant change in 2015 was our decision to publish RIME papers as a supplement of Academic Medicine, as opposed to including them in a regular issue of the journal. This change—a return to the recent past—was made with much thought and deliberation. First, we met with the editorial staff of Academic Medicine and discussed, in detail, the pros and cons of each approach. We then reached out to several colleagues in the community to gain their perspective on the issue. Finally, we conducted a citation analysis comparing four years of research papers published in the RIME supplement with regular issue research papers published during the same month in Academic Medicine. Overall, we found no statistically significant differences in the
Si
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Foreword
Table 1 Features of RIME Submissions That Were Accepted in 2015 Frequency of response, no. (%)a
Feature Important, timely, relevant problem or identified gap in the literature Appropriate/innovative methodology for the research question(s) or study purpose
5 (63) 5 (63)
Well-written manuscript
3 (38)
Clear conceptual or theoretical framework
2 (25)
High-quality discussion that provides additional insight into the problem
2 (25)
Findings that build on existing literature
1 (13)
Findings that generalize beyond a single institution
1 (13)
Practical/useful implications
1 (13)
Well-written review/synthesis of the literature
1 (13)
High-quality outcome measures (e.g., patient outcomes)
1 (13)
Column numbers represent the number and percentage of the total sample of eight RIME committee members who provided a particular response. Some committee members listed fewer than three features.
a
citation rates between the two groups of papers. With these data in hand, and after much debate, we decided to go back to the tradition of publishing RIME papers in a supplement of Academic Medicine. In making this decision, the RIME committee identified several benefits to the medical education community. First, this change will improve dissemination— an important component of scholarship5—by increasing the total number of medical education research articles published in the journal each year. When published in a regular issue of Academic Medicine, the number of
RIME papers that could be accepted was restricted by the physical space available in the journal. Moreover, with RIME papers filling a regular issue of Academic Medicine, the publication of regularly scheduled research articles had to be delayed. By putting RIME papers back into a supplement, we have alleviated this problem, which effectively increases the total number of medical education research articles that can be published in the journal each year. This change also allowed the RIME PPC to loosen the 3,000-word restriction and consider longer articles in 2015 (up to a maximum of 4,000 words).
A second, more practical benefit has to do with timing and editorial independence. The RIME PPC has a long-standing tradition of autonomy with respect to the content of the program and the timeline for submissions. When RIME papers were included as part of a regular issue, the result was a substantial increase in both the review and processing times necessary given the workload of the Academic Medicine staff. When RIME papers are published as a supplement, the timeline for the call becomes more flexible: it gives reviewers more time for reviews and authors more time for revisions. Moreover, using the supplement model gives the program committee a bit more time, and some added editorial independence, when attempting to select those research papers most applicable for the RIME program. Nonetheless, all papers published in the supplement still require editorial review by the journal’s expert staff, which is the final qualitycontrol procedure prior to publication. Finally, the move back to a supplement allows Academic Medicine to provide access to all RIME articles on the Web and on the journal’s mobile app free of charge to readers. Consistent with recent research on the positive effects of openaccess publishing,6 the RIME committee expects that free access to RIME papers will increase future citation rates. In sum, considering the benefits described above, the RIME committee determined that returning to a supplement format is ideal for 2015 and beyond.
Lack of a conceptual or theoretical framework Insufficient generalizability (e.g., single site, small sample, limited outcomes)
5 (63) 5 (63)
Inadequate/incomplete review of the literature
3 (38)
Poorly written manuscript
3 (38)
Unimportant results with questionable applicability to medical education
2 (25)
Method disconnected from research question(s) or study purpose
1 (13)
Unclear framing and importance of the research question(s)
1 (13)
Overinterpretation of results
1 (13)
Inappropriate/incomplete statistical analysis for the research question(s) and data collected
1 (13)
Failure to follow accepted methodological standards (e.g., PRISMA guidelines for review articles)
1 (13)
Another important change for 2015 has to do with how we communicate expectations to our peer reviewers. Anyone who has ever served on an editorial board knows that getting high-quality reviews is critical to making good editorial decisions. Unfortunately, not every review received by the RIME committee met our expectations. To enhance the quality of future reviews, this year we provided reviewers feedback on the quality of their peer reviews. One way to accomplish this goal is to “carbon copy” reviewers on manuscript decisions, with all reviewer comments included. We have employed this technique, one that is commonly used by most medical education journals, and will continue to do so for the foreseeable future.
Column numbers represent the number and percentage of the total sample of eight RIME committee members who provided a particular response. Some committee members listed fewer than three features.
One last change to this year’s review process relates to research abstracts and symposia.
Table 2 Features of RIME Submissions That Were Rejected in 2015
Feature
a
Sii
Frequency of response, no. (%)a
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Foreword
In past years, the RIME committee reviewed and selected the research abstracts and symposia that would be presented as part of the RIME program. This year, the AAMC’s Medical Education Meeting Planning Committee performed this task, which gave the RIME PPC more time to devote to the review and selection of research and review papers. Of note, a change that was initiated in 2014 places the RIME chair on the Medical Education Meeting Planning Committee, which allows the RIME chair to assist with the selection of research abstracts and symposia. In closing, we are grateful for several individuals and groups who helped make this year’s RIME program a reality. First, we thank the dedicated and superbly talented AAMC staff. Without them, the committee simply could not do its work efficiently. Nesha Brown, Steve McKenzie, and Kate McOwen worked tirelessly with us and with our reviewers and authors. Their attention to detail ensured that we met each and every deadline. In addition, we thank our external peer reviewers. Having been RIME reviewers ourselves, we understand the time and effort required to conduct thorough reviews
and provide constructive feedback, both of which are invaluable to the program committee and the individual authors. We also thank the incredible scholars who compose the RIME PPC. It is truly an honor and a pleasure to work with these colleagues. The planning committee comprises individuals who are genuinely committed to promoting the art and science of medical education: Drs. John “Jack” Boulet, Monica Lypson, Lynne Robins, Bonnie Miller, Reena Karani, and Karen Miller. We offer special thanks to Dr. Ann Frye, who graciously stepped in this year to serve as our acting “past chair,” in place of Dr. Maryellen Gusic, who now serves as the chief medical education officer at the AAMC. Finally, we thank Dr. David Sklar, Anne Farmakidis, Mary Beth DeVilbiss, and the rest of the editorial team at Academic Medicine. We sincerely appreciate their continued support of medical education research and the RIME program. They provide a prestigious venue through which our community of scholars can publish scholarship that advances the theory, research, and practice of medical education.
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Disclaimer: The views expressed are those of the authors and do not necessarily reflect the official views of the Uniformed Services University of the Health Sciences, the U.S. Navy, or the Department of Defense. Anthony R. Artino Jr., PhD, Daniel C. West, MD, and Maryellen E. Gusic, MD
References 1 Pangaro L. Foreword. Acad Med. 2003;78(10 suppl):Siii–Siv. 2 Shea JA. Foreword. Acad Med. 2010;85(10 suppl):Si–Sii. 3 Bordage G. Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Acad Med. 2001;76:889–896. 4 Miller KH, Miller BM, Karani R. Considering research outcomes as essential tools for medical education decision making. Acad Med. 2015;90(11 suppl):S1–S4. 5 Shulman L. Taking learning seriously. Change. 1999;31:10–17. 6 Norris M, Oppenheim C, Rowland F. The citation advantage of open-access articles. J Am Soc Inf Sci Technol. 2008;59:1963–1972.
Siii
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Foreword: The More Things Change, the More They Stay the Same
On behalf of the Research in Medical
Education (RIME) Program Planning Committee (PPC), we are delighted to share with you the content of the 54th annual Association of American Medical Colleges (AAMC) RIME program. For more than five decades, the RIME program has provided a snapshot of some of the best scholarship in medical education from across the globe, and this year is no exception. We had a total of 100 submissions in response to the call for papers in 2015: 87 research papers and 13 review articles. As a committee and with the help of 120 reviewers, we accepted 13 papers: 10 research papers and 3 review articles. The depth and breadth of the topics examined in this work is quite impressive, with diverse studies that span the medical education continuum. In addition, the submissions for 2015 were methodologically varied: a little over half of the research articles used quantitative methods, some used qualitative methods, and one featured a mixture of both. Finally, we were fortunate to have submissions from around the world, including the United States, Canada, the United Kingdom, Singapore, and Taiwan. Altogether, the papers accepted for this year’s RIME program represent a heterogeneous group of high-quality investigations in medical education. These studies ask and address many important questions and, like all good research, have generated additional questions and future directions for the field. As the AAMC continues to revise and reimagine how we disseminate, discuss, and review medical education scholarship—with the evolution of the annual Medical Education Meeting— the RIME PPC has maintained several important processes. At the same time, we have made a number of significant changes to how we review and disseminate RIME submissions. In the remainder of this foreword, we outline that which has remained the same and that which has changed since last year.
Acad Med. 2015;90:Si–Siii. doi: 10.1097/ACM.0000000000000899
What Has Remained the Same?
In 2003, Dr. Louis Pangaro1 outlined the major processes of the RIME PPC, including the solicitation of submissions, selection of papers, and steps leading to dissemination. Many of these processes are largely unchanged. We still solicit both research and review papers and attempt to select the best research using both external peer reviewers and internal review by members of the RIME committee. Even now, reviewing and selecting the best papers is our most challenging work. As in past years, decisions about individual submissions must take into account the relatively short timeline available for RIME manuscript revisions. Each year we have excellent papers that require major revisions and thus have to be rejected in light of our strict timeline, which allows for publication in Academic Medicine before the annual Medical Education Meeting. With this in mind, and in keeping with the RIME tradition of providing peer mentorship for our fellow scholars, we provide detailed reviews and recommend revisions on rejected papers so that authors can revise their work for submission elsewhere or for next year’s RIME program. Also unchanged from previous years are the typical reasons why we accept or reject RIME submissions. To summarize the good and the bad from 2015, we borrowed (and slightly modified) Dr. July Shea’s2 method of asking our RIME committee members to reflect on the research papers they reviewed. In particular, we asked them to provide two short lists: three features of the papers they accepted this year, and three features of the papers they rejected. The results are remarkably similar to previous findings.2,3 Starting with the good, Table 1 provides an overview of the features of RIME research articles that one might say “always impress.”2 Superior research papers in 2015 were well written, addressed relevant problems, were grounded in the relevant literature, used a conceptual framework, and employed appropriate research methods and statistical analyses. For a more in-depth review of the accepted RIME papers,
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
please see the Commentary written by three PPC members, included in this supplement.4 Conversely, rejected papers contained a number of issues, some of which were less about study design, per se, and more about writing quality and study framing (see Table 2). Many of the rejected papers did not present an adequate review of the literature, nor did they specifically identify and justify the gap in the literature being filled by their study. What’s more, many of the rejected articles failed to clearly articulate a conceptual or theoretical framework to guide the study. In addition, several were focused on issues that seemed applicable only to the institution in which they were performed and, therefore, were insufficiently generalizable. RIME committee members also noted that some rejected articles were not well written, used inappropriate statistical analyses, and overinterpreted their findings. And so, while our community and the quality of our research have certainly come a long way in the last two decades, some things are as true today as they were back in 2001 when Dr. George Bordage3 noted, “Scientific writing demands both conducting good science and writing good manuscripts.” What Has Changed?
Probably the most significant change in 2015 was our decision to publish RIME papers as a supplement of Academic Medicine, as opposed to including them in a regular issue of the journal. This change—a return to the recent past—was made with much thought and deliberation. First, we met with the editorial staff of Academic Medicine and discussed, in detail, the pros and cons of each approach. We then reached out to several colleagues in the community to gain their perspective on the issue. Finally, we conducted a citation analysis comparing four years of research papers published in the RIME supplement with regular issue research papers published during the same month in Academic Medicine. Overall, we found no statistically significant differences in the
Si
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Foreword
Table 1 Features of RIME Submissions That Were Accepted in 2015 Frequency of response, no. (%)a
Feature Important, timely, relevant problem or identified gap in the literature Appropriate/innovative methodology for the research question(s) or study purpose
5 (63) 5 (63)
Well-written manuscript
3 (38)
Clear conceptual or theoretical framework
2 (25)
High-quality discussion that provides additional insight into the problem
2 (25)
Findings that build on existing literature
1 (13)
Findings that generalize beyond a single institution
1 (13)
Practical/useful implications
1 (13)
Well-written review/synthesis of the literature
1 (13)
High-quality outcome measures (e.g., patient outcomes)
1 (13)
Column numbers represent the number and percentage of the total sample of eight RIME committee members who provided a particular response. Some committee members listed fewer than three features.
a
citation rates between the two groups of papers. With these data in hand, and after much debate, we decided to go back to the tradition of publishing RIME papers in a supplement of Academic Medicine. In making this decision, the RIME committee identified several benefits to the medical education community. First, this change will improve dissemination— an important component of scholarship5—by increasing the total number of medical education research articles published in the journal each year. When published in a regular issue of Academic Medicine, the number of
RIME papers that could be accepted was restricted by the physical space available in the journal. Moreover, with RIME papers filling a regular issue of Academic Medicine, the publication of regularly scheduled research articles had to be delayed. By putting RIME papers back into a supplement, we have alleviated this problem, which effectively increases the total number of medical education research articles that can be published in the journal each year. This change also allowed the RIME PPC to loosen the 3,000-word restriction and consider longer articles in 2015 (up to a maximum of 4,000 words).
A second, more practical benefit has to do with timing and editorial independence. The RIME PPC has a long-standing tradition of autonomy with respect to the content of the program and the timeline for submissions. When RIME papers were included as part of a regular issue, the result was a substantial increase in both the review and processing times necessary given the workload of the Academic Medicine staff. When RIME papers are published as a supplement, the timeline for the call becomes more flexible: it gives reviewers more time for reviews and authors more time for revisions. Moreover, using the supplement model gives the program committee a bit more time, and some added editorial independence, when attempting to select those research papers most applicable for the RIME program. Nonetheless, all papers published in the supplement still require editorial review by the journal’s expert staff, which is the final qualitycontrol procedure prior to publication. Finally, the move back to a supplement allows Academic Medicine to provide access to all RIME articles on the Web and on the journal’s mobile app free of charge to readers. Consistent with recent research on the positive effects of openaccess publishing,6 the RIME committee expects that free access to RIME papers will increase future citation rates. In sum, considering the benefits described above, the RIME committee determined that returning to a supplement format is ideal for 2015 and beyond.
Lack of a conceptual or theoretical framework Insufficient generalizability (e.g., single site, small sample, limited outcomes)
5 (63) 5 (63)
Inadequate/incomplete review of the literature
3 (38)
Poorly written manuscript
3 (38)
Unimportant results with questionable applicability to medical education
2 (25)
Method disconnected from research question(s) or study purpose
1 (13)
Unclear framing and importance of the research question(s)
1 (13)
Overinterpretation of results
1 (13)
Inappropriate/incomplete statistical analysis for the research question(s) and data collected
1 (13)
Failure to follow accepted methodological standards (e.g., PRISMA guidelines for review articles)
1 (13)
Another important change for 2015 has to do with how we communicate expectations to our peer reviewers. Anyone who has ever served on an editorial board knows that getting high-quality reviews is critical to making good editorial decisions. Unfortunately, not every review received by the RIME committee met our expectations. To enhance the quality of future reviews, this year we provided reviewers feedback on the quality of their peer reviews. One way to accomplish this goal is to “carbon copy” reviewers on manuscript decisions, with all reviewer comments included. We have employed this technique, one that is commonly used by most medical education journals, and will continue to do so for the foreseeable future.
Column numbers represent the number and percentage of the total sample of eight RIME committee members who provided a particular response. Some committee members listed fewer than three features.
One last change to this year’s review process relates to research abstracts and symposia.
Table 2 Features of RIME Submissions That Were Rejected in 2015
Feature
a
Sii
Frequency of response, no. (%)a
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Foreword
In past years, the RIME committee reviewed and selected the research abstracts and symposia that would be presented as part of the RIME program. This year, the AAMC’s Medical Education Meeting Planning Committee performed this task, which gave the RIME PPC more time to devote to the review and selection of research and review papers. Of note, a change that was initiated in 2014 places the RIME chair on the Medical Education Meeting Planning Committee, which allows the RIME chair to assist with the selection of research abstracts and symposia. In closing, we are grateful for several individuals and groups who helped make this year’s RIME program a reality. First, we thank the dedicated and superbly talented AAMC staff. Without them, the committee simply could not do its work efficiently. Nesha Brown, Steve McKenzie, and Kate McOwen worked tirelessly with us and with our reviewers and authors. Their attention to detail ensured that we met each and every deadline. In addition, we thank our external peer reviewers. Having been RIME reviewers ourselves, we understand the time and effort required to conduct thorough reviews
and provide constructive feedback, both of which are invaluable to the program committee and the individual authors. We also thank the incredible scholars who compose the RIME PPC. It is truly an honor and a pleasure to work with these colleagues. The planning committee comprises individuals who are genuinely committed to promoting the art and science of medical education: Drs. John “Jack” Boulet, Monica Lypson, Lynne Robins, Bonnie Miller, Reena Karani, and Karen Miller. We offer special thanks to Dr. Ann Frye, who graciously stepped in this year to serve as our acting “past chair,” in place of Dr. Maryellen Gusic, who now serves as the chief medical education officer at the AAMC. Finally, we thank Dr. David Sklar, Anne Farmakidis, Mary Beth DeVilbiss, and the rest of the editorial team at Academic Medicine. We sincerely appreciate their continued support of medical education research and the RIME program. They provide a prestigious venue through which our community of scholars can publish scholarship that advances the theory, research, and practice of medical education.
Academic Medicine, Vol. 90, No. 11 / November Supplement 2015
Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Disclaimer: The views expressed are those of the authors and do not necessarily reflect the official views of the Uniformed Services University of the Health Sciences, the U.S. Navy, or the Department of Defense. Anthony R. Artino Jr., PhD, Daniel C. West, MD, and Maryellen E. Gusic, MD
References 1 Pangaro L. Foreword. Acad Med. 2003;78(10 suppl):Siii–Siv. 2 Shea JA. Foreword. Acad Med. 2010;85(10 suppl):Si–Sii. 3 Bordage G. Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Acad Med. 2001;76:889–896. 4 Miller KH, Miller BM, Karani R. Considering research outcomes as essential tools for medical education decision making. Acad Med. 2015;90(11 suppl):S1–S4. 5 Shulman L. Taking learning seriously. Change. 1999;31:10–17. 6 Norris M, Oppenheim C, Rowland F. The citation advantage of open-access articles. J Am Soc Inf Sci Technol. 2008;59:1963–1972.
Siii
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
