Departmental Administration
A Guide to the External Review of an Academic Radiology Department Jannette Collins, MD, MEd, FCCP, FACR, E. Stephen Amis, Jr., MD, FACR, Norman J. Beauchamp, Jr., MD, MHS, Alexander M. Norbash, MD, MS, Carolyn C. Meltzer, MD, FACR, from the Society of Chairs of Academic Radiology Departments (SCARD) External reviews are used to evaluate a department on a routine basis or prior to reappointment or recruitment of a department chair. The Society of Chairs of Academic Radiology Departments (SCARD) developed a template that outlines important components of an external review report and a table that outlines the objective information that can be requested from the institution/department prior to the reviewer’s site visit. The template is meant to facilitate a high-quality review and serve as a guide to a chair who is preparing for his/her first review, chairs who serve as external consultants, and institutional officials seeking review of a radiology department. Key Words: Radiology Department Review; External Review. ªAUR, 2014
PURPOSE OF EXTERNAL REVIEWS
E
xternal reviews are used to evaluate a department on a routine basis or before reappointment or recruitment of a department chair. The goals of such a review are to: (1) document the department’s achievements and progress, (2) assess gaps, opportunities, and strengths as a framework for strategic planning, (3) establish goals and priorities for the future, (4) foster continuous improvement of programs and services, (5) provide feedback to the department regarding its performance and the relationship of that performance with institutional expectations and national benchmarks, (6) educate the institution as to the status, directions, and needs of the department, (7) provide expert advice to both the institution and the department, (8) provide a mechanism by which the members of a department can express their views concerning the competence and responsiveness of the chair (and other department leaders) and confidence in their leadership, and (9) serve as the basis for dialog between the chair, dean, and hospital leadership (1). Chairs of academic radiology departments are accountable for their department’s performance in meeting the tripartite mission of patient care, education, and research. Depending on the particular institu-
Acad Radiol 2014; 21:400–406 From the Department of Radiology, University of Cincinnati College of Medicine, 234 Goodman St, 670761 Cincinnati, OH (J.C.); Department of Radiology, Albert Einstein College of Medicine, Bronx, NY (E.S.A.); Department of Radiology, University of Washington College of Medicine, Seattle, WA (N.J.B.); Department of Radiology, Boston University School of Medicine, Boston, MA (A.M.N.); Department of Radiology and Imaging Sciences; Neurology; and Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, GA (C.C.M.). Received November 22, 2013; accepted November 23, 2013. Address correspondence to: J.C. e-mail: [email protected] ªAUR, 2014 http://dx.doi.org/10.1016/j.acra.2013.11.020
400
tion, a wide variety of stated and unstated expectations exist regarding the performance targets in each of these three domains. Many radiology chairs are asked to serve as external consultants to review departments at other medical schools. Participating in external reviews can be viewed by the chair as an important service to the radiology community, a mechanism to provide guidance or mentorship to a fellow leader in radiology, an opportunity for self-development and to compare one’s own department with others, and to learn things that may benefit the reviewer’s department. The Society of Chairs of Academic Radiology Departments (SCARD) views external reviews as a tool that when optimally used can foster the success of an individual chair and academic department and ultimately contribute to continual process improvement across the field of radiology. SCARD developed a template (Appendix) that outlines important components of an external review report and a table (Table 1) that outlines the objective information that can be requested from the institution/department before the reviewer’s site visit. The template is meant to facilitate a highquality review process by outlining standardized elements based on the best practices. It may further serve as a guide to a chair who is preparing for his/her first review, chairs who serve as external consultants, and institutional officials seeking a review of a radiology department. Although no approach fits all institutional needs, the review process should reflect the unique context and individual circumstances of the institution and the department under review, and the availability of a comprehensive standardized template can be valuable. Objective criteria are critical for measuring a department’s progress and comparing a department’s performance with national benchmarks. These objective metrics should be considered in concert with subjective data to create a report that reflects a department’s strengths and weaknesses
Academic Radiology, Vol 21, No 3, March 2014
GUIDE TO CONDUCT A DEPARTMENT REVIEW
TABLE 1. Information to Request Before Radiology Department Review Site Visit Overview Chair’s personal performance
Organizational structure
Financial performance
Clinical performance
Department Chair self-evaluation, summary of department changes over the past 5 years, including both current and recent past departmental goals and major initiatives Clinical activity (type and schedule) Scholarly activity (number and brief description of grants, publications, national presentations, and teaching activities) Service activity (number and description of local, regional, and national committees, society officerships, editorial boards, and study sections) Faculty full-time-equivalents (FTEs) (number and percentage of women and other minorities and rank distribution) Curriculum vitae of all faculty members Department organizational chart showing the titles and names of the Chair, Vice Chairs, Directors, Section Chiefs, other faculty, and staff, and their reporting structure List of department committees Roles of department members in hospital and medical school committees (e.g., promotions, medical executive, quality ,and safety), including any reporting structures of departmental faculty to extradepartmental entities Description of how faculty mentoring is provided and how professional development opportunities are communicated to faculty Description of formal faculty evaluation process Methods of communication (e.g., department website, department newsletter, department e-mail notices, department digital signage, faculty meetings, and external communication outlets) List of department support personnel (e.g., business administrator, administrative assistants, billing/coding staff, department budget analyst, department research assistants, program coordinators, and information technology support staff) Description of faculty clinical scheduling process and sample schedules Vacation and meeting policy Description of faculty academic time (education, research, scholarship, and administrative) List of revenue sources (e.g., clinical, college, hospital, state, grants, endowments, other) Description of department budgeting process Five-year history of actual revenues and expenses versus budget Description of department reserves, endowments, and ongoing philanthropic efforts Faculty compensation plan document (description of plan/process, components such as base/ incentive/call coverage) Mean salaries for each rank compared to national benchmarks Travel and expense allowance policy Faculty work relative value units adjusted for FTE, per section and department total, compared to national benchmarks (e.g., Medical Group Management Association [MGMA], University HealthSystem Consortium [UHC], and Association of Administrators in Academic Radiology [AAARAD]/Society of Chairs of Academic Radiology Departments) Number of examinations by modality and site for past 5 years Description of the range of diagnostic and therapeutic services and any deficiencies Method of 24/7/365 faculty coverage (e.g., faculty available on or off-site, teleradiology) Equipment inventory with dates of purchase Description of capital budget process Equipment purchase and maintenance process Radiology information system, picture archiving and communication system, and voice recognition equipment and support Description of how physical space and facilities are maintained and renovated Description of quality, safety, and efficiency initiatives (e.g., American College of Radiology [ACR] accreditation, dose reduction and monitoring, ACR General Radiology Information Database, days to appointment, complication rate, decision support integrated with computerized physician order entry, and meaningful use participation) Patient satisfaction survey results for past 2 years Ordering provider satisfaction survey results for past 2 years Samples of department dashboards Report of department report turnaround times for past 2 years (Continued)
401
COLLINS ET AL
Academic Radiology, Vol 21, No 3, March 2014
TABLE 1. (Continued) Information to Request Before Radiology Department Review Site Visit Educational programs
Research/scholarly activity
Institutional model for funding teaching Description of medical student clerkship (mandatory or voluntary, length, and curriculum) Medical student evaluations of clerkship for past 2 years Description of core residency program (number approved and current number, funding source/s) The most recent Radiology Review Committee report The most recent resident survey results The most recent internal residency program review Residency match results for past 5 years American Board of Radiology pass/condition rate List of resident publications and presentations over past 5 years Description of fellowship programs (accredited versus nonaccredited, type and number, number of fellows in each program over past 5 years, the most recent RRC site visit reports, number of fellow publications and presentations over past 5 years) Description of department-sponsored Continuing Education Programs Description of laboratories and other dedicated research space List of grants over past 5 years (federal, foundation, society, and internal) to include salary support and percent effort Submitted grants in past year Number of faculty presentations in past 5 years Number of faculty publications in past 5 years (listed as peer-reviewed and nonpeer-reviewed) Number of faculty committee memberships (local, regional, and national), society officerships, editorial board memberships, and study section memberships over past 5 years Awards, honors, and distinctions of faculty/department/staff/students
and provides specific actionable recommendations for performance improvement. A complete description of the operational aspects of a radiology department and metrics that can be used to judge a department’s performance are beyond the scope of this article and can be found elsewhere (2). A detailed process for radiology services review is also available from the Royal College of Radiologists in London, England (3,4).
REVIEW PROCESS The ‘‘classic’’ department review typically occurs at regular intervals (usually every 5–7 years), is directed by the medical school dean, and is conducted by both an internal review committee and external advisors who produce a formal report (1). Internal and external reviewers can work separately or as a combined committee. Some schools of medicine have begun to rethink the traditional departmental review process, citing ‘‘too much data and too little plan’’ (1). Other criticisms of reviews have included a little long-lasting benefit as perceived by the department and dean’s office, lack of follow up of the report’s recommendations, ineffective strategic planning resulting from the retrospective analysis of the department, and limited incentive by reviewers to document weaknesses, if the review is seen primarily as an opportunity to request additional resources. Additionally, reviews are costly to an institution in terms of actual expense and staff time. The Stanford University School of Medicine estimated the average
402
cost per review to be $20,000 for honoraria, travel, and supplies, and $20,000 for dean’s office staff time; the additional costs of Radiology department faculty and staff time was not calculated (1). The University of Michigan Medical School estimated the annual cost of conducting reviews to be up to $240,000 (1). In an effort to gain a greater return on investment, they revised their departmental review process to link strategic goals, performance measures, evaluation systems, and rewards (1). The final report is a strategic plan containing a SWOT (strengths, weaknesses, opportunities, and threats) analysis and strategic goals for the department, with related action plans. The Michigan model employs outside experts from the same as well as from a different medical discipline and from a nonmedical discipline (e.g., strategic planning expert). The review of a department should include an assessment of the overall leadership and management performance of the department chair, presence of good faculty development practices, integrity of the teaching programs, excellence of the research program and associated research infrastructure, and quality and effectiveness of the clinical programs (5). Similarly, there should be a stated understanding of favorable or unfavorable trends in the department, including the evolving nature and structure of the department and its effectiveness over time in addressing the tripartite mission. Although chair job expectations may vary from institution to institution, there is a general agreement on the main areas of accountability. The American Association of Chairs of Departments of Psychiatry established a tool kit for new
Academic Radiology, Vol 21, No 3, March 2014
chairs that included key issues and responsibilities of a chair and benchmarking information on other psychiatry departments nationally (6). In evaluating the department chair, performance dimensions can include general administrative and leadership effectiveness, his/her approach to professional and staff development and management, financial stewardship, academic leadership, and ability to promote good working relationships with other departmental and institutional leaders. The effectiveness should be assessed across the domains of clinical excellence, teaching, and research. The assessment should also include the chair’s attention to gender climate, hiring and maintaining underrepresented minorities, and proportion of women among department leaders. A selfevaluation and/or 360-degree feedback may be used. The five-year departmental review process at the University of North Carolina School of Medicine starts with a self-study generated by the department Chair. An internal review committee interviews and receives comments from at least two departmental professors, two associate professors, and two assistant professors who have reviewed the self-study. Written comments from all faculty members are invited (1). At the University of Wisconsin Medical School, all members of each department are given the opportunity to express their preference for the Chair each year by secret ballot. This ballot is advisory to the dean and is used in the annual evaluation of department Chairs (1). At Emory University School of Medicine, the dean conducts a formal anonymous survey of the faculty annually that targets how effectively their chair supports professional development. The results are shared with each chair during his/her annual performance review (personal communication: Sharon Weiss, MD, Emory University School of Medicine). External reviewers should receive information from the department to be reviewed before conducting the review. The reviewer may need to request additional information that is not initially provided (see Table 1). After reviewing this information, a preliminary report can be prepared that will help guide the reviewer’s on-site evaluation. The SCARD report template can be prepopulated before the site visit (Appendix), enabling the reviewer to use his/her time most efficiently while conducting on-site interviews. If more than one external reviewer is involved, one should be selected to serve as the chair of the external review committee to distribute tasks efficiently. The reviewer should be provided with an itinerary that outlines a schedule of interviews with the medical school dean(s), radiology chair, chairs from other departments, hospital executive(s), faculty practice plan executive(s), department faculty (including vice chairs and section chiefs), residents, and the department business administrator. There may be additional individuals the reviewer will want to talk to, such as educational program directors and coordinators and research assistants. The previsit information packet may uncover areas of concern or opportunity that suggest benefit
GUIDE TO CONDUCT A DEPARTMENT REVIEW
from meeting with specific individuals that are not on the itinerary. It is advisable to meet with the individual(s) requesting the review at the start of the visit as a mechanism to clarify the institutional goals. In addition, the authors have found it valuable to meet with the chair early in the review to elucidate his/ her concerns and insights and again late in the visit to clarify issues or opportunities uncovered during the visit. The site review is usually conducted over 1–2 days and should include a tour of the department. The time provided may be too limited to meet with everyone. Reviewers may want to have discussions in parallel or to coordinate a few phone interviews before or after the visit. The site visit will often include a ‘‘wrap-up’’ session when reviewers provide senior leadership with a preliminary summary. The reviewers should discuss their thoughts before the wrap-up to facilitate a summary that addresses the most challenging issues and needs for a department. After the site visit, the reviewer should complete the report template, write a summary of the department’s strengths, weaknesses, and opportunities, and make specific constructive and realistic recommendations based on the subjective and objective data obtained. When appropriate, these recommendations should include requests for additional support and resources for the department, team building activities between department and organizational leaders, and additional leadership training or mentorship for the chair (7). The report should be submitted as soon as possible, generally within 3 weeks of the visit. Most institutions will expect the report and any information provided to the reviewer to be kept confidential.
SUMMARY Department reviews require careful preparation, active listening, the ability to respond unemotionally to issues that strike chords in oneself, and, most of all, the ability to distill extensive subjective and objective information into a clear plan of action (1). SCARD members have worked to develop a standardized template to guide highly effective department reviews and to promote the best practices throughout the academic radiology community. The reviewer is rewarded with a sense of being appreciated, an opportunity to meet new colleagues, and a deepened perspective of both the breadth and value of their specialty to medical schools and the community. Performing reviews is something to which chairs should aspire, if only because they will gain as much as they give (1).
REFERENCES 1. Association of American Medical Colleges. The successful medical school department chair: a guide to good institutional practice, Module 3: Performance, evaluation, rewards, renewal, Chapter 2, Examples of good practice in departmental reviews. Washington, DC: Association of American Medical Colleges, 2003.
403
COLLINS ET AL
2. Yousem DM, Beauchamp NJ Jr, eds. Radiology business practice: how to succeed. Philadelphia, PA: Saunders/Elsevier, 2008. 3. Board of the Faculty of Clinical Radiology. The Royal College of Radiologists, College review of radiology services, 3rd ed. The Royal College of Radiologists. http://www.rcr.ac.uk/docs/radiology/pdf/College_review_ third_edn.pdf. Accessed February 2012. 4. The process for service reviews conducted by the service review committee. Board of the Faculty of Clinical Radiology. The Royal College of Radi-
404
Academic Radiology, Vol 21, No 3, March 2014
ologists, http://www.rcr.ac.uk/docs/radiology/pdf/Process_SRC_final.pdf; 2012. Accessed August 10, 2013. 5. Hitchcock MA, Stritter FT, Bland CJ. Faculty development in the health professions: conclusions and recommendations. Med Teach 1992; 14:295–309. 6. Monro SA. Tool kit for new chairs. Acad Psychiatry 2006; 30. 7. Wolverton M, Ackerman R, Holt S. Preparing for leadership: what academic department chairs need to know. J High Educ Policy Manage 2013; 27: 227–238.
Academic Radiology, Vol 21, No 3, March 2014
GUIDE TO CONDUCT A DEPARTMENT REVIEW
APPENDIX. RADIOLOGY DEPARTMENT REVIEW REPORT TEMPLATE
Department Finances
Overview
Revenue sources (clinical, college, hospital, state, grants, other) Expenses, including overhead Budget process Five-year history actuals versus budget Reserves/endowments/ongoing philanthropic efforts Faculty compensation plan (description of plan/process, components such as base/incentive/call coverage, salaries compared to national benchmarks) Travel and expense allowance policy Hospital and medical school funding for clinical shortfall and for administrative roles performed by the department for the benefit of the organization
Brief history Chair/department relationship to practice plan, medical school, hospital(s), and university Changes in past 5 years Current department goals (clinical/financial/educational/ research/scholarship, described as measurable objectives)
Chair’s Personal Performance
Years as Chair, title, and rank Clinical activity Scholarly activities (grants, publications, national presentations, and teaching) Service (local, regional, and national committees, society officership, editorial boards, and study sections) Department/institution evaluation of leadership Organizational Structure
Faculty (full-time-equivalents [FTEs], #/% women and other minorities, rank distribution) Department organizational chart Faculty governance (vice chairs, section chiefs, clinical directors, educational directors, and committees) and associated protected time to serve leadership roles Hospital and medical school governance and the role of the chair in key decision making committees Mentoring/professional development opportunities for leadership team and individual faculty Formal faculty evaluation process Communication (department website, department newsletter, department e-mail notices, department digital signage, faculty meetings, individual member and group leadership team meetings with the chair, meetings with key support personnel [e.g., weekly meetings with departmental administrator, monthly meetings with billing and coding team, individual meetings with faculty members, and chair interactions with the residents] and external communication outlets) Support personnel (administrative assistants, department IT, department coding/billing, department budget analyst, department business administrator, department research assistants) Faculty clinical scheduling process Vacation and meeting policy Faculty academic time (education, research/scholarship, and administrative)
Clinical Performance
Faculty work relative value units adjusted for FTE, compared to national benchmarks Volume by modality and site past 5 years Range of diagnostic and therapeutic services/deficiencies Service opportunities and their barriers (e.g., poor access, perceived quality issues, resource or expertise limitations, and inappropriate incentives for referral outside the institution) Method of 24/7/365 coverage Equipment (inventory/deficiencies, capital budget process, and equipment purchase and maintenance process) Radiology information system/picture archiving and communication system/voice recognition equipment and support Maintenance of physical space/facilities Quality/safety/efficiency initiatives (peer review, Image Gently, Image Wisely, ACR CT Dose Index Registry, ACR accreditation, ACR General Radiology Information Database, report turnaround time, days to appointment, complication rate, patient satisfaction surveys, ordering provider satisfaction surveys, decision support integrated with computerized physician order entry, and meaningful use participation)
Educational Programs
Medical student clerkship (mandatory or voluntary, length, curriculum, and student ratings) Residency program (number approved/current number, funding source, the most recent RRC site visit report, the most recent resident survey results, annual internal pro-
405
COLLINS ET AL
Academic Radiology, Vol 21, No 3, March 2014
gram review, match results past 5 years, ABR pass/condition rate, and scholarly activity) Fellowship programs (type and number past 5 years, funding source, accredited/nonaccredited, the most recent RRC site visit report, scholarly activity) Continuing education (formal continuing medical education lectures/courses)
Service (local, regional and national committees, society officership, editorial boards, and study sections) Departmental cores provided to support research Research training and mentorship
Research/Scholarly Activity
Lab/other dedicated research space Grants (federal, foundation, society, internal, percent salary support, and percent faculty effort) National presentations past 5 years Publications past 5 years
406
Tools and Templates (the examples listed are not meant to represent a complete list)
Staffing grid to guide recruitment Electronic professional activities report Structured agendas Dashboards Summary and Assessment. Recommendations.
A Guide to the External Review of an Academic Radiology Department Jannette Collins, MD, MEd, FCCP, FACR, E. Stephen Amis, Jr., MD, FACR, Norman J. Beauchamp, Jr., MD, MHS, Alexander M. Norbash, MD, MS, Carolyn C. Meltzer, MD, FACR, from the Society of Chairs of Academic Radiology Departments (SCARD) External reviews are used to evaluate a department on a routine basis or prior to reappointment or recruitment of a department chair. The Society of Chairs of Academic Radiology Departments (SCARD) developed a template that outlines important components of an external review report and a table that outlines the objective information that can be requested from the institution/department prior to the reviewer’s site visit. The template is meant to facilitate a high-quality review and serve as a guide to a chair who is preparing for his/her first review, chairs who serve as external consultants, and institutional officials seeking review of a radiology department. Key Words: Radiology Department Review; External Review. ªAUR, 2014
PURPOSE OF EXTERNAL REVIEWS
E
xternal reviews are used to evaluate a department on a routine basis or before reappointment or recruitment of a department chair. The goals of such a review are to: (1) document the department’s achievements and progress, (2) assess gaps, opportunities, and strengths as a framework for strategic planning, (3) establish goals and priorities for the future, (4) foster continuous improvement of programs and services, (5) provide feedback to the department regarding its performance and the relationship of that performance with institutional expectations and national benchmarks, (6) educate the institution as to the status, directions, and needs of the department, (7) provide expert advice to both the institution and the department, (8) provide a mechanism by which the members of a department can express their views concerning the competence and responsiveness of the chair (and other department leaders) and confidence in their leadership, and (9) serve as the basis for dialog between the chair, dean, and hospital leadership (1). Chairs of academic radiology departments are accountable for their department’s performance in meeting the tripartite mission of patient care, education, and research. Depending on the particular institu-
Acad Radiol 2014; 21:400–406 From the Department of Radiology, University of Cincinnati College of Medicine, 234 Goodman St, 670761 Cincinnati, OH (J.C.); Department of Radiology, Albert Einstein College of Medicine, Bronx, NY (E.S.A.); Department of Radiology, University of Washington College of Medicine, Seattle, WA (N.J.B.); Department of Radiology, Boston University School of Medicine, Boston, MA (A.M.N.); Department of Radiology and Imaging Sciences; Neurology; and Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, GA (C.C.M.). Received November 22, 2013; accepted November 23, 2013. Address correspondence to: J.C. e-mail: [email protected] ªAUR, 2014 http://dx.doi.org/10.1016/j.acra.2013.11.020
400
tion, a wide variety of stated and unstated expectations exist regarding the performance targets in each of these three domains. Many radiology chairs are asked to serve as external consultants to review departments at other medical schools. Participating in external reviews can be viewed by the chair as an important service to the radiology community, a mechanism to provide guidance or mentorship to a fellow leader in radiology, an opportunity for self-development and to compare one’s own department with others, and to learn things that may benefit the reviewer’s department. The Society of Chairs of Academic Radiology Departments (SCARD) views external reviews as a tool that when optimally used can foster the success of an individual chair and academic department and ultimately contribute to continual process improvement across the field of radiology. SCARD developed a template (Appendix) that outlines important components of an external review report and a table (Table 1) that outlines the objective information that can be requested from the institution/department before the reviewer’s site visit. The template is meant to facilitate a highquality review process by outlining standardized elements based on the best practices. It may further serve as a guide to a chair who is preparing for his/her first review, chairs who serve as external consultants, and institutional officials seeking a review of a radiology department. Although no approach fits all institutional needs, the review process should reflect the unique context and individual circumstances of the institution and the department under review, and the availability of a comprehensive standardized template can be valuable. Objective criteria are critical for measuring a department’s progress and comparing a department’s performance with national benchmarks. These objective metrics should be considered in concert with subjective data to create a report that reflects a department’s strengths and weaknesses
Academic Radiology, Vol 21, No 3, March 2014
GUIDE TO CONDUCT A DEPARTMENT REVIEW
TABLE 1. Information to Request Before Radiology Department Review Site Visit Overview Chair’s personal performance
Organizational structure
Financial performance
Clinical performance
Department Chair self-evaluation, summary of department changes over the past 5 years, including both current and recent past departmental goals and major initiatives Clinical activity (type and schedule) Scholarly activity (number and brief description of grants, publications, national presentations, and teaching activities) Service activity (number and description of local, regional, and national committees, society officerships, editorial boards, and study sections) Faculty full-time-equivalents (FTEs) (number and percentage of women and other minorities and rank distribution) Curriculum vitae of all faculty members Department organizational chart showing the titles and names of the Chair, Vice Chairs, Directors, Section Chiefs, other faculty, and staff, and their reporting structure List of department committees Roles of department members in hospital and medical school committees (e.g., promotions, medical executive, quality ,and safety), including any reporting structures of departmental faculty to extradepartmental entities Description of how faculty mentoring is provided and how professional development opportunities are communicated to faculty Description of formal faculty evaluation process Methods of communication (e.g., department website, department newsletter, department e-mail notices, department digital signage, faculty meetings, and external communication outlets) List of department support personnel (e.g., business administrator, administrative assistants, billing/coding staff, department budget analyst, department research assistants, program coordinators, and information technology support staff) Description of faculty clinical scheduling process and sample schedules Vacation and meeting policy Description of faculty academic time (education, research, scholarship, and administrative) List of revenue sources (e.g., clinical, college, hospital, state, grants, endowments, other) Description of department budgeting process Five-year history of actual revenues and expenses versus budget Description of department reserves, endowments, and ongoing philanthropic efforts Faculty compensation plan document (description of plan/process, components such as base/ incentive/call coverage) Mean salaries for each rank compared to national benchmarks Travel and expense allowance policy Faculty work relative value units adjusted for FTE, per section and department total, compared to national benchmarks (e.g., Medical Group Management Association [MGMA], University HealthSystem Consortium [UHC], and Association of Administrators in Academic Radiology [AAARAD]/Society of Chairs of Academic Radiology Departments) Number of examinations by modality and site for past 5 years Description of the range of diagnostic and therapeutic services and any deficiencies Method of 24/7/365 faculty coverage (e.g., faculty available on or off-site, teleradiology) Equipment inventory with dates of purchase Description of capital budget process Equipment purchase and maintenance process Radiology information system, picture archiving and communication system, and voice recognition equipment and support Description of how physical space and facilities are maintained and renovated Description of quality, safety, and efficiency initiatives (e.g., American College of Radiology [ACR] accreditation, dose reduction and monitoring, ACR General Radiology Information Database, days to appointment, complication rate, decision support integrated with computerized physician order entry, and meaningful use participation) Patient satisfaction survey results for past 2 years Ordering provider satisfaction survey results for past 2 years Samples of department dashboards Report of department report turnaround times for past 2 years (Continued)
401
COLLINS ET AL
Academic Radiology, Vol 21, No 3, March 2014
TABLE 1. (Continued) Information to Request Before Radiology Department Review Site Visit Educational programs
Research/scholarly activity
Institutional model for funding teaching Description of medical student clerkship (mandatory or voluntary, length, and curriculum) Medical student evaluations of clerkship for past 2 years Description of core residency program (number approved and current number, funding source/s) The most recent Radiology Review Committee report The most recent resident survey results The most recent internal residency program review Residency match results for past 5 years American Board of Radiology pass/condition rate List of resident publications and presentations over past 5 years Description of fellowship programs (accredited versus nonaccredited, type and number, number of fellows in each program over past 5 years, the most recent RRC site visit reports, number of fellow publications and presentations over past 5 years) Description of department-sponsored Continuing Education Programs Description of laboratories and other dedicated research space List of grants over past 5 years (federal, foundation, society, and internal) to include salary support and percent effort Submitted grants in past year Number of faculty presentations in past 5 years Number of faculty publications in past 5 years (listed as peer-reviewed and nonpeer-reviewed) Number of faculty committee memberships (local, regional, and national), society officerships, editorial board memberships, and study section memberships over past 5 years Awards, honors, and distinctions of faculty/department/staff/students
and provides specific actionable recommendations for performance improvement. A complete description of the operational aspects of a radiology department and metrics that can be used to judge a department’s performance are beyond the scope of this article and can be found elsewhere (2). A detailed process for radiology services review is also available from the Royal College of Radiologists in London, England (3,4).
REVIEW PROCESS The ‘‘classic’’ department review typically occurs at regular intervals (usually every 5–7 years), is directed by the medical school dean, and is conducted by both an internal review committee and external advisors who produce a formal report (1). Internal and external reviewers can work separately or as a combined committee. Some schools of medicine have begun to rethink the traditional departmental review process, citing ‘‘too much data and too little plan’’ (1). Other criticisms of reviews have included a little long-lasting benefit as perceived by the department and dean’s office, lack of follow up of the report’s recommendations, ineffective strategic planning resulting from the retrospective analysis of the department, and limited incentive by reviewers to document weaknesses, if the review is seen primarily as an opportunity to request additional resources. Additionally, reviews are costly to an institution in terms of actual expense and staff time. The Stanford University School of Medicine estimated the average
402
cost per review to be $20,000 for honoraria, travel, and supplies, and $20,000 for dean’s office staff time; the additional costs of Radiology department faculty and staff time was not calculated (1). The University of Michigan Medical School estimated the annual cost of conducting reviews to be up to $240,000 (1). In an effort to gain a greater return on investment, they revised their departmental review process to link strategic goals, performance measures, evaluation systems, and rewards (1). The final report is a strategic plan containing a SWOT (strengths, weaknesses, opportunities, and threats) analysis and strategic goals for the department, with related action plans. The Michigan model employs outside experts from the same as well as from a different medical discipline and from a nonmedical discipline (e.g., strategic planning expert). The review of a department should include an assessment of the overall leadership and management performance of the department chair, presence of good faculty development practices, integrity of the teaching programs, excellence of the research program and associated research infrastructure, and quality and effectiveness of the clinical programs (5). Similarly, there should be a stated understanding of favorable or unfavorable trends in the department, including the evolving nature and structure of the department and its effectiveness over time in addressing the tripartite mission. Although chair job expectations may vary from institution to institution, there is a general agreement on the main areas of accountability. The American Association of Chairs of Departments of Psychiatry established a tool kit for new
Academic Radiology, Vol 21, No 3, March 2014
chairs that included key issues and responsibilities of a chair and benchmarking information on other psychiatry departments nationally (6). In evaluating the department chair, performance dimensions can include general administrative and leadership effectiveness, his/her approach to professional and staff development and management, financial stewardship, academic leadership, and ability to promote good working relationships with other departmental and institutional leaders. The effectiveness should be assessed across the domains of clinical excellence, teaching, and research. The assessment should also include the chair’s attention to gender climate, hiring and maintaining underrepresented minorities, and proportion of women among department leaders. A selfevaluation and/or 360-degree feedback may be used. The five-year departmental review process at the University of North Carolina School of Medicine starts with a self-study generated by the department Chair. An internal review committee interviews and receives comments from at least two departmental professors, two associate professors, and two assistant professors who have reviewed the self-study. Written comments from all faculty members are invited (1). At the University of Wisconsin Medical School, all members of each department are given the opportunity to express their preference for the Chair each year by secret ballot. This ballot is advisory to the dean and is used in the annual evaluation of department Chairs (1). At Emory University School of Medicine, the dean conducts a formal anonymous survey of the faculty annually that targets how effectively their chair supports professional development. The results are shared with each chair during his/her annual performance review (personal communication: Sharon Weiss, MD, Emory University School of Medicine). External reviewers should receive information from the department to be reviewed before conducting the review. The reviewer may need to request additional information that is not initially provided (see Table 1). After reviewing this information, a preliminary report can be prepared that will help guide the reviewer’s on-site evaluation. The SCARD report template can be prepopulated before the site visit (Appendix), enabling the reviewer to use his/her time most efficiently while conducting on-site interviews. If more than one external reviewer is involved, one should be selected to serve as the chair of the external review committee to distribute tasks efficiently. The reviewer should be provided with an itinerary that outlines a schedule of interviews with the medical school dean(s), radiology chair, chairs from other departments, hospital executive(s), faculty practice plan executive(s), department faculty (including vice chairs and section chiefs), residents, and the department business administrator. There may be additional individuals the reviewer will want to talk to, such as educational program directors and coordinators and research assistants. The previsit information packet may uncover areas of concern or opportunity that suggest benefit
GUIDE TO CONDUCT A DEPARTMENT REVIEW
from meeting with specific individuals that are not on the itinerary. It is advisable to meet with the individual(s) requesting the review at the start of the visit as a mechanism to clarify the institutional goals. In addition, the authors have found it valuable to meet with the chair early in the review to elucidate his/ her concerns and insights and again late in the visit to clarify issues or opportunities uncovered during the visit. The site review is usually conducted over 1–2 days and should include a tour of the department. The time provided may be too limited to meet with everyone. Reviewers may want to have discussions in parallel or to coordinate a few phone interviews before or after the visit. The site visit will often include a ‘‘wrap-up’’ session when reviewers provide senior leadership with a preliminary summary. The reviewers should discuss their thoughts before the wrap-up to facilitate a summary that addresses the most challenging issues and needs for a department. After the site visit, the reviewer should complete the report template, write a summary of the department’s strengths, weaknesses, and opportunities, and make specific constructive and realistic recommendations based on the subjective and objective data obtained. When appropriate, these recommendations should include requests for additional support and resources for the department, team building activities between department and organizational leaders, and additional leadership training or mentorship for the chair (7). The report should be submitted as soon as possible, generally within 3 weeks of the visit. Most institutions will expect the report and any information provided to the reviewer to be kept confidential.
SUMMARY Department reviews require careful preparation, active listening, the ability to respond unemotionally to issues that strike chords in oneself, and, most of all, the ability to distill extensive subjective and objective information into a clear plan of action (1). SCARD members have worked to develop a standardized template to guide highly effective department reviews and to promote the best practices throughout the academic radiology community. The reviewer is rewarded with a sense of being appreciated, an opportunity to meet new colleagues, and a deepened perspective of both the breadth and value of their specialty to medical schools and the community. Performing reviews is something to which chairs should aspire, if only because they will gain as much as they give (1).
REFERENCES 1. Association of American Medical Colleges. The successful medical school department chair: a guide to good institutional practice, Module 3: Performance, evaluation, rewards, renewal, Chapter 2, Examples of good practice in departmental reviews. Washington, DC: Association of American Medical Colleges, 2003.
403
COLLINS ET AL
2. Yousem DM, Beauchamp NJ Jr, eds. Radiology business practice: how to succeed. Philadelphia, PA: Saunders/Elsevier, 2008. 3. Board of the Faculty of Clinical Radiology. The Royal College of Radiologists, College review of radiology services, 3rd ed. The Royal College of Radiologists. http://www.rcr.ac.uk/docs/radiology/pdf/College_review_ third_edn.pdf. Accessed February 2012. 4. The process for service reviews conducted by the service review committee. Board of the Faculty of Clinical Radiology. The Royal College of Radi-
404
Academic Radiology, Vol 21, No 3, March 2014
ologists, http://www.rcr.ac.uk/docs/radiology/pdf/Process_SRC_final.pdf; 2012. Accessed August 10, 2013. 5. Hitchcock MA, Stritter FT, Bland CJ. Faculty development in the health professions: conclusions and recommendations. Med Teach 1992; 14:295–309. 6. Monro SA. Tool kit for new chairs. Acad Psychiatry 2006; 30. 7. Wolverton M, Ackerman R, Holt S. Preparing for leadership: what academic department chairs need to know. J High Educ Policy Manage 2013; 27: 227–238.
Academic Radiology, Vol 21, No 3, March 2014
GUIDE TO CONDUCT A DEPARTMENT REVIEW
APPENDIX. RADIOLOGY DEPARTMENT REVIEW REPORT TEMPLATE
Department Finances
Overview
Revenue sources (clinical, college, hospital, state, grants, other) Expenses, including overhead Budget process Five-year history actuals versus budget Reserves/endowments/ongoing philanthropic efforts Faculty compensation plan (description of plan/process, components such as base/incentive/call coverage, salaries compared to national benchmarks) Travel and expense allowance policy Hospital and medical school funding for clinical shortfall and for administrative roles performed by the department for the benefit of the organization
Brief history Chair/department relationship to practice plan, medical school, hospital(s), and university Changes in past 5 years Current department goals (clinical/financial/educational/ research/scholarship, described as measurable objectives)
Chair’s Personal Performance
Years as Chair, title, and rank Clinical activity Scholarly activities (grants, publications, national presentations, and teaching) Service (local, regional, and national committees, society officership, editorial boards, and study sections) Department/institution evaluation of leadership Organizational Structure
Faculty (full-time-equivalents [FTEs], #/% women and other minorities, rank distribution) Department organizational chart Faculty governance (vice chairs, section chiefs, clinical directors, educational directors, and committees) and associated protected time to serve leadership roles Hospital and medical school governance and the role of the chair in key decision making committees Mentoring/professional development opportunities for leadership team and individual faculty Formal faculty evaluation process Communication (department website, department newsletter, department e-mail notices, department digital signage, faculty meetings, individual member and group leadership team meetings with the chair, meetings with key support personnel [e.g., weekly meetings with departmental administrator, monthly meetings with billing and coding team, individual meetings with faculty members, and chair interactions with the residents] and external communication outlets) Support personnel (administrative assistants, department IT, department coding/billing, department budget analyst, department business administrator, department research assistants) Faculty clinical scheduling process Vacation and meeting policy Faculty academic time (education, research/scholarship, and administrative)
Clinical Performance
Faculty work relative value units adjusted for FTE, compared to national benchmarks Volume by modality and site past 5 years Range of diagnostic and therapeutic services/deficiencies Service opportunities and their barriers (e.g., poor access, perceived quality issues, resource or expertise limitations, and inappropriate incentives for referral outside the institution) Method of 24/7/365 coverage Equipment (inventory/deficiencies, capital budget process, and equipment purchase and maintenance process) Radiology information system/picture archiving and communication system/voice recognition equipment and support Maintenance of physical space/facilities Quality/safety/efficiency initiatives (peer review, Image Gently, Image Wisely, ACR CT Dose Index Registry, ACR accreditation, ACR General Radiology Information Database, report turnaround time, days to appointment, complication rate, patient satisfaction surveys, ordering provider satisfaction surveys, decision support integrated with computerized physician order entry, and meaningful use participation)
Educational Programs
Medical student clerkship (mandatory or voluntary, length, curriculum, and student ratings) Residency program (number approved/current number, funding source, the most recent RRC site visit report, the most recent resident survey results, annual internal pro-
405
COLLINS ET AL
Academic Radiology, Vol 21, No 3, March 2014
gram review, match results past 5 years, ABR pass/condition rate, and scholarly activity) Fellowship programs (type and number past 5 years, funding source, accredited/nonaccredited, the most recent RRC site visit report, scholarly activity) Continuing education (formal continuing medical education lectures/courses)
Service (local, regional and national committees, society officership, editorial boards, and study sections) Departmental cores provided to support research Research training and mentorship
Research/Scholarly Activity
Lab/other dedicated research space Grants (federal, foundation, society, internal, percent salary support, and percent faculty effort) National presentations past 5 years Publications past 5 years
406
Tools and Templates (the examples listed are not meant to represent a complete list)
Staffing grid to guide recruitment Electronic professional activities report Structured agendas Dashboards Summary and Assessment. Recommendations.
