E D U C A T I O N A L IN N O V A T I O N
A Citation Tracking System to Facilitate Sponsoring Institution Oversight of ACGME-Accredited Programs
Timothy R. Long, MD John D. Poe, MBA Richard S. Zimmerman, MD Steven H. Rose, MD
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
Abstract Background The Accreditation Council for Graduate Medical Education (ACGME) requires the graduate medical education committee and the designated institutional official to ensure that citations for noncompliance with the accreditation standards and institutional trends in citations are reviewed and corrected. Objective To describe a citation tracking system (CTS) that uses Microsoft Office Access to efficiently catalogue, monitor, and document resolution of citations. Innovation The CTS was implemented in a sponsoring institution with oversight of 133 ACGME-accredited programs. The designated institutional official and the graduate medical education committee review all program letters of notification and enter citations into the CTS. A program-correction plan is required for each citation and is entered into the database. Open citations and action plans are reviewed by the graduate medical education committee and the designated institutional
Editor’s Notes: The citation tracking system database developed for this study is available from the authors on request.
Introduction
The Accreditation Council for Graduate Medical Education (ACGME) requires sponsoring institutions to oversee accredited programs through an organized administrative
All authors are in the College of Medicine at Mayo Clinic. Timothy R. Long, MD, is Assistant Professor of anesthesiology and Program Director of anesthesiology residency (Minnesota); John D. Poe, MBA, is Administrator; Richard S. Zimmerman, MD, is Associate Professor of neurosurgery and Associate Dean for graduate medical education (Arizona); Steven H. Rose, MD, is Professor of anesthesiology and Vice Dean/Designated Institutional Official at the Mayo School of Graduate Medical Education.
official on a quarterly basis, with decisions ranging from ‘‘closing’’ the citation to approving the action plan in process to requiring a new or modified action plan. Citation categories and subcategories are accessed on the ACGME website and entered into the CTS to identify trends. Results All 236 citations received since the 2006 Mayo School of Graduate Medical Education institutional site visit were entered into the CTS. On November 22, 2011, 26 of 236 citations (11%) were in active status with ongoing action plans, and 210 (89%) citations had been resolved and were closed. Conclusions The CTS uses commercially available software to ensure citations are monitored and addressed and to simplify analysis of citation trends. The approach requires minimal staff time for data input and updates and can be performed without institutional information technology assistance.
system led by a designated institutional official (DIO) in collaboration with a graduate medical education committee (GMEC).1 Reviewing ACGME letters of notification to identify citations, developing corrective action plans, and following up to ensure citations have been fully resolved are critical elements of that oversight. Ensuring that citations have been resolved is particularly challenging in institutions that oversee many programs. To date, no publication to our knowledge has addressed the use of technology to ensure citation resolution. We describe a citation tracking system (CTS) that uses a Microsoft (Redmond, WA) Office Access relational-database management system to efficiently catalogue, monitor, and analyze citations.
Funding: The authors report no external funding source for this study.
Methods
Corresponding author: Timothy R. Long, MD, Department of Anesthesiology, College of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, 507.255.3298, [email protected]
We developed and implemented the CTS in a sponsoring institution with oversight of 133 ACGME-accredited programs in Rochester, Minnesota; Jacksonville, Florida; and Phoenix/Scottsdale, Arizona. The DIO and GMEC review all ACGME program letters of notification. Graduate
Received December 16, 2011; revisions received February 24, 2012, and April 5 2012; accepted April 10, 2012. DOI: http://dx.doi.org/10.4300/JGME-D-11-00313.1
500 Journal of Graduate Medical Education, December 2012
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
EDUCATIONAL INNOVATION
medical education (GME) office personnel manually enter the program name, the date of the letter of notification, the citation category and subcategory (from WebAds, Attachment 1; https://www.acgme.org/ads/default.asp), and the exact text of each citation into the CTS. The DIO forwards copies of the citations to the program director, program coordinator, and respective department/division chair. Specific deadlines are established for submission of a corrective action plan that includes one or more specific outcome metrics. The DIO and GMEC review all open citations quarterly at regularly scheduled GMEC meetings. After this review, the DIO and GMEC may close the citation by documenting a satisfactory outcome, approve the action plan but keep the citation open (in process) until a satisfactory outcome is demonstrated, or require a revised corrective action plan and/or establish additional outcome
What was known Sponsoring institutions are expected to ensure that citations for noncompliance with the accreditation standards are reviewed and addressed.
What is new A citation tracking system in a large sponsoring institution allowed the tracking, review, and disposition of citations, as well as review and assessment of corrective action plans by the graduate medical education committee (GMEC).
Limitations The tracking system interfaced with a high-performing GMEC and may not have the same positive effect in institutions without this body for review of corrective action plans.
Bottom line A simple citation tracking system that uses commercially available software allowed the review and disposition of citations with most citations declared resolved and ‘‘closed’’ by the institution.
metrics. The procedures for populating and managing the CTS are summarized in F I G U R E 1 . An annual report is prepared using CTS data that lists the status of all citations (open or closed). These data are exported to Microsoft Excel to create pie charts that graphically present the distribution of total and open citations by category and subcategory. The data are included in the annual report to the governing body(s) and organized medical staff member(s). Comparing the distribution of citations by category and subcategory provides the DIO and GMEC with data to better understand citation trends and to identify citation categories and subcategories that are difficult to resolve. The relational database allows the end user to develop queries that mine the data by program (eg, general surgery), by citation category (eg, citations related to faculty), and/or by citation subcategory (eg, qualifications of faculty). The CTS database is available from the authors on request. Results
FIGURE 1
MSGME Program Administrative Oversight
Since its 2006 institutional site visit, the sponsoring institution has received a total of 236 citations among its 133 ACGME-accredited programs. These were entered into the CTS. On November 22, 2011, there were 26 of the 236 citations (11%) open (in active status) with ongoing corrective action plans, and the remaining 210 citations (89%) had been resolved and were closed by the DIO and GMEC (T A B L E ). The distribution of total citations by category is presented in F I G U R E 2 , and open citations by category are shown in F I G U R E 3 . Specialty-specific program citations are not directly related to the sponsoring institution. For example, the Journal of Graduate Medical Education, December 2012 501
E D U C A T I O N A L IN N O V A T I O N
TABLE
Citation Database Reporting of Accreditation Council for Graduate Medical Education Citations
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
Category
Open, No (%)
Closed, No (%)
Total, No (%)
1. Institutional support
2 (0.8)
19 (8.1)
21 (8.9)
2. Resident appointment/ resident contracts
0 (0.0)
6 (2.5)
6 (2.5)
3. Faculty
5 (2.1)
41 (17.4)
46 (19.5)
4. Education program
5 (2.1)
93 (39.4)
98 (41.5)
5. Evaluation
3 (1.3)
40 (16.9)
43 (18.2)
6. Experimentation and innovation
0 (0)
4 (1.7)
4 (1.7)
15 (6.4)
203 (86.0)
218 (92.4)
11 (4.7)
7 (3.0)
18 (7.6)
26 (11.0)
210 (89.0)
236 (100)
Subtotal Institutional Citations 7. Program citation Total Citations
ACGME describes a citation for insufficient morbidity and mortality conferences as a specialty-specific program citation. These citations are not recorded by the ACGME on Attachment 1 A (Institutional Review Document, Part II; http://www.acgme.org). Eighteen of 236 total citations (7.6%) were categorized by the ACGME as specialtyspecific program citations; however, 11 of 26 open citations (42.3%) were categorized as specialty-specific citations. These data suggest that specialty-specific program citations are less common but may be difficult to resolve.
FIGURE 2
Total ACGME Citations by Category (n=236)
502 Journal of Graduate Medical Education, December 2012
Most program directors regard the CTS favorably and appreciate the periodic reminders that ensure citations are being addressed. The DIO and GMEC benefit from a systems approach that ensures no citations escape timely review. Establishing and maintaining the CTS requires familiarity with the Microsoft Access database and basic clerical skills. The CTS was developed and implemented without institutional information technology resources, and no information technology support is required to maintain the system. The time required to enter data into the CTS varies somewhat, although GME office staff do
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
EDUCATIONAL INNOVATION
FIGURE 3
Open ACGME Citations by Category (n=26)
not spend more than 10 hours per quarter entering citations and monitoring their resolution. Staff has reported improved efficiency of monitoring and resolving citations by using the centralized CTS process. The average time to resolve citations has decreased since instituting the quarterly CTS database review because this aspect of our process ensures all open citations are reviewed by the DIO and the GMEC on at least a quarterly basis. The mean time to citation resolution has decreased by approximately 40%.
1 of 4 key issues requiring attention in GME.2 Successful implementation of electronic tools that support compliance with ACGME duty hour requirements, simplify assessment and documentation of ACGME competency achievement, or otherwise facilitate GME oversight and governance have been reported by others.3–6 The need for improved systems and procedures to support GME is made more prominent by the increased complexity of ACGME, the Joint Commission, and other regulatory and accreditation requirements.7
Discussion
Conclusion
The CTS can be used to generate summary data for inclusion in the ACGME-required annual report to the organized medical staff and governing body of the sponsoring institution and major participating sites that do not sponsor GME programs.1 This tool also ensures citations related to the safety of patient care are quickly recognized and resolved. For example, a citation related to inadequate faculty supervision of resident activities in an outpatient clinic was recognized and addressed in a timely way through meetings with the program director, department chair, and institutional leaders. The CTS is also useful in identifying common citation themes and facilitates preparation for institutional site visits, and it is well suited to demonstrate program governance and oversight should the ACGME require annual monitoring of key data and offer longer accreditation cycle lengths in the future. The American Association of Medical Colleges has included ‘‘institutional oversight and program support’’ as
We describe an electronic CTS database to ensure citations from ACGME accreditation reviews are monitored and addressed. The CTS facilitates institutional oversight of programs by ensuring the DIO and GMEC regularly review all citations, monitor corrective action plans, and do not close citations until they have been effectively resolved. It also facilitates analysis of citation trends by the DIO and GMEC to improve educational program oversight. These activities closely parallel the ACGME ‘‘Quality Loop.’’8 The CTS uses using Microsoft Access, a relatively inexpensive, commercially available, software application that can be easily customized to meet the needs of other sponsoring institutions. Although we describe implementation of the CTS in a large sponsoring institution, this technology could be adapted to an institution of any size because entering data into the system requires minimal office staff time and expertise. Journal of Graduate Medical Education, December 2012 503
E D U C A T I O N A L IN N O V A T I O N References
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
1 Accreditation Council for Graduate Medical Education. ACGME Institutional Requirements. http://www.acgme.org/acgmeweb/Portals/ 0/irc_IRCpr07012007.pdf. Effective July 1, 2007. Accessed October 3, 2012. 2 Executive Council, Association of American Medical Colleges. AAMC policy guidance on graduate medical education: assuring quality patient care and quality education. Acad Med. 2003;78(1):112–116. 3 Afrin LB, Arana GW, Medio FJ, Ybarra AF, Clarke HS Jr. Improving oversight of the graduate medical education enterprise: one institution’s strategies and tools. Acad Med. 2006;81(5):419–425. 4 Curry RH, Burgener AJ, Dooley SL, Christopher RP. Collaborative governance of multiinstitutional graduate medical education: lessons from the
504 Journal of Graduate Medical Education, December 2012
McGaw Medical Center of Northwestern University. Acad Med. 2008;83(6):568–573. 5 Gauger PG, Davis JW, Orr PJ. A novel web-based graduate medical education management system including ACGME compliance algorithms. Acad Med. 2002;77(9):928. 6 Tabuenca A, Catalano R, Gollin G, Shieck J. An Internet-based residency assessment application that fulfills the outcome project’s requirements. Curr Surg. 2003;60(1):89–93. 7 Jackson VP. Institutional requirements. Acad Radiol. 2003;(10)(suppl 1): S31–S34. 8 Institutional Review Committee. SES04: Institutional Review: Practices and Possibilities. Paper presented at: Beyond Boundaries: ACGME Annual Educational Conference; March 4, 2011; Nashville, TN.
A Citation Tracking System to Facilitate Sponsoring Institution Oversight of ACGME-Accredited Programs
Timothy R. Long, MD John D. Poe, MBA Richard S. Zimmerman, MD Steven H. Rose, MD
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
Abstract Background The Accreditation Council for Graduate Medical Education (ACGME) requires the graduate medical education committee and the designated institutional official to ensure that citations for noncompliance with the accreditation standards and institutional trends in citations are reviewed and corrected. Objective To describe a citation tracking system (CTS) that uses Microsoft Office Access to efficiently catalogue, monitor, and document resolution of citations. Innovation The CTS was implemented in a sponsoring institution with oversight of 133 ACGME-accredited programs. The designated institutional official and the graduate medical education committee review all program letters of notification and enter citations into the CTS. A program-correction plan is required for each citation and is entered into the database. Open citations and action plans are reviewed by the graduate medical education committee and the designated institutional
Editor’s Notes: The citation tracking system database developed for this study is available from the authors on request.
Introduction
The Accreditation Council for Graduate Medical Education (ACGME) requires sponsoring institutions to oversee accredited programs through an organized administrative
All authors are in the College of Medicine at Mayo Clinic. Timothy R. Long, MD, is Assistant Professor of anesthesiology and Program Director of anesthesiology residency (Minnesota); John D. Poe, MBA, is Administrator; Richard S. Zimmerman, MD, is Associate Professor of neurosurgery and Associate Dean for graduate medical education (Arizona); Steven H. Rose, MD, is Professor of anesthesiology and Vice Dean/Designated Institutional Official at the Mayo School of Graduate Medical Education.
official on a quarterly basis, with decisions ranging from ‘‘closing’’ the citation to approving the action plan in process to requiring a new or modified action plan. Citation categories and subcategories are accessed on the ACGME website and entered into the CTS to identify trends. Results All 236 citations received since the 2006 Mayo School of Graduate Medical Education institutional site visit were entered into the CTS. On November 22, 2011, 26 of 236 citations (11%) were in active status with ongoing action plans, and 210 (89%) citations had been resolved and were closed. Conclusions The CTS uses commercially available software to ensure citations are monitored and addressed and to simplify analysis of citation trends. The approach requires minimal staff time for data input and updates and can be performed without institutional information technology assistance.
system led by a designated institutional official (DIO) in collaboration with a graduate medical education committee (GMEC).1 Reviewing ACGME letters of notification to identify citations, developing corrective action plans, and following up to ensure citations have been fully resolved are critical elements of that oversight. Ensuring that citations have been resolved is particularly challenging in institutions that oversee many programs. To date, no publication to our knowledge has addressed the use of technology to ensure citation resolution. We describe a citation tracking system (CTS) that uses a Microsoft (Redmond, WA) Office Access relational-database management system to efficiently catalogue, monitor, and analyze citations.
Funding: The authors report no external funding source for this study.
Methods
Corresponding author: Timothy R. Long, MD, Department of Anesthesiology, College of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, 507.255.3298, [email protected]
We developed and implemented the CTS in a sponsoring institution with oversight of 133 ACGME-accredited programs in Rochester, Minnesota; Jacksonville, Florida; and Phoenix/Scottsdale, Arizona. The DIO and GMEC review all ACGME program letters of notification. Graduate
Received December 16, 2011; revisions received February 24, 2012, and April 5 2012; accepted April 10, 2012. DOI: http://dx.doi.org/10.4300/JGME-D-11-00313.1
500 Journal of Graduate Medical Education, December 2012
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
EDUCATIONAL INNOVATION
medical education (GME) office personnel manually enter the program name, the date of the letter of notification, the citation category and subcategory (from WebAds, Attachment 1; https://www.acgme.org/ads/default.asp), and the exact text of each citation into the CTS. The DIO forwards copies of the citations to the program director, program coordinator, and respective department/division chair. Specific deadlines are established for submission of a corrective action plan that includes one or more specific outcome metrics. The DIO and GMEC review all open citations quarterly at regularly scheduled GMEC meetings. After this review, the DIO and GMEC may close the citation by documenting a satisfactory outcome, approve the action plan but keep the citation open (in process) until a satisfactory outcome is demonstrated, or require a revised corrective action plan and/or establish additional outcome
What was known Sponsoring institutions are expected to ensure that citations for noncompliance with the accreditation standards are reviewed and addressed.
What is new A citation tracking system in a large sponsoring institution allowed the tracking, review, and disposition of citations, as well as review and assessment of corrective action plans by the graduate medical education committee (GMEC).
Limitations The tracking system interfaced with a high-performing GMEC and may not have the same positive effect in institutions without this body for review of corrective action plans.
Bottom line A simple citation tracking system that uses commercially available software allowed the review and disposition of citations with most citations declared resolved and ‘‘closed’’ by the institution.
metrics. The procedures for populating and managing the CTS are summarized in F I G U R E 1 . An annual report is prepared using CTS data that lists the status of all citations (open or closed). These data are exported to Microsoft Excel to create pie charts that graphically present the distribution of total and open citations by category and subcategory. The data are included in the annual report to the governing body(s) and organized medical staff member(s). Comparing the distribution of citations by category and subcategory provides the DIO and GMEC with data to better understand citation trends and to identify citation categories and subcategories that are difficult to resolve. The relational database allows the end user to develop queries that mine the data by program (eg, general surgery), by citation category (eg, citations related to faculty), and/or by citation subcategory (eg, qualifications of faculty). The CTS database is available from the authors on request. Results
FIGURE 1
MSGME Program Administrative Oversight
Since its 2006 institutional site visit, the sponsoring institution has received a total of 236 citations among its 133 ACGME-accredited programs. These were entered into the CTS. On November 22, 2011, there were 26 of the 236 citations (11%) open (in active status) with ongoing corrective action plans, and the remaining 210 citations (89%) had been resolved and were closed by the DIO and GMEC (T A B L E ). The distribution of total citations by category is presented in F I G U R E 2 , and open citations by category are shown in F I G U R E 3 . Specialty-specific program citations are not directly related to the sponsoring institution. For example, the Journal of Graduate Medical Education, December 2012 501
E D U C A T I O N A L IN N O V A T I O N
TABLE
Citation Database Reporting of Accreditation Council for Graduate Medical Education Citations
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
Category
Open, No (%)
Closed, No (%)
Total, No (%)
1. Institutional support
2 (0.8)
19 (8.1)
21 (8.9)
2. Resident appointment/ resident contracts
0 (0.0)
6 (2.5)
6 (2.5)
3. Faculty
5 (2.1)
41 (17.4)
46 (19.5)
4. Education program
5 (2.1)
93 (39.4)
98 (41.5)
5. Evaluation
3 (1.3)
40 (16.9)
43 (18.2)
6. Experimentation and innovation
0 (0)
4 (1.7)
4 (1.7)
15 (6.4)
203 (86.0)
218 (92.4)
11 (4.7)
7 (3.0)
18 (7.6)
26 (11.0)
210 (89.0)
236 (100)
Subtotal Institutional Citations 7. Program citation Total Citations
ACGME describes a citation for insufficient morbidity and mortality conferences as a specialty-specific program citation. These citations are not recorded by the ACGME on Attachment 1 A (Institutional Review Document, Part II; http://www.acgme.org). Eighteen of 236 total citations (7.6%) were categorized by the ACGME as specialtyspecific program citations; however, 11 of 26 open citations (42.3%) were categorized as specialty-specific citations. These data suggest that specialty-specific program citations are less common but may be difficult to resolve.
FIGURE 2
Total ACGME Citations by Category (n=236)
502 Journal of Graduate Medical Education, December 2012
Most program directors regard the CTS favorably and appreciate the periodic reminders that ensure citations are being addressed. The DIO and GMEC benefit from a systems approach that ensures no citations escape timely review. Establishing and maintaining the CTS requires familiarity with the Microsoft Access database and basic clerical skills. The CTS was developed and implemented without institutional information technology resources, and no information technology support is required to maintain the system. The time required to enter data into the CTS varies somewhat, although GME office staff do
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
EDUCATIONAL INNOVATION
FIGURE 3
Open ACGME Citations by Category (n=26)
not spend more than 10 hours per quarter entering citations and monitoring their resolution. Staff has reported improved efficiency of monitoring and resolving citations by using the centralized CTS process. The average time to resolve citations has decreased since instituting the quarterly CTS database review because this aspect of our process ensures all open citations are reviewed by the DIO and the GMEC on at least a quarterly basis. The mean time to citation resolution has decreased by approximately 40%.
1 of 4 key issues requiring attention in GME.2 Successful implementation of electronic tools that support compliance with ACGME duty hour requirements, simplify assessment and documentation of ACGME competency achievement, or otherwise facilitate GME oversight and governance have been reported by others.3–6 The need for improved systems and procedures to support GME is made more prominent by the increased complexity of ACGME, the Joint Commission, and other regulatory and accreditation requirements.7
Discussion
Conclusion
The CTS can be used to generate summary data for inclusion in the ACGME-required annual report to the organized medical staff and governing body of the sponsoring institution and major participating sites that do not sponsor GME programs.1 This tool also ensures citations related to the safety of patient care are quickly recognized and resolved. For example, a citation related to inadequate faculty supervision of resident activities in an outpatient clinic was recognized and addressed in a timely way through meetings with the program director, department chair, and institutional leaders. The CTS is also useful in identifying common citation themes and facilitates preparation for institutional site visits, and it is well suited to demonstrate program governance and oversight should the ACGME require annual monitoring of key data and offer longer accreditation cycle lengths in the future. The American Association of Medical Colleges has included ‘‘institutional oversight and program support’’ as
We describe an electronic CTS database to ensure citations from ACGME accreditation reviews are monitored and addressed. The CTS facilitates institutional oversight of programs by ensuring the DIO and GMEC regularly review all citations, monitor corrective action plans, and do not close citations until they have been effectively resolved. It also facilitates analysis of citation trends by the DIO and GMEC to improve educational program oversight. These activities closely parallel the ACGME ‘‘Quality Loop.’’8 The CTS uses using Microsoft Access, a relatively inexpensive, commercially available, software application that can be easily customized to meet the needs of other sponsoring institutions. Although we describe implementation of the CTS in a large sponsoring institution, this technology could be adapted to an institution of any size because entering data into the system requires minimal office staff time and expertise. Journal of Graduate Medical Education, December 2012 503
E D U C A T I O N A L IN N O V A T I O N References
Journal of Graduate Medical Education 2012.4:500-504. Downloaded from www.jgme.org by 181.215.39.76 on 01/13/19. For personal use only.
1 Accreditation Council for Graduate Medical Education. ACGME Institutional Requirements. http://www.acgme.org/acgmeweb/Portals/ 0/irc_IRCpr07012007.pdf. Effective July 1, 2007. Accessed October 3, 2012. 2 Executive Council, Association of American Medical Colleges. AAMC policy guidance on graduate medical education: assuring quality patient care and quality education. Acad Med. 2003;78(1):112–116. 3 Afrin LB, Arana GW, Medio FJ, Ybarra AF, Clarke HS Jr. Improving oversight of the graduate medical education enterprise: one institution’s strategies and tools. Acad Med. 2006;81(5):419–425. 4 Curry RH, Burgener AJ, Dooley SL, Christopher RP. Collaborative governance of multiinstitutional graduate medical education: lessons from the
504 Journal of Graduate Medical Education, December 2012
McGaw Medical Center of Northwestern University. Acad Med. 2008;83(6):568–573. 5 Gauger PG, Davis JW, Orr PJ. A novel web-based graduate medical education management system including ACGME compliance algorithms. Acad Med. 2002;77(9):928. 6 Tabuenca A, Catalano R, Gollin G, Shieck J. An Internet-based residency assessment application that fulfills the outcome project’s requirements. Curr Surg. 2003;60(1):89–93. 7 Jackson VP. Institutional requirements. Acad Radiol. 2003;(10)(suppl 1): S31–S34. 8 Institutional Review Committee. SES04: Institutional Review: Practices and Possibilities. Paper presented at: Beyond Boundaries: ACGME Annual Educational Conference; March 4, 2011; Nashville, TN.
