Practical Radiation Oncology (2014) xx, xxx–xxx
www.practicalradonc.org
Commentary
Toward safe and high quality care through peer review in radiation oncology: Need for more evidence Ronald C. Chen MD, MPH ⁎ Department of Radiation Oncology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina University of North Carolina-Lineberger Comprehensive Cancer Center, Chapel Hill, North Carolina Received 29 December 2013; accepted 30 December 2013
The planning process involved prior to starting radiation treatment, which can often take up to 1 to 2 weeks, and the daily nature of treatment for multiple weeks, provide a unique opportunity for radiation oncologists to perform peer review of treatment plans. Indeed, routine peer review occurs in the majority of radiation oncology centers. 1,2 In contrast to “morbidity and mortality” conferences, in which quality improvement and an infrastructure for learning is built around patient events that have already occurred, the goal of the peer review program in radiation oncology is to identify potential errors either before treatment starts or early enough during the treatment course to prevent harm to the patient. In the current era where there is an increasing emphasis on “quality” medical care and patient safety, radiation oncology leads by example with our longstanding culture for peer review and creation of treatment summaries, the latter concisely documenting treatment received and helping prevent future unintentional reirradiation. While peer review within each radiation oncology center is common, Ballo et al (in this issue of the journal) describe a novel program where an academic center performs peer review remotely with its satellite centers. 3 In this MD Anderson Cancer Center program, which started in 2007, all nonpalliative cases from satellite centers were required to be presented during twice-a-week peer review conferences. For each patient, the presentation included relevant history and clinical data, overall ⁎ Corresponding author. Department of Radiation Oncology, University of North Carolina at Chapel Hill, 101 Manning Dr, CB #7512, Chapel Hill, NC 27516. E-mail address: [email protected] (R.C. Chen).
treatment plan (including uses of surgery, radiation, and chemotherapy), and radiation plan (including contours and dose–volume histograms). The main goals of this retrospective study were to determine feasibility of remote peer review as assessed by the proportion of cases from satellite centers which were reviewed, and assess whether this peer review program increased compliance with MD Anderson institutional guidelines that cover all aspects of patient management “from consultation and simulation to target contouring, treatment planning, and delivery.” 3 The authors found that this program was highly feasible: 91.8% of eligible cases were presented for review in the first year; by year 4, this increased to 97.3%. Guideline compliance also improved; the proportion of cases for which a change was recommended as a result of peer review decreased dramatically from year 1 (16.5%) to year 4 (7.8%). While peer review is common, it may be administratively difficult for small community centers. As part of the American Society for Radiation Oncology (ASTRO)'s Target Safety Campaign, a white paper was recently published entitled “Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: Executive summary.” 4 This ASTRO report recommended that peer review be considered part of standard practice; for small centers, it specifically recommended creation of peer review relationships with other centers. In light of these recommendations, the study by Ballo et al 3 is timely. While it was conducted within a group of centers that have a unique main center–satellite relationship, where all physicians were obligated to participate in this peer review program and follow institutional guidelines, the study clearly demonstrated that the technologic and personnel infrastructure can be
1879-8500/$ – see front matter © 2014 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.prro.2013.12.006
2
R.C. Chen
created to successfully conduct remote peer review for every curative-intent patient, even in centers with large patient volumes. The practical implication of the study is that peer review can be accessible to every center—large or small, urban or rural—and for every cancer patient treated with radiation therapy. If desired, peer review partnerships should also be feasible between academic and community centers, even if the centers do not have a formal affiliation. Because errors do occur—various studies in radiation oncology have reported 4% to 12% prescription errors or treatment changes resulting from peer review 3,5,6—an ability to offer peer review for all patients will likely improve patient care and outcomes. Long term, this ability to perform remote peer review in radiation oncology has significant promise to reduce disparities in cancer care. Ballo et al 3 clearly show that peer review affects patient care, but demonstrating an impact on patient outcomes is much more difficult. In the oncologic literature there are many studies examining guideline compliance as a surrogate for “high quality” care, likely because the former is easier to measure and quantify. “Quality,” on the other hand, is a much more elusive concept, but consistent with the primary goal for peer review should be directly tied to patient outcomes. While guideline compliance is related to high quality care, there are important limitations to using this surrogate which must be considered when interpreting results of published studies, including this one. 7 First, guidelines are based on a combination of research evidence and expert opinion; therefore, many instances of guideline noncompliance may not constitute poor quality care. A recent analysis of the National Comprehensive Cancer Network guidelines found that only 6% of recommendations were based on level 1 evidence. 8 Guidelines on target contouring in radiation oncology are probably more likely to be based on expert opinion rather than high-level research evidence, and different experts can have significantly different target definitions. 9 Further, many patients may not fit neatly into guideline categories, and patient care often involves decisions that do not have clearly right or wrong answers. 4 Given these uncertainties, it is unclear what level of guideline compliance should be acceptable or whether guideline compliance should even be the central measure in peer review. In clinical scenarios where there is currently a knowledge gap and more than 1 acceptable treatment option, such as whether to include nodal irradiation for high-risk prostate cancer, a strict requirement for consistency through guideline compliance may prevent opportunities for future knowledge gained through retrospective outcomes reviews of patients treated differently. In the MD Anderson program, if the primary goal is to provide MD Anderson-style radiation treatment at all satellite centers, then a requirement for institutional guideline compliance is logical. However, on a broader
Practical Radiation Oncology: Month 2014
level of using peer review to prevent medical errors, and in the future where guideline compliance may be tied to reimbursement, the strength of evidence that forms a particular guideline must be considered. Compliance with recommendations based on level 1 evidence should be high, although given unique patient considerations and a need for individualized decision making, the bar should not be 100%. Another approach to peer review would be to assess whether each patient's treatment plan is “reasonable,” allowing for practice variation where there is currently no clear right versus wrong approach (each of us can probably think of innumerable such clinical scenarios) but making strong recommendations for change when errors have clearly been made. While peer review has been a longstanding part of the radiation oncology culture, there is a need for research on the “efficacy of peer review techniques and how these efforts can help improve the safety and quality” of radiation treatment. 4 The study by Ballo et al has answered this call by ASTRO, and many more studies are needed. There is wide variation on how peer review is conducted across centers, 1,2 and empiric data are needed to address many fundamental but unanswered questions. Which curative or palliative patients should be reviewed? When should peer review occur—before or after treatment initiation? What components of the treatment plan should be reviewed? How much does peer review cost, and perhaps the hardest question to answer, how much do patients benefit? Studies that assess patient outcomes, in addition to processes, are needed. With an emphasis on research efforts, radiation oncology can continue to lead in the area of patient quality and safety and make important contributions to medicine when our practices and research findings translate into practice in other specialties.
References 1. Lawrence YR, Whiton MA, Symon Z, et al. Quality assurance peer review chart rounds in 2011: A survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys. 2012;84:590-595. 2. Hoopes D, Johnstone P, Chapin P, et al. Early results from the ASTRO Practice Patterns in Peer Review Survey Project. Int J Radiation Oncol Biol Phys. 2013;87:S119. 3. Ballo M, Chronowski G, Schlembach P, et al. Prospective peer review quality assurance for outpatient radiation therapy. Pract Radiat Oncol. 2013 Dec 20 [Epub ahead of print] http://dx.doi.org/10.1016/j. prro.2013.11.004. 4. Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of caseoriented peer review to improve quality and safety in radiation oncology: Executive summary. Pract Radiat Oncol. 2013;3:149-156. 5. Boxer M, Forstner D, Kneebone A, et al. Impact of a real-time peer review audit on patient management in a radiation oncology department. J Med Imaging Radiat Oncol. 2009;53:405-411. 6. Brundage MD, Dixon PF, Mackillop WJ, et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys. 1999;43:115-124. 7. Chen RC. Guideline-adherent care vs quality care in cancer patients: Twins or distant cousins? JAMA Intern Med. 2013;173:569-570.
Practical Radiation Oncology: Month 2014 8. Poonacha TK, Go RS. Level of scientific evidence underlying recommendations arising from the National Comprehensive Cancer Network clinical practice guidelines. J Clin Oncol. 2011;29: 186-191.
Peer review in radiation oncolocgy
3
9. Lawton CA, Michalski J, El-Naqa I, et al. Variation in the definition of clinical target volumes for pelvic nodal conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2009;74: 377-382.
www.practicalradonc.org
Commentary
Toward safe and high quality care through peer review in radiation oncology: Need for more evidence Ronald C. Chen MD, MPH ⁎ Department of Radiation Oncology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina University of North Carolina-Lineberger Comprehensive Cancer Center, Chapel Hill, North Carolina Received 29 December 2013; accepted 30 December 2013
The planning process involved prior to starting radiation treatment, which can often take up to 1 to 2 weeks, and the daily nature of treatment for multiple weeks, provide a unique opportunity for radiation oncologists to perform peer review of treatment plans. Indeed, routine peer review occurs in the majority of radiation oncology centers. 1,2 In contrast to “morbidity and mortality” conferences, in which quality improvement and an infrastructure for learning is built around patient events that have already occurred, the goal of the peer review program in radiation oncology is to identify potential errors either before treatment starts or early enough during the treatment course to prevent harm to the patient. In the current era where there is an increasing emphasis on “quality” medical care and patient safety, radiation oncology leads by example with our longstanding culture for peer review and creation of treatment summaries, the latter concisely documenting treatment received and helping prevent future unintentional reirradiation. While peer review within each radiation oncology center is common, Ballo et al (in this issue of the journal) describe a novel program where an academic center performs peer review remotely with its satellite centers. 3 In this MD Anderson Cancer Center program, which started in 2007, all nonpalliative cases from satellite centers were required to be presented during twice-a-week peer review conferences. For each patient, the presentation included relevant history and clinical data, overall ⁎ Corresponding author. Department of Radiation Oncology, University of North Carolina at Chapel Hill, 101 Manning Dr, CB #7512, Chapel Hill, NC 27516. E-mail address: [email protected] (R.C. Chen).
treatment plan (including uses of surgery, radiation, and chemotherapy), and radiation plan (including contours and dose–volume histograms). The main goals of this retrospective study were to determine feasibility of remote peer review as assessed by the proportion of cases from satellite centers which were reviewed, and assess whether this peer review program increased compliance with MD Anderson institutional guidelines that cover all aspects of patient management “from consultation and simulation to target contouring, treatment planning, and delivery.” 3 The authors found that this program was highly feasible: 91.8% of eligible cases were presented for review in the first year; by year 4, this increased to 97.3%. Guideline compliance also improved; the proportion of cases for which a change was recommended as a result of peer review decreased dramatically from year 1 (16.5%) to year 4 (7.8%). While peer review is common, it may be administratively difficult for small community centers. As part of the American Society for Radiation Oncology (ASTRO)'s Target Safety Campaign, a white paper was recently published entitled “Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: Executive summary.” 4 This ASTRO report recommended that peer review be considered part of standard practice; for small centers, it specifically recommended creation of peer review relationships with other centers. In light of these recommendations, the study by Ballo et al 3 is timely. While it was conducted within a group of centers that have a unique main center–satellite relationship, where all physicians were obligated to participate in this peer review program and follow institutional guidelines, the study clearly demonstrated that the technologic and personnel infrastructure can be
1879-8500/$ – see front matter © 2014 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.prro.2013.12.006
2
R.C. Chen
created to successfully conduct remote peer review for every curative-intent patient, even in centers with large patient volumes. The practical implication of the study is that peer review can be accessible to every center—large or small, urban or rural—and for every cancer patient treated with radiation therapy. If desired, peer review partnerships should also be feasible between academic and community centers, even if the centers do not have a formal affiliation. Because errors do occur—various studies in radiation oncology have reported 4% to 12% prescription errors or treatment changes resulting from peer review 3,5,6—an ability to offer peer review for all patients will likely improve patient care and outcomes. Long term, this ability to perform remote peer review in radiation oncology has significant promise to reduce disparities in cancer care. Ballo et al 3 clearly show that peer review affects patient care, but demonstrating an impact on patient outcomes is much more difficult. In the oncologic literature there are many studies examining guideline compliance as a surrogate for “high quality” care, likely because the former is easier to measure and quantify. “Quality,” on the other hand, is a much more elusive concept, but consistent with the primary goal for peer review should be directly tied to patient outcomes. While guideline compliance is related to high quality care, there are important limitations to using this surrogate which must be considered when interpreting results of published studies, including this one. 7 First, guidelines are based on a combination of research evidence and expert opinion; therefore, many instances of guideline noncompliance may not constitute poor quality care. A recent analysis of the National Comprehensive Cancer Network guidelines found that only 6% of recommendations were based on level 1 evidence. 8 Guidelines on target contouring in radiation oncology are probably more likely to be based on expert opinion rather than high-level research evidence, and different experts can have significantly different target definitions. 9 Further, many patients may not fit neatly into guideline categories, and patient care often involves decisions that do not have clearly right or wrong answers. 4 Given these uncertainties, it is unclear what level of guideline compliance should be acceptable or whether guideline compliance should even be the central measure in peer review. In clinical scenarios where there is currently a knowledge gap and more than 1 acceptable treatment option, such as whether to include nodal irradiation for high-risk prostate cancer, a strict requirement for consistency through guideline compliance may prevent opportunities for future knowledge gained through retrospective outcomes reviews of patients treated differently. In the MD Anderson program, if the primary goal is to provide MD Anderson-style radiation treatment at all satellite centers, then a requirement for institutional guideline compliance is logical. However, on a broader
Practical Radiation Oncology: Month 2014
level of using peer review to prevent medical errors, and in the future where guideline compliance may be tied to reimbursement, the strength of evidence that forms a particular guideline must be considered. Compliance with recommendations based on level 1 evidence should be high, although given unique patient considerations and a need for individualized decision making, the bar should not be 100%. Another approach to peer review would be to assess whether each patient's treatment plan is “reasonable,” allowing for practice variation where there is currently no clear right versus wrong approach (each of us can probably think of innumerable such clinical scenarios) but making strong recommendations for change when errors have clearly been made. While peer review has been a longstanding part of the radiation oncology culture, there is a need for research on the “efficacy of peer review techniques and how these efforts can help improve the safety and quality” of radiation treatment. 4 The study by Ballo et al has answered this call by ASTRO, and many more studies are needed. There is wide variation on how peer review is conducted across centers, 1,2 and empiric data are needed to address many fundamental but unanswered questions. Which curative or palliative patients should be reviewed? When should peer review occur—before or after treatment initiation? What components of the treatment plan should be reviewed? How much does peer review cost, and perhaps the hardest question to answer, how much do patients benefit? Studies that assess patient outcomes, in addition to processes, are needed. With an emphasis on research efforts, radiation oncology can continue to lead in the area of patient quality and safety and make important contributions to medicine when our practices and research findings translate into practice in other specialties.
References 1. Lawrence YR, Whiton MA, Symon Z, et al. Quality assurance peer review chart rounds in 2011: A survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys. 2012;84:590-595. 2. Hoopes D, Johnstone P, Chapin P, et al. Early results from the ASTRO Practice Patterns in Peer Review Survey Project. Int J Radiation Oncol Biol Phys. 2013;87:S119. 3. Ballo M, Chronowski G, Schlembach P, et al. Prospective peer review quality assurance for outpatient radiation therapy. Pract Radiat Oncol. 2013 Dec 20 [Epub ahead of print] http://dx.doi.org/10.1016/j. prro.2013.11.004. 4. Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of caseoriented peer review to improve quality and safety in radiation oncology: Executive summary. Pract Radiat Oncol. 2013;3:149-156. 5. Boxer M, Forstner D, Kneebone A, et al. Impact of a real-time peer review audit on patient management in a radiation oncology department. J Med Imaging Radiat Oncol. 2009;53:405-411. 6. Brundage MD, Dixon PF, Mackillop WJ, et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys. 1999;43:115-124. 7. Chen RC. Guideline-adherent care vs quality care in cancer patients: Twins or distant cousins? JAMA Intern Med. 2013;173:569-570.
Practical Radiation Oncology: Month 2014 8. Poonacha TK, Go RS. Level of scientific evidence underlying recommendations arising from the National Comprehensive Cancer Network clinical practice guidelines. J Clin Oncol. 2011;29: 186-191.
Peer review in radiation oncolocgy
3
9. Lawton CA, Michalski J, El-Naqa I, et al. Variation in the definition of clinical target volumes for pelvic nodal conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2009;74: 377-382.
