Focus on Quality
Original Contribution
Changes in the Care of Non–Small-Cell Lung Cancer After Audit and Feedback: The Florida Initiative for Quality Cancer Care
H. Lee Moffitt Cancer Center and Research Institute, Tampa; Center for Cancer Care and Research at Watson Clinic, Lakeland; Florida Cancer Specialists and Research Institute, Sarasota; Spacecoast Cancer Center, Titusville; Ocala Oncology, Ocala; Coastal Oncology and Hematology, Stuart; Mayo Clinic, Jacksonville; Cancer Center of Florida, Orlando; North Broward Medical Center, Deerfield Beach; Tallahassee Memorial Healthcare, Tallahassee; and University of Florida, Gainesville, FL
Results: A statistically significant increase in adherence was
Abstract Purpose: Audit and feedback have been widely used to enhance the performance of various medical practices. Non–smallcell lung cancer (NSCLC) is one of the most common diseases encountered in medical oncology practice. We investigated the use of audit and feedback to improve the care of NSCLC.
Methods: Medical records were reviewed for patients with NSCLC first seen by a medical oncologist in 2006 (n ⫽ 518) and 2009 (n ⫽ 573) at 10 oncology practices participating in the Florida Initiative for Quality Cancer Care. In 2008, feedback from 2006 audit results was provided to practices, which then independently undertook steps to improve their performance. Sixteen quality-of-care indicators (QCIs) were evaluated on both time points and were examined for changes in adherence over time.
Introduction Non–small-cell lung cancer (NSCLC) is a commonly encountered cancer, with more than 400,000 new cases diagnosed yearly in the United States.1 Most patients with NSCLC, at some point in time, are cared for by a medical oncologist. This is in part because of the wide range of services medical oncologists offer, including diagnosis, staging, chemotherapy, and surveillance. The past decades have seen many changes in the management of NSCLC. To keep oncologists up to date with the rapid pace of research advances, several evidence- and consensus-based guidelines for the treatment of NSCLC have been issued by key organizations, such as the American Society of Clinical Oncology, the American College of Chest Physicians, and the National Comprehensive Cancer Center Network (NCCN).2-4 These practice guidelines hold the promise of optimizing patient outcomes by providing practitioners with the best available evidence to guide practice. Nevertheless, available data suggest that there exists substantial variation in the pattern of care for NSCLC. For example, a survey of oncology practices in 2006 on the use of adjuvant chemotherapy for NSCLC indicated that ⬍ 50% of patients Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
observed for five of 16 QCIs. Adherence to brain staging using magnetic resonance imaging or computed tomography scan for stage III NSCLC (57.8% in 2006 v 82.8% in 2009; P ⫽ .001), availability of chemotherapy flow sheet (89.2% v 97.0%; P ⬍ .001), documentation of performance status for stage III and IV disease (43.4% v 51.3%; P ⬍ .001), availability of pathology report for patients undergoing surgery (95.2% v 99.2%; P ⫽ .02), and availability of signed chemotherapy consent (69.5% v 76.3%; P ⫽ .04). There were no statistically significant decreases in adherence on any QCIs.
Conclusion: Audit with feedback was associated with a modest but important improvement in the treatment of NSCLC. Whether these changes are durable will require longterm follow-up.
who were eligible for adjuvant chemotherapy actually underwent the treatment.5 Specifically for elderly patients, the adherence rate to adjuvant chemotherapy was ⬍ 20%, although many phase III clinical trials showed an absolute 5-year survival benefit of 5.4% associated with adjuvant chemotherapy.6 In addition, other aspects of care, including pain control, seem to be suboptimal, especially in patients with lung cancer, when compared with patients with other cancers.7 Audits, defined as a summary of the clinical performance of a health care provider over a specified period, have been widely used as a strategy to improve professional practices.8 Practice audits have been shown to improve adherence to good clinical practice in medical conditions such as asthma9 or ducubitus ulcer.10 Audits provide an opportunity for health care workforces to learn about their performance against the accepted standard practice. In addition, the feedback can be shared among practices to enhance motivation. For medical oncology, some voluntary audit programs, such as the Quality Oncology Practice Initiative, have been associated with an improvement in adherence to established care guidelines.11-13 However, some of these programs lack baseline patient characteristics, thus •
jop.ascopubs.org
e247
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
By Tawee Tanvetyanon, MD, Ji-Hyun Lee, DrPH, William J. Fulp, MS, Fred Schreiber, MD, Richard H. Brown, MD, Richard M. Levine, MD, Thomas H. Cartwright, MD, Guillermo Abesada-Terk Jr, MD, George P. Kim, MD, Carlos Alemany, MD, Douglas Faig, MD, Philip V. Sharp, MD, Merry-Jennifer Markham, MD, Mokenge Malafa, MD, and Paul B. Jacobsen, PhD
Tanvetyanon et al
Methods Participating Sites
Disclosure of 2006 Findings
This report is part of a larger quality-of-care project, which also includes patients with colorectal and breast cancers previously detailed elsewhere.15 Participating sites were FIQCC practices that met the following criteria for initial participation: (1) medical oncology services provided by ⬎ one oncologist; (2) availability of a medical record abstractor; and (3) estimate of ⱖ 40 cases each of colorectal cancer, breast cancer, and NSCLC for calendar year 2006. In 2006, participating sites consisted of 11 medical oncology practices. This report constituted data from 10 practices with complete information available from both 2006 and 2009. The study received approval from the institutional review board at each participating institution. On the basis of an exempt status, informed consent from patients was not required to retrospectively review medical records under Code of Federal Regulations Title 45 Part 46.116(d).
Measures of Adherence to Processes of Care Outcome of interest was adherence to the quality-of-care indicators (QCIs) for NSCLC. The methodology to derive these indicators was previously outlined.14 Briefly, candidate indicators were selected from literature review and evidence-based practice guidelines available in 2006, including those from the NCCN, the American College of Chest Physicians, the European Society for Medical Oncology, Cancer Care Ontario, and the European Lung Cancer Working Party.16-20 A list of candidate indicators was formulated into a survey, which was conducted among the 2007 NCCN NSCLC guideline-writing panelists. Only QCIs with high ratings and with no disagreement among panelists were retained.
Audit Medical records of patients with a diagnosis of NSCLC first seen by medical oncologists in 2006 and 2009 were randomly e248
JOURNAL
sampled for review. Each site provided a list of eligible patient records to be sampled. A random number– generating program was then used to identify the records to be reviewed up until the prespecified number of required records from that site was met. Records were excluded if patients were age ⬍ 18 years or diagnosed with types of cancer other than NSCLC. A medical record abstraction manual was created and followed to ensure that the procedure for abstraction was similar between 2006 and 2009. A computerized Web-based program was used for uniform data capture. Each facility provided a medical record abstractor who was trained by a chief record abstractor. Regular on-site visits by the chief record abstractor took place to ensure quality and progress in data acquisition. Abstracted and deidentified data were not entered into the data abstraction tool until inter-rater agreement was ⱖ 70% for five consecutive patient cases. To maintain patient privacy, deidentification of patients was performed at the practice site before transmission to central data management. Quality control of the data was maintained through two reviews performed by the abstractor trainer when each site had completed one third and two thirds of the abstraction of the assigned total number of records for review.
OF
ONCOLOGY PRACTICE
•
The results of the 2006 audit were disclosed among members and served as feedback. Each practice site was given a unique site name (eg, site A); thus, all results were revealed in a blinded fashion, with each site knowing only its own identity. After all results were disclosed, a representative from each practice site had the opportunity to discuss the strategy for maintaining or improving quality care at his or her facility by giving a presentation to the group. At least one delegate from each practice was required to attend this feedback-sharing meeting. FIQCC members held a meeting at the start of this study and once yearly thereafter. In each meeting, members were asked to present challenges unique to their practice with regard to care in NSCLC along with remedial strategies.
Statistical Analysis We used descriptive statistics and graphical illustrations to summarize all indicator variables. Compliance proportions with 95% CIs were calculated based on exact binomial distribution. Statistical comparisons across practice sites were made using Pearson’s 2 exact test, with Monte Carlo estimation for each 2006 and 2009 analysis. Statistical comparisons between 2006 and 2009 data across practice site were also made using Pearson’s 2 exact test. Multivariable logistic regression models were used to evaluate effects of time and practice site variation across time on key indicator variables. An interaction term of practice site and time was tested in a logistic regression model to evaluate the effect of practice site variation across time. Significant demographic confounders with time were also included in the logistic regression models. Firth’s penalized maximum likelihood estimation was use to fit the logistic regression models. A P value of .05 (two-sided test) was considered significant. All analyses were conducted using SAS software (version 9.3; SAS Institute, Cary, NC).
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
making comparison across time difficult. In addition, data on audits and feedback specific to NSCLC are still limited. It remains unclear if audits and feedback can change the pattern of care for this challenging disease and, if so, to what extent. The Florida Initiative for Quality Cancer Care (FIQCC), formed in 2004, is a group of academic and community-based medical oncology practices in Florida. Its goal is to improve the care of patients with cancer in the state. In 2008, the group evaluated quality of care by measuring adherence to care delivery processes among patients with NSCLC first seen by a medical oncologist in 2006 and found large variation in adherence to processes of care.14 After the disclosure of these findings, the group reevaluated the care delivered to patients with NSCLC first seen by a medical oncologist in 2009 at the same practices. The primary objective of this study was to understand if there were significant changes in care quality between 2006 and 2009. In addition, we examined whether observed differences were independent of patient characteristics that may have varied over time and whether the direction or magnitude of change over time varied significantly among the participating practices.
Lung Cancer Care Audit and Feedback
Table 1. Demographic and Clinical Characteristics of Patient Cases of NSCLC Within FIQCC Consortium Total (N ⴝ 1,091) Variable
No.
%
Year 2006 (n ⴝ 518)
Year 2009 (n ⴝ 573)
No.
No.
%
%
Age, years
.47
Median
68
69
67
Range
29-93
36-91
29-92
Sex
.90 524
48.0
250
48.3
274
47.8
Male
567
52.0
268
51.7
299
52.2
White
907
83.1
425
82.0
482
84.1
Black
57
5.2
29
5.6
28
4.9
1
0.1
0
0.0
1
0.2
Race
Asian
.81
9
0.8
5
1.0
4
0.7
117
10.7
59
11.4
58
10.1
Private
287
26.3
140
27.0
147
25.7
Medicare
Missing Health insurance
.003
679
62.2
338
65.3
341
59.5
Medicaid/charity
77
7.1
27
5.2
50
8.7
None/unclassified
48
4.4
13
2.5
35
6.1
156
14.3
72
13.9
84
14.7
Disease stage I
.19
II
80
7.3
31
6.0
49
8.6
III
255
23.4
132
25.5
123
21.5
IV or wet IIIB
557
51.1
255
49.2
302
52.7
43
3.9
28
5.4
15
2.6
Adenocarcinoma
494
45.3
200
38.6
294
51.3
Squamous cell
256
23.5
99
19.1
157
27.4
52
4.8
26
5.0
26
4.5
243
22.3
163
31.5
80
14.0
43
3.9
28
5.4
15
2.6
Missing†
⬍ .001
Histology
Large cell Nonspecified Missing
Abbreviations: FIQCC, Florida Initiative for Quality Cancer Care; NSCLC, non–small-cell lung cancer. * P values calculated using 2 test, with exact method and Monte Carlo estimation, excluded missing level for P value calculation. † No documented stage in 43 patient cases.
Results Patient Characteristics A total of 10 practice sites contributed 1,091 unique patient medical records for review and analysis (Table 1). The proportion of patients sampled from each practice site was comparable between 2006 and 2009, ranging from 14 to 60 patients per site in 2006 and 38 to 60 patients per site in 2009. The patient characteristics between the 2 years were similar, with the exception of health insurance status and histology. In 2009, there was a higher proportion of patients with Medicaid charity or selfpay insurance (7.1% v 5.2%; P ⫽ .003) and a lower proportion of unspecified histology (31.5% v 14.0%; P ⬍ .001) than in 2006.
Change in General Care General care for patients with NSCLC was assessed using five QCIs (Table 2). Overall, practices maintained a high adCopyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
herence to the documentation of staging information, availability of a pathology report, and pain assessment in both 2006 and 2009. There was a significant increase in adherence to the staging documentation, from 94.6% in 2006 to 97.4% in 2009 (P ⫽ .02). Although there was also a numeric increase in adherence to the availability of a pathology report, from 95.2% to 99.2%, this was not significant (P ⫽ .09). Regarding chemotherapy, in 2006, the availability of a flow sheet was 89.2%, and availability of signed consent was 69.5%. Several FIQCC members sought improvement by implementing a policy requiring these documents be present before chemotherapy was dispensed. In 2009, the availability of a flow sheet significantly increased to 97.0% (P ⬍ .001), and availability of signed consent increased to 76.3% (P ⫽ .04). Finally, assessment of smoking status was high at both time points: 97.8% in 2006 and 99.3% in 2009 (marginally significant at P ⫽ .07). •
jop.ascopubs.org
e249
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Female
American Indian
P*
Tanvetyanon et al
Table 2. Change in Adherence to Quality Indicators Between 2006 and 2009 Year 2006
Year 2009
Adherence (%)
No. of Eligible Patient Cases
Adherence (%)
No. of Eligible Patient Cases
Staging information documented within 1 month of first office visit
94.6
490 of 518
97.4
558 of 573
2.8
.02
For patients who underwent surgical resection of lung cancer, pathology report available in medical record
95.2
99 of 104
99.2
119 of 120
4
.09
For patients who received chemotherapy, flow sheet of chemotherapy and blood counts available
89.2
313 of 351
97.0
385 of 397
7.8
⬍ .001
For patients who received chemotherapy, signed patient consent for chemotherapy available
69.5
244 of 351
76.3
303 of 397
6.8
.04
Documentation of smoking status assessment
97.8
479 of 490
99.3
554 of 558
1.5
.07
For patients with stage I or II disease, surgical resection undertaken
68.9
71 of 103
73.7
98 of 133
4.8
.47
For patients who underwent surgical resection, surveillance CT scan performed within 6 months of surgery
63.4
45 of 71
77.6
76 of 98
14.2
.06
For patients who underwent surgical resection with negative margins, adjuvant radiation therapy not administered
91.2
62 of 68
97.8
89 of 91
6.6
.08
For patients who underwent surgical resection, ⱖ two mediastinal lymph nodes examined
76.3
74 of 97
84.8
95 of 112
8.5
.15
For patients with stage II or III disease who underwent surgical resection, physician discussed, recommended, or referred for adjuvant chemotherapy
83.3
45 of 54
82.5
52 of 63
⫺0.8
1.00
For patients with stage II or III disease who underwent surgical resection and received chemotherapy at facility, patients received cisplatin-based doublet
56.5
13 of 23
64.7
22 of 34
8.2
.59
For patients with unresected stage III disease, physician discussed, recommended, or referred for concurrent chemoradiation
85.6
86 of 97
92.1
93 of 101
6.5
.18
For patients referred for concurrent chemoradiation, brain staging with CT or MRI performed
57.8
48 of 83
82.8
77 of 93
25.0
⬍ .001
For patients with stage III or IV disease, performance status assessment documented
43.4
168 of 387
51.3
218 of 425
7.9
.03
For patients age ⬍ 70 years who received chemotherapy, platinum doublet regimen used
85.7
96 of 112
88.8
127 of 143
3.1
.57
For patients who received bevacizumab, histology was nonsquamous cell
96.3
26 of 27
98.0
50 of 51
1.7
1.00
Indicator
Change (%)*
P†
General quality of care
Locally advanced or advanced-stage NSCLC
Abbreviations: CT, computed tomography; MRI, magnetic resonance imaging; NSCLC, non–small-cell lung cancer. * Difference in percent adherence between 2009 and 2006. † Exact Pearson’s 2 P values computed using Monte Carlo estimation.
Change in Care of Early-Stage NSCLC The care specific to early-stage NSCLC was assessed using six QCIs (Table 2). For patients with stage I to II NSCLC, the proportions of patients who underwent surgical resection were not significantly different between 2006 and 2009 (68.9% v 73.7%, respectively; P ⫽ .47). In addition, among those who underwent surgical resection, the proportions of patients with ⱖ two mediastinal lymph nodes examined were not significantly different (76.3% in 2006 v 84.8% in 2009; P ⫽ .15). e250
JOURNAL
OF
ONCOLOGY PRACTICE
•
Likewise, the avoidance of adjuvant radiation therapy among patients who underwent surgical resection did not significantly change over time (91.2% in 2006 v 97.8% in 2009; P ⫽ .08). We found an improvement in follow-up computed tomography scan performed within 6 months of surgery for patients with early-stage NSCLC (63.4% in 2006 v 77.6% in 2009; marginally significant at P ⫽ .06). Finally, we found no significant change in referral for adjuvant chemotherapy (83.3% in 2006 v 82.5% in 2009; P ⫽ 1.00) or choice of cisplatin-based
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Early-stage NSCLC
Lung Cancer Care Audit and Feedback
A
Change in Care of Locally Advanced or Advanced NSCLC
Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
40
20
B
C
D
E
F
G
H
I
J
G
H
I
J
G
H
I
J
Site
B 100
80
60
40
20
0 A
B
C
D
E
F
Site
Adherence to Chemotherapy Flowsheet Availability (%)
C
Variability in Change in Adherence Across Practice Sites To examine if adherence changes over time were consistent across the 10 medical oncology practice sites, an interaction term of practice site and time was tested in logistic regression models (data not shown). There were significant differences among practices in the change in adherence rates for three QCIs: availability of chemotherapy consent (P ⫽ .02), performance status assessment (P ⬍ .001), and availability of a chemotherapy flow sheet (P ⫽ .03). As shown in Figure 1A, adherence to chemotherapy consent increased at several sites, remained relatively constant at other sites, and decreased markedly (⬎ 20%) at one site (site A). With regard to performance status, as shown in Figure 1B, adherence remained relatively constant at several sites and increased markedly (⬎ 20%) at sites C, I, and J. Finally, with regard to the availability of a chemotherapy flow sheet, as shown in Figure 1C, adherence increased markedly (⬎ 20%) at sites A, H, and J.
60
A
Multivariable Analysis of Change We examined the changes of adherence for each indicator that seemed to reach or approach statistical significance by controlling for patient health insurance status and tumor histology. Five general QCIs were examined in a multivariable model (Appendix Table A1, online only). A notable change was the increase in chemotherapy consent availability, which no longer reached statistical significance (P ⫽ .12). Among NSCLC-specific QCIs, four QCIs were examined in a multivariable model. A notable change was the avoidance of adjuvant radiation therapy for stage I to II NSCLC, which became statistically significant after controlling for patient health insurance and histology (P ⫽ .04).
80
0
Adherence to Performance Status Documentation (%)
The care of patients with locally advanced or advanced NSCLC was assessed using five QCIs (Table 2). Practices maintained high adherence to the use of concurrent chemoradiotherapy for unresected stage III NSCLC in both 2006 and 2009 (85.6% v 92.1%, respectively). In addition, among patients referred for concurrent chemoradiotherapy, there was a significant increase in adherence to brain imaging using computed tomography or magnetic resonance imaging scan (57.8% in 2006 v 82.8% in 2009; P ⬍ .001). Adherence to performance status assessment remained low both in 2006 and 2009, despite a statistically significant improvement (43.4% v 51.3%, respectively; P ⫽ .03). We found that adherence to the use of standard platinum-based doublet chemotherapy for patients age ⬍ 70 years was high in both 2006 and 2009 (85.7% v 88.8%, respectively). Finally, adherence to use of bevacizumab for nonsquamous carcinoma histology was high in both years (96.3% in 2006 v 98.0% in 2009, respectively).
100
100
80
60
40
20
0 A
B
C
D
E
F
Site Figure 1. Adherence across practice sites to (A) chemotherapy consent availability, (B) performance status documentation for advanced non–small-cell lung cancer, and (C) chemotherapy flow sheet availability. Gold bars, 2006; blue bars, 2009.
Discussion We report here on the quality of care in NSCLC among FIQCC members before and after an audit with feedback. In total, 16 care measures—five general measures and 11 NSCLC-specific measures—were evaluated. A significant improvement in the adherence rate occurred for five QCIs. The greatest improvement was noted for adherence to brain imaging in patients with stage III •
jop.ascopubs.org
e251
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Adherence to Chemotherapy Consent (%)
adjuvant chemotherapy regimen (56.5% in 2006 v 64.7% in 2009; P ⫽ .59).
Tanvetyanon et al
e252
JOURNAL
OF
ONCOLOGY PRACTICE
•
disease, which showed no change with time. In addition, although this study is among the largest quality improvement projects in lung cancer with individual baseline patient characteristics available to account for confounders, the number of patients available to evaluate several care measures was low, making statistical estimates imprecise. Finally, we were not able to control for improvements in medical record systems, including the implementation of electronic medical records, which may account for some of the improvement in measures related to assessment and documentation. In summary, audit and feedback seem to be a useful method to improve the quality of care in NSCLC among medical oncology practices in some aspects. In this study, the greatest magnitude of improvement was observed in adherence to brain staging for patients with stage III disease referred for concurrent chemoradiotherapy. The adherence rate to this measure uniformly increased in all practices and reached 100% in several practice sites, suggesting that this QCI is highly actionable. Whether these improvements will be sustained will require long-term follow-up. Future study is necessary to understand the degree to which improvement in care delivery processes translates into improvement in patient outcomes, such as survival. Acknowledgment Supported by an unrestricted grant from Pfizer. Authors’ Disclosures of Potential Conflicts of Interest Although all authors completed the disclosure declaration, the following author(s) and/or an author’s immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO’s conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors. Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Douglas Faig, Pfizer Honoraria: None Research Funding: Tawee Tanvetyanon, Pfizer; Ji-Hyun Lee, Pfizer; William J. Fulp, Pfizer; Fred Schreiber, Pfizer; Richard H. Brown, Pfizer; Thomas H. Cartwright, Pfizer; Guillermo Abesada-Terk Jr, Pfizer; George P. Kim, Pfizer; Carlos Alemany, Pfizer; Douglas Faig, Pfizer; Philip V. Sharp, Pfizer; Merry-Jennifer Markham, Pfizer; Mokenge Malafa, Pfizer; Paul B. Jacobsen, Pfizer Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None Author Contributions Conception and design: Paul B. Jacobsen Financial support: Paul B. Jacobsen Administrative support: Paul B. Jacobsen Data analysis and interpretation: All authors Manuscript writing: All authors Final approval of manuscript: All authors Corresponding author: Tawee Tanvetyanon, MD, H. Lee Moffitt Cancer Center, University of South Florida, 12902 Magnolia Dr, Tampa, FL 33612; e-mail: [email protected].
DOI: 10.1200/JOP.2013.001275; published online ahead of print at jop.ascopubs.org on April 15, 2014.
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
NSCLC referred for concurrent chemoradiotherapy (57.8% in 2006 v 82.8% in 2009; P ⬍ .001). Analyses indicated that the rate of improvement on this indicator did not differ significantly among practice sites. Another notable improvement was in adherence to performance status assessment in advanced-stage NSCLC. Although adherence to performance status significantly increased, the mean adherence rates still remained low in 2006 and 2009 (43.4% v 51.3%, respectively; P ⫽ .03). Our study is among the first to our knowledge to report on the use of audit and feedback to improve medical oncology practices specifically for the processes of care in NSCLC. Many reasons may explain the lack of improvement in the other nine QCIs. First, according to a systematic review of 140 randomized trials on the effect of audit and feedback, improvement is likely to be seen when the baseline performance is low.8 In our study, an adherence rate ⬎ 85% at baseline already existed for eight of 16 QCIs. Therefore, improvement on these indicators was less expected because of a ceiling effect. Second, audit and feedback are more likely to work when an actionable plan is provided.21 In our study, although the QCI related to brain staging for stage III NSCLC is actionable by medical oncologists, the QCIs related to surgical resection for stage I to II NSCLC and mediastinal lymph node examination are not within the scope of medical oncology practice. Third, specific medical contraindication and patient preference may limit the possibility of improvement in some QCIs.22 For example, cisplatin is not advisable for frail patients, because of its toxicity profile, and many patients, if given a choice, choose to avoid it.23 Although statistical improvement was seen in several aspects of care, the extent to which these changes are clinically significant is difficult to determine. We have only measured the process of care, not care outcomes,24 and whether the outcomes will represent good value is unknown.25 In fact, clinical benefit associated with improvement in adherence may not be easily quantified for some QCIs. For instance, the QCI on performance status assessment, although accepted as a good clinical practice, cannot be directly translated into any measurable clinical benefit. However, change in some measures, such as brain staging for patients with stage III disease, may be specifically translated into a reduction in the discovery of unexpected symptomatic brain metastasis during chemoradiotherapy for such patients. Previous studies have suggested that up to 20% of patients with apparent stage III NSCLC already have an asymptomatic brain metastasis at baseline.26,27 Such patients actually have stage IV NSCLC. Our study illustrates the feasibility of collaboration among various medical practices across the state to improve the care of patients with NSCLC. Nevertheless, a number of limitations must be considered. The observed improvement in adherence to QCIs may be simply a result of the time trend and not a result of the audit and feedback. For example, the increase in histologic specificity of NSCLC diagnosis was likely because the selection of chemotherapeutic agent for NSCLC has increasingly relied on histologic subtype.28 However, for some QCIs, time trend alone is an unlikely explanation. This is evident by comparing the magnitude of the change with other measures that are not affected by audit and feedback to medical oncologists, such as surgery for stage I to II
Lung Cancer Care Audit and Feedback
References 1. Siegel R, Naishadham D, Jemal A: Cancer statistics, 2013. CA Cancer J Clin 63:11-30, 2013 2. Ettinger DS, Akerley W, Borghaei H, et al: Non-small cell lung cancer. J Natl Compr Canc Netw 10:1236-1271, 2012 3. Alberts WM: Introduction to the third edition: Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:38S-40S, 2013 (suppl) 4. Azzoli CG, Temin S, Giaccone G: 2011 focused update of 2009 American Society of Clinical Oncology clinical practice guideline update on chemotherapy for stage IV non–small-cell lung cancer. J Oncol Pract 8:63-66, 2012
6. Pignon JP, Tribodet H, Scagliotti GV, et al: Lung adjuvant cisplatin evaluation: A pooled analysis by the LACE Collaborative Group. J Clin Oncol 26:3552-3559, 2008 7. Malin JL, O’Neill SM, Asch SM, et al: Quality of supportive care for patients with advanced cancer in a VA medical center. J Palliat Med 14:573-577, 2011 8. Ivers N, Jamtvedt G, Flottorp S, et al: Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 6:CD000259, 2012 9. Bell D, Layton AJ, Gabbay J: Use of a guideline based questionnaire to audit hospital care of acute asthma. BMJ 302:1440-1443, 1991 10. Sullivan N, Schoelles KM: Preventing in-facility pressure ulcers as a patient safety strategy: A systematic review. Ann Intern Med 158:410-416, 2013 11. Ang E, Chow YL: General pain assessment among patients with cancer in an acute care setting: A best practice implementation project. Int J Evid Based Healthc 8:90-96, 2010 12. Neuss MN, Malin JL, Chan S, et al: Measuring the improving quality of outpatient care in medical oncology practices in the United States. J Clin Oncol 31:1471-1477, 2013 13. Blayney DW, McNiff K, Hanauer D, et al: Implementation of the Quality Oncology Practice Initiative at a university comprehensive cancer center. J Clin Oncol 27:3802-3807, 2009 14. Tanvetyanon T, Corman M, Lee JH, et al: Quality of care in non–small-cell lung cancer: Findings from 11 oncology practices in Florida. J Oncol Pract 7:e25e31, 2011
Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
16. National Comprehensive Cancer Network: NCCN Clinical Practice Guidelines in Oncology: Non-Small Cell Lung Cancer. http://www.nccn.org 17. American College of Chest Physicians: Diagnosis and Management of Lung Cancer Executive Summary: ACCP Evidence-Based Clinical Practice Guidelines (ed 2) http://www.chestnet.org/accp/guidelines 18. European Society for Medical Oncology: ESMO Clinical Practice Guidelines. http://www.esmo.org/Guidelines-Practice/Clinical-Practice-Guidelines 19. Cancer Care Ontario: Quality Guidelines and Standards. http://www. cancercare.on.ca/toolbox/qualityguidelines/ 20. European Lung Cancer Working Party: Guidelines: Non-Small Cell Lung Cancer—Metastatic (advanced) Stages (October 2006). http://www.elcwp.org/ New/En/index.html 21. Hurst D: Audit and feedback had small but potentially important improvements in professional practice. Evid Based Dent 14:8-9, 2013 22. Ryoo JJ, Ordin DL, Antonio AL, et al: Patient preference and contraindications in measuring quality of care: What do administrative data miss? J Clin Oncol 31:2716-2723, 2013 23. Goffin J, Lacchetti C, Ellis PM, et al: First-line systemic chemotherapy in the treatment of advanced non-small cell lung cancer: A systematic review. J Thorac Oncol 5:260-274, 2010 24. Komaki R, Cox JD, Stark R: Frequency of brain metastasis in adenocarcinoma and large cell carcinoma of the lung: Correlation with survival. Int J Radiat Oncol Biol Phys 9:1467-1470, 1983 25. Komaki R, Scott CB, Byhardt R, et al: Failure patterns by prognostic group determined by recursive partitioning analysis (RPA) of 1547 patients on four radiation therapy oncology group (RTOG) studies in inoperable nonsmall-cell lung cancer (NSCLC). Int J Radiat Oncol Biol Phys 42:263-267, 1998 26. Donabedian A: The quality of care. How can it be assessed? JAMA 260: 1743-1748, 1988 27. Porter ME: What is value in health care? N Engl J Med 363:2477-2481, 2010 28. Scagliotti G, Brodowicz T, Shepherd FA, et al: Treatment-by-histology interaction analyses in three phase III trials show superiority of pemetrexed in nonsquamous non-small cell lung cancer. J Thorac Oncol 6:64-70, 2011
•
jop.ascopubs.org
e253
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
5. Cuffe S, Booth CM, Peng Y, et al: Adjuvant chemotherapy for non-small-cell lung cancer in the elderly: A population-based study in Ontario, Canada. J Clin Oncol 30:1813-1821, 2012
15. Siegel EM, Jacobsen PB, Lee JH, et al: Florida Initiative for Quality Cancer Care: Improvements on colorectal cancer quality care indicators over a 3-year interval. J Am Coll Surg 218:16-25, e1-e4, 2014
Tanvetyanon et al
Appendix Table A1. Multivariable Analysis Adjusting for Histology, Insurance Status, and Practice Site P 2006 Adherence (%)
2009 Adherence (%)
Staging information documented within 1 month of first office visit
94.6
97.4
2.8
.02
For patients who underwent surgical resection of lung cancer, pathology report available in medical record
95.2
99.2
4
.09
For patients who received chemotherapy, flow sheet of chemotherapy and blood counts available
89.2
97.0
7.8
⬍ .001
⬍ .001
For patients who received chemotherapy, signed patient consent for chemotherapy available
69.5
76.3
6.8
.04
.12
Documentation of smoking status assessment within 1 month of first office visit
97.8
99.3
1.5
.07
.06
For patients who underwent surgical resection, surveillance CT scan performed within 6 months of surgery
63.4
77.6
14.2
.06
For patients who underwent surgical resection with negative margins, adjuvant radiation therapy not administered
91.2
97.8
6.6
.08
.04
For patients referred for concurrent chemoradiation, brain staging with CT or MRI performed
57.8
82.8
25.0
⬍ .001
.001
For patients with stage III or IV disease, performance status assessment documented
43.4
51.3
7.9
.03
⬍ .001
Indicator
Change (%)
Univariable
Adjusted
General quality of care .02
NSCLC specific NA*
Abbreviations: CT, computed tomography; MRI, magnetic resonance imaging; NA, not available; NSCLC, non–small-cell lung cancer. * Model does not converge because of sparseness of data.
e254
JOURNAL
OF
ONCOLOGY PRACTICE
•
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
NA*
Original Contribution
Changes in the Care of Non–Small-Cell Lung Cancer After Audit and Feedback: The Florida Initiative for Quality Cancer Care
H. Lee Moffitt Cancer Center and Research Institute, Tampa; Center for Cancer Care and Research at Watson Clinic, Lakeland; Florida Cancer Specialists and Research Institute, Sarasota; Spacecoast Cancer Center, Titusville; Ocala Oncology, Ocala; Coastal Oncology and Hematology, Stuart; Mayo Clinic, Jacksonville; Cancer Center of Florida, Orlando; North Broward Medical Center, Deerfield Beach; Tallahassee Memorial Healthcare, Tallahassee; and University of Florida, Gainesville, FL
Results: A statistically significant increase in adherence was
Abstract Purpose: Audit and feedback have been widely used to enhance the performance of various medical practices. Non–smallcell lung cancer (NSCLC) is one of the most common diseases encountered in medical oncology practice. We investigated the use of audit and feedback to improve the care of NSCLC.
Methods: Medical records were reviewed for patients with NSCLC first seen by a medical oncologist in 2006 (n ⫽ 518) and 2009 (n ⫽ 573) at 10 oncology practices participating in the Florida Initiative for Quality Cancer Care. In 2008, feedback from 2006 audit results was provided to practices, which then independently undertook steps to improve their performance. Sixteen quality-of-care indicators (QCIs) were evaluated on both time points and were examined for changes in adherence over time.
Introduction Non–small-cell lung cancer (NSCLC) is a commonly encountered cancer, with more than 400,000 new cases diagnosed yearly in the United States.1 Most patients with NSCLC, at some point in time, are cared for by a medical oncologist. This is in part because of the wide range of services medical oncologists offer, including diagnosis, staging, chemotherapy, and surveillance. The past decades have seen many changes in the management of NSCLC. To keep oncologists up to date with the rapid pace of research advances, several evidence- and consensus-based guidelines for the treatment of NSCLC have been issued by key organizations, such as the American Society of Clinical Oncology, the American College of Chest Physicians, and the National Comprehensive Cancer Center Network (NCCN).2-4 These practice guidelines hold the promise of optimizing patient outcomes by providing practitioners with the best available evidence to guide practice. Nevertheless, available data suggest that there exists substantial variation in the pattern of care for NSCLC. For example, a survey of oncology practices in 2006 on the use of adjuvant chemotherapy for NSCLC indicated that ⬍ 50% of patients Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
observed for five of 16 QCIs. Adherence to brain staging using magnetic resonance imaging or computed tomography scan for stage III NSCLC (57.8% in 2006 v 82.8% in 2009; P ⫽ .001), availability of chemotherapy flow sheet (89.2% v 97.0%; P ⬍ .001), documentation of performance status for stage III and IV disease (43.4% v 51.3%; P ⬍ .001), availability of pathology report for patients undergoing surgery (95.2% v 99.2%; P ⫽ .02), and availability of signed chemotherapy consent (69.5% v 76.3%; P ⫽ .04). There were no statistically significant decreases in adherence on any QCIs.
Conclusion: Audit with feedback was associated with a modest but important improvement in the treatment of NSCLC. Whether these changes are durable will require longterm follow-up.
who were eligible for adjuvant chemotherapy actually underwent the treatment.5 Specifically for elderly patients, the adherence rate to adjuvant chemotherapy was ⬍ 20%, although many phase III clinical trials showed an absolute 5-year survival benefit of 5.4% associated with adjuvant chemotherapy.6 In addition, other aspects of care, including pain control, seem to be suboptimal, especially in patients with lung cancer, when compared with patients with other cancers.7 Audits, defined as a summary of the clinical performance of a health care provider over a specified period, have been widely used as a strategy to improve professional practices.8 Practice audits have been shown to improve adherence to good clinical practice in medical conditions such as asthma9 or ducubitus ulcer.10 Audits provide an opportunity for health care workforces to learn about their performance against the accepted standard practice. In addition, the feedback can be shared among practices to enhance motivation. For medical oncology, some voluntary audit programs, such as the Quality Oncology Practice Initiative, have been associated with an improvement in adherence to established care guidelines.11-13 However, some of these programs lack baseline patient characteristics, thus •
jop.ascopubs.org
e247
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
By Tawee Tanvetyanon, MD, Ji-Hyun Lee, DrPH, William J. Fulp, MS, Fred Schreiber, MD, Richard H. Brown, MD, Richard M. Levine, MD, Thomas H. Cartwright, MD, Guillermo Abesada-Terk Jr, MD, George P. Kim, MD, Carlos Alemany, MD, Douglas Faig, MD, Philip V. Sharp, MD, Merry-Jennifer Markham, MD, Mokenge Malafa, MD, and Paul B. Jacobsen, PhD
Tanvetyanon et al
Methods Participating Sites
Disclosure of 2006 Findings
This report is part of a larger quality-of-care project, which also includes patients with colorectal and breast cancers previously detailed elsewhere.15 Participating sites were FIQCC practices that met the following criteria for initial participation: (1) medical oncology services provided by ⬎ one oncologist; (2) availability of a medical record abstractor; and (3) estimate of ⱖ 40 cases each of colorectal cancer, breast cancer, and NSCLC for calendar year 2006. In 2006, participating sites consisted of 11 medical oncology practices. This report constituted data from 10 practices with complete information available from both 2006 and 2009. The study received approval from the institutional review board at each participating institution. On the basis of an exempt status, informed consent from patients was not required to retrospectively review medical records under Code of Federal Regulations Title 45 Part 46.116(d).
Measures of Adherence to Processes of Care Outcome of interest was adherence to the quality-of-care indicators (QCIs) for NSCLC. The methodology to derive these indicators was previously outlined.14 Briefly, candidate indicators were selected from literature review and evidence-based practice guidelines available in 2006, including those from the NCCN, the American College of Chest Physicians, the European Society for Medical Oncology, Cancer Care Ontario, and the European Lung Cancer Working Party.16-20 A list of candidate indicators was formulated into a survey, which was conducted among the 2007 NCCN NSCLC guideline-writing panelists. Only QCIs with high ratings and with no disagreement among panelists were retained.
Audit Medical records of patients with a diagnosis of NSCLC first seen by medical oncologists in 2006 and 2009 were randomly e248
JOURNAL
sampled for review. Each site provided a list of eligible patient records to be sampled. A random number– generating program was then used to identify the records to be reviewed up until the prespecified number of required records from that site was met. Records were excluded if patients were age ⬍ 18 years or diagnosed with types of cancer other than NSCLC. A medical record abstraction manual was created and followed to ensure that the procedure for abstraction was similar between 2006 and 2009. A computerized Web-based program was used for uniform data capture. Each facility provided a medical record abstractor who was trained by a chief record abstractor. Regular on-site visits by the chief record abstractor took place to ensure quality and progress in data acquisition. Abstracted and deidentified data were not entered into the data abstraction tool until inter-rater agreement was ⱖ 70% for five consecutive patient cases. To maintain patient privacy, deidentification of patients was performed at the practice site before transmission to central data management. Quality control of the data was maintained through two reviews performed by the abstractor trainer when each site had completed one third and two thirds of the abstraction of the assigned total number of records for review.
OF
ONCOLOGY PRACTICE
•
The results of the 2006 audit were disclosed among members and served as feedback. Each practice site was given a unique site name (eg, site A); thus, all results were revealed in a blinded fashion, with each site knowing only its own identity. After all results were disclosed, a representative from each practice site had the opportunity to discuss the strategy for maintaining or improving quality care at his or her facility by giving a presentation to the group. At least one delegate from each practice was required to attend this feedback-sharing meeting. FIQCC members held a meeting at the start of this study and once yearly thereafter. In each meeting, members were asked to present challenges unique to their practice with regard to care in NSCLC along with remedial strategies.
Statistical Analysis We used descriptive statistics and graphical illustrations to summarize all indicator variables. Compliance proportions with 95% CIs were calculated based on exact binomial distribution. Statistical comparisons across practice sites were made using Pearson’s 2 exact test, with Monte Carlo estimation for each 2006 and 2009 analysis. Statistical comparisons between 2006 and 2009 data across practice site were also made using Pearson’s 2 exact test. Multivariable logistic regression models were used to evaluate effects of time and practice site variation across time on key indicator variables. An interaction term of practice site and time was tested in a logistic regression model to evaluate the effect of practice site variation across time. Significant demographic confounders with time were also included in the logistic regression models. Firth’s penalized maximum likelihood estimation was use to fit the logistic regression models. A P value of .05 (two-sided test) was considered significant. All analyses were conducted using SAS software (version 9.3; SAS Institute, Cary, NC).
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
making comparison across time difficult. In addition, data on audits and feedback specific to NSCLC are still limited. It remains unclear if audits and feedback can change the pattern of care for this challenging disease and, if so, to what extent. The Florida Initiative for Quality Cancer Care (FIQCC), formed in 2004, is a group of academic and community-based medical oncology practices in Florida. Its goal is to improve the care of patients with cancer in the state. In 2008, the group evaluated quality of care by measuring adherence to care delivery processes among patients with NSCLC first seen by a medical oncologist in 2006 and found large variation in adherence to processes of care.14 After the disclosure of these findings, the group reevaluated the care delivered to patients with NSCLC first seen by a medical oncologist in 2009 at the same practices. The primary objective of this study was to understand if there were significant changes in care quality between 2006 and 2009. In addition, we examined whether observed differences were independent of patient characteristics that may have varied over time and whether the direction or magnitude of change over time varied significantly among the participating practices.
Lung Cancer Care Audit and Feedback
Table 1. Demographic and Clinical Characteristics of Patient Cases of NSCLC Within FIQCC Consortium Total (N ⴝ 1,091) Variable
No.
%
Year 2006 (n ⴝ 518)
Year 2009 (n ⴝ 573)
No.
No.
%
%
Age, years
.47
Median
68
69
67
Range
29-93
36-91
29-92
Sex
.90 524
48.0
250
48.3
274
47.8
Male
567
52.0
268
51.7
299
52.2
White
907
83.1
425
82.0
482
84.1
Black
57
5.2
29
5.6
28
4.9
1
0.1
0
0.0
1
0.2
Race
Asian
.81
9
0.8
5
1.0
4
0.7
117
10.7
59
11.4
58
10.1
Private
287
26.3
140
27.0
147
25.7
Medicare
Missing Health insurance
.003
679
62.2
338
65.3
341
59.5
Medicaid/charity
77
7.1
27
5.2
50
8.7
None/unclassified
48
4.4
13
2.5
35
6.1
156
14.3
72
13.9
84
14.7
Disease stage I
.19
II
80
7.3
31
6.0
49
8.6
III
255
23.4
132
25.5
123
21.5
IV or wet IIIB
557
51.1
255
49.2
302
52.7
43
3.9
28
5.4
15
2.6
Adenocarcinoma
494
45.3
200
38.6
294
51.3
Squamous cell
256
23.5
99
19.1
157
27.4
52
4.8
26
5.0
26
4.5
243
22.3
163
31.5
80
14.0
43
3.9
28
5.4
15
2.6
Missing†
⬍ .001
Histology
Large cell Nonspecified Missing
Abbreviations: FIQCC, Florida Initiative for Quality Cancer Care; NSCLC, non–small-cell lung cancer. * P values calculated using 2 test, with exact method and Monte Carlo estimation, excluded missing level for P value calculation. † No documented stage in 43 patient cases.
Results Patient Characteristics A total of 10 practice sites contributed 1,091 unique patient medical records for review and analysis (Table 1). The proportion of patients sampled from each practice site was comparable between 2006 and 2009, ranging from 14 to 60 patients per site in 2006 and 38 to 60 patients per site in 2009. The patient characteristics between the 2 years were similar, with the exception of health insurance status and histology. In 2009, there was a higher proportion of patients with Medicaid charity or selfpay insurance (7.1% v 5.2%; P ⫽ .003) and a lower proportion of unspecified histology (31.5% v 14.0%; P ⬍ .001) than in 2006.
Change in General Care General care for patients with NSCLC was assessed using five QCIs (Table 2). Overall, practices maintained a high adCopyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
herence to the documentation of staging information, availability of a pathology report, and pain assessment in both 2006 and 2009. There was a significant increase in adherence to the staging documentation, from 94.6% in 2006 to 97.4% in 2009 (P ⫽ .02). Although there was also a numeric increase in adherence to the availability of a pathology report, from 95.2% to 99.2%, this was not significant (P ⫽ .09). Regarding chemotherapy, in 2006, the availability of a flow sheet was 89.2%, and availability of signed consent was 69.5%. Several FIQCC members sought improvement by implementing a policy requiring these documents be present before chemotherapy was dispensed. In 2009, the availability of a flow sheet significantly increased to 97.0% (P ⬍ .001), and availability of signed consent increased to 76.3% (P ⫽ .04). Finally, assessment of smoking status was high at both time points: 97.8% in 2006 and 99.3% in 2009 (marginally significant at P ⫽ .07). •
jop.ascopubs.org
e249
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Female
American Indian
P*
Tanvetyanon et al
Table 2. Change in Adherence to Quality Indicators Between 2006 and 2009 Year 2006
Year 2009
Adherence (%)
No. of Eligible Patient Cases
Adherence (%)
No. of Eligible Patient Cases
Staging information documented within 1 month of first office visit
94.6
490 of 518
97.4
558 of 573
2.8
.02
For patients who underwent surgical resection of lung cancer, pathology report available in medical record
95.2
99 of 104
99.2
119 of 120
4
.09
For patients who received chemotherapy, flow sheet of chemotherapy and blood counts available
89.2
313 of 351
97.0
385 of 397
7.8
⬍ .001
For patients who received chemotherapy, signed patient consent for chemotherapy available
69.5
244 of 351
76.3
303 of 397
6.8
.04
Documentation of smoking status assessment
97.8
479 of 490
99.3
554 of 558
1.5
.07
For patients with stage I or II disease, surgical resection undertaken
68.9
71 of 103
73.7
98 of 133
4.8
.47
For patients who underwent surgical resection, surveillance CT scan performed within 6 months of surgery
63.4
45 of 71
77.6
76 of 98
14.2
.06
For patients who underwent surgical resection with negative margins, adjuvant radiation therapy not administered
91.2
62 of 68
97.8
89 of 91
6.6
.08
For patients who underwent surgical resection, ⱖ two mediastinal lymph nodes examined
76.3
74 of 97
84.8
95 of 112
8.5
.15
For patients with stage II or III disease who underwent surgical resection, physician discussed, recommended, or referred for adjuvant chemotherapy
83.3
45 of 54
82.5
52 of 63
⫺0.8
1.00
For patients with stage II or III disease who underwent surgical resection and received chemotherapy at facility, patients received cisplatin-based doublet
56.5
13 of 23
64.7
22 of 34
8.2
.59
For patients with unresected stage III disease, physician discussed, recommended, or referred for concurrent chemoradiation
85.6
86 of 97
92.1
93 of 101
6.5
.18
For patients referred for concurrent chemoradiation, brain staging with CT or MRI performed
57.8
48 of 83
82.8
77 of 93
25.0
⬍ .001
For patients with stage III or IV disease, performance status assessment documented
43.4
168 of 387
51.3
218 of 425
7.9
.03
For patients age ⬍ 70 years who received chemotherapy, platinum doublet regimen used
85.7
96 of 112
88.8
127 of 143
3.1
.57
For patients who received bevacizumab, histology was nonsquamous cell
96.3
26 of 27
98.0
50 of 51
1.7
1.00
Indicator
Change (%)*
P†
General quality of care
Locally advanced or advanced-stage NSCLC
Abbreviations: CT, computed tomography; MRI, magnetic resonance imaging; NSCLC, non–small-cell lung cancer. * Difference in percent adherence between 2009 and 2006. † Exact Pearson’s 2 P values computed using Monte Carlo estimation.
Change in Care of Early-Stage NSCLC The care specific to early-stage NSCLC was assessed using six QCIs (Table 2). For patients with stage I to II NSCLC, the proportions of patients who underwent surgical resection were not significantly different between 2006 and 2009 (68.9% v 73.7%, respectively; P ⫽ .47). In addition, among those who underwent surgical resection, the proportions of patients with ⱖ two mediastinal lymph nodes examined were not significantly different (76.3% in 2006 v 84.8% in 2009; P ⫽ .15). e250
JOURNAL
OF
ONCOLOGY PRACTICE
•
Likewise, the avoidance of adjuvant radiation therapy among patients who underwent surgical resection did not significantly change over time (91.2% in 2006 v 97.8% in 2009; P ⫽ .08). We found an improvement in follow-up computed tomography scan performed within 6 months of surgery for patients with early-stage NSCLC (63.4% in 2006 v 77.6% in 2009; marginally significant at P ⫽ .06). Finally, we found no significant change in referral for adjuvant chemotherapy (83.3% in 2006 v 82.5% in 2009; P ⫽ 1.00) or choice of cisplatin-based
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Early-stage NSCLC
Lung Cancer Care Audit and Feedback
A
Change in Care of Locally Advanced or Advanced NSCLC
Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
40
20
B
C
D
E
F
G
H
I
J
G
H
I
J
G
H
I
J
Site
B 100
80
60
40
20
0 A
B
C
D
E
F
Site
Adherence to Chemotherapy Flowsheet Availability (%)
C
Variability in Change in Adherence Across Practice Sites To examine if adherence changes over time were consistent across the 10 medical oncology practice sites, an interaction term of practice site and time was tested in logistic regression models (data not shown). There were significant differences among practices in the change in adherence rates for three QCIs: availability of chemotherapy consent (P ⫽ .02), performance status assessment (P ⬍ .001), and availability of a chemotherapy flow sheet (P ⫽ .03). As shown in Figure 1A, adherence to chemotherapy consent increased at several sites, remained relatively constant at other sites, and decreased markedly (⬎ 20%) at one site (site A). With regard to performance status, as shown in Figure 1B, adherence remained relatively constant at several sites and increased markedly (⬎ 20%) at sites C, I, and J. Finally, with regard to the availability of a chemotherapy flow sheet, as shown in Figure 1C, adherence increased markedly (⬎ 20%) at sites A, H, and J.
60
A
Multivariable Analysis of Change We examined the changes of adherence for each indicator that seemed to reach or approach statistical significance by controlling for patient health insurance status and tumor histology. Five general QCIs were examined in a multivariable model (Appendix Table A1, online only). A notable change was the increase in chemotherapy consent availability, which no longer reached statistical significance (P ⫽ .12). Among NSCLC-specific QCIs, four QCIs were examined in a multivariable model. A notable change was the avoidance of adjuvant radiation therapy for stage I to II NSCLC, which became statistically significant after controlling for patient health insurance and histology (P ⫽ .04).
80
0
Adherence to Performance Status Documentation (%)
The care of patients with locally advanced or advanced NSCLC was assessed using five QCIs (Table 2). Practices maintained high adherence to the use of concurrent chemoradiotherapy for unresected stage III NSCLC in both 2006 and 2009 (85.6% v 92.1%, respectively). In addition, among patients referred for concurrent chemoradiotherapy, there was a significant increase in adherence to brain imaging using computed tomography or magnetic resonance imaging scan (57.8% in 2006 v 82.8% in 2009; P ⬍ .001). Adherence to performance status assessment remained low both in 2006 and 2009, despite a statistically significant improvement (43.4% v 51.3%, respectively; P ⫽ .03). We found that adherence to the use of standard platinum-based doublet chemotherapy for patients age ⬍ 70 years was high in both 2006 and 2009 (85.7% v 88.8%, respectively). Finally, adherence to use of bevacizumab for nonsquamous carcinoma histology was high in both years (96.3% in 2006 v 98.0% in 2009, respectively).
100
100
80
60
40
20
0 A
B
C
D
E
F
Site Figure 1. Adherence across practice sites to (A) chemotherapy consent availability, (B) performance status documentation for advanced non–small-cell lung cancer, and (C) chemotherapy flow sheet availability. Gold bars, 2006; blue bars, 2009.
Discussion We report here on the quality of care in NSCLC among FIQCC members before and after an audit with feedback. In total, 16 care measures—five general measures and 11 NSCLC-specific measures—were evaluated. A significant improvement in the adherence rate occurred for five QCIs. The greatest improvement was noted for adherence to brain imaging in patients with stage III •
jop.ascopubs.org
e251
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
Adherence to Chemotherapy Consent (%)
adjuvant chemotherapy regimen (56.5% in 2006 v 64.7% in 2009; P ⫽ .59).
Tanvetyanon et al
e252
JOURNAL
OF
ONCOLOGY PRACTICE
•
disease, which showed no change with time. In addition, although this study is among the largest quality improvement projects in lung cancer with individual baseline patient characteristics available to account for confounders, the number of patients available to evaluate several care measures was low, making statistical estimates imprecise. Finally, we were not able to control for improvements in medical record systems, including the implementation of electronic medical records, which may account for some of the improvement in measures related to assessment and documentation. In summary, audit and feedback seem to be a useful method to improve the quality of care in NSCLC among medical oncology practices in some aspects. In this study, the greatest magnitude of improvement was observed in adherence to brain staging for patients with stage III disease referred for concurrent chemoradiotherapy. The adherence rate to this measure uniformly increased in all practices and reached 100% in several practice sites, suggesting that this QCI is highly actionable. Whether these improvements will be sustained will require long-term follow-up. Future study is necessary to understand the degree to which improvement in care delivery processes translates into improvement in patient outcomes, such as survival. Acknowledgment Supported by an unrestricted grant from Pfizer. Authors’ Disclosures of Potential Conflicts of Interest Although all authors completed the disclosure declaration, the following author(s) and/or an author’s immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO’s conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors. Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Douglas Faig, Pfizer Honoraria: None Research Funding: Tawee Tanvetyanon, Pfizer; Ji-Hyun Lee, Pfizer; William J. Fulp, Pfizer; Fred Schreiber, Pfizer; Richard H. Brown, Pfizer; Thomas H. Cartwright, Pfizer; Guillermo Abesada-Terk Jr, Pfizer; George P. Kim, Pfizer; Carlos Alemany, Pfizer; Douglas Faig, Pfizer; Philip V. Sharp, Pfizer; Merry-Jennifer Markham, Pfizer; Mokenge Malafa, Pfizer; Paul B. Jacobsen, Pfizer Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None Author Contributions Conception and design: Paul B. Jacobsen Financial support: Paul B. Jacobsen Administrative support: Paul B. Jacobsen Data analysis and interpretation: All authors Manuscript writing: All authors Final approval of manuscript: All authors Corresponding author: Tawee Tanvetyanon, MD, H. Lee Moffitt Cancer Center, University of South Florida, 12902 Magnolia Dr, Tampa, FL 33612; e-mail: [email protected].
DOI: 10.1200/JOP.2013.001275; published online ahead of print at jop.ascopubs.org on April 15, 2014.
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
NSCLC referred for concurrent chemoradiotherapy (57.8% in 2006 v 82.8% in 2009; P ⬍ .001). Analyses indicated that the rate of improvement on this indicator did not differ significantly among practice sites. Another notable improvement was in adherence to performance status assessment in advanced-stage NSCLC. Although adherence to performance status significantly increased, the mean adherence rates still remained low in 2006 and 2009 (43.4% v 51.3%, respectively; P ⫽ .03). Our study is among the first to our knowledge to report on the use of audit and feedback to improve medical oncology practices specifically for the processes of care in NSCLC. Many reasons may explain the lack of improvement in the other nine QCIs. First, according to a systematic review of 140 randomized trials on the effect of audit and feedback, improvement is likely to be seen when the baseline performance is low.8 In our study, an adherence rate ⬎ 85% at baseline already existed for eight of 16 QCIs. Therefore, improvement on these indicators was less expected because of a ceiling effect. Second, audit and feedback are more likely to work when an actionable plan is provided.21 In our study, although the QCI related to brain staging for stage III NSCLC is actionable by medical oncologists, the QCIs related to surgical resection for stage I to II NSCLC and mediastinal lymph node examination are not within the scope of medical oncology practice. Third, specific medical contraindication and patient preference may limit the possibility of improvement in some QCIs.22 For example, cisplatin is not advisable for frail patients, because of its toxicity profile, and many patients, if given a choice, choose to avoid it.23 Although statistical improvement was seen in several aspects of care, the extent to which these changes are clinically significant is difficult to determine. We have only measured the process of care, not care outcomes,24 and whether the outcomes will represent good value is unknown.25 In fact, clinical benefit associated with improvement in adherence may not be easily quantified for some QCIs. For instance, the QCI on performance status assessment, although accepted as a good clinical practice, cannot be directly translated into any measurable clinical benefit. However, change in some measures, such as brain staging for patients with stage III disease, may be specifically translated into a reduction in the discovery of unexpected symptomatic brain metastasis during chemoradiotherapy for such patients. Previous studies have suggested that up to 20% of patients with apparent stage III NSCLC already have an asymptomatic brain metastasis at baseline.26,27 Such patients actually have stage IV NSCLC. Our study illustrates the feasibility of collaboration among various medical practices across the state to improve the care of patients with NSCLC. Nevertheless, a number of limitations must be considered. The observed improvement in adherence to QCIs may be simply a result of the time trend and not a result of the audit and feedback. For example, the increase in histologic specificity of NSCLC diagnosis was likely because the selection of chemotherapeutic agent for NSCLC has increasingly relied on histologic subtype.28 However, for some QCIs, time trend alone is an unlikely explanation. This is evident by comparing the magnitude of the change with other measures that are not affected by audit and feedback to medical oncologists, such as surgery for stage I to II
Lung Cancer Care Audit and Feedback
References 1. Siegel R, Naishadham D, Jemal A: Cancer statistics, 2013. CA Cancer J Clin 63:11-30, 2013 2. Ettinger DS, Akerley W, Borghaei H, et al: Non-small cell lung cancer. J Natl Compr Canc Netw 10:1236-1271, 2012 3. Alberts WM: Introduction to the third edition: Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:38S-40S, 2013 (suppl) 4. Azzoli CG, Temin S, Giaccone G: 2011 focused update of 2009 American Society of Clinical Oncology clinical practice guideline update on chemotherapy for stage IV non–small-cell lung cancer. J Oncol Pract 8:63-66, 2012
6. Pignon JP, Tribodet H, Scagliotti GV, et al: Lung adjuvant cisplatin evaluation: A pooled analysis by the LACE Collaborative Group. J Clin Oncol 26:3552-3559, 2008 7. Malin JL, O’Neill SM, Asch SM, et al: Quality of supportive care for patients with advanced cancer in a VA medical center. J Palliat Med 14:573-577, 2011 8. Ivers N, Jamtvedt G, Flottorp S, et al: Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 6:CD000259, 2012 9. Bell D, Layton AJ, Gabbay J: Use of a guideline based questionnaire to audit hospital care of acute asthma. BMJ 302:1440-1443, 1991 10. Sullivan N, Schoelles KM: Preventing in-facility pressure ulcers as a patient safety strategy: A systematic review. Ann Intern Med 158:410-416, 2013 11. Ang E, Chow YL: General pain assessment among patients with cancer in an acute care setting: A best practice implementation project. Int J Evid Based Healthc 8:90-96, 2010 12. Neuss MN, Malin JL, Chan S, et al: Measuring the improving quality of outpatient care in medical oncology practices in the United States. J Clin Oncol 31:1471-1477, 2013 13. Blayney DW, McNiff K, Hanauer D, et al: Implementation of the Quality Oncology Practice Initiative at a university comprehensive cancer center. J Clin Oncol 27:3802-3807, 2009 14. Tanvetyanon T, Corman M, Lee JH, et al: Quality of care in non–small-cell lung cancer: Findings from 11 oncology practices in Florida. J Oncol Pract 7:e25e31, 2011
Copyright © 2014 by American Society of Clinical Oncology
J U L Y 2014
16. National Comprehensive Cancer Network: NCCN Clinical Practice Guidelines in Oncology: Non-Small Cell Lung Cancer. http://www.nccn.org 17. American College of Chest Physicians: Diagnosis and Management of Lung Cancer Executive Summary: ACCP Evidence-Based Clinical Practice Guidelines (ed 2) http://www.chestnet.org/accp/guidelines 18. European Society for Medical Oncology: ESMO Clinical Practice Guidelines. http://www.esmo.org/Guidelines-Practice/Clinical-Practice-Guidelines 19. Cancer Care Ontario: Quality Guidelines and Standards. http://www. cancercare.on.ca/toolbox/qualityguidelines/ 20. European Lung Cancer Working Party: Guidelines: Non-Small Cell Lung Cancer—Metastatic (advanced) Stages (October 2006). http://www.elcwp.org/ New/En/index.html 21. Hurst D: Audit and feedback had small but potentially important improvements in professional practice. Evid Based Dent 14:8-9, 2013 22. Ryoo JJ, Ordin DL, Antonio AL, et al: Patient preference and contraindications in measuring quality of care: What do administrative data miss? J Clin Oncol 31:2716-2723, 2013 23. Goffin J, Lacchetti C, Ellis PM, et al: First-line systemic chemotherapy in the treatment of advanced non-small cell lung cancer: A systematic review. J Thorac Oncol 5:260-274, 2010 24. Komaki R, Cox JD, Stark R: Frequency of brain metastasis in adenocarcinoma and large cell carcinoma of the lung: Correlation with survival. Int J Radiat Oncol Biol Phys 9:1467-1470, 1983 25. Komaki R, Scott CB, Byhardt R, et al: Failure patterns by prognostic group determined by recursive partitioning analysis (RPA) of 1547 patients on four radiation therapy oncology group (RTOG) studies in inoperable nonsmall-cell lung cancer (NSCLC). Int J Radiat Oncol Biol Phys 42:263-267, 1998 26. Donabedian A: The quality of care. How can it be assessed? JAMA 260: 1743-1748, 1988 27. Porter ME: What is value in health care? N Engl J Med 363:2477-2481, 2010 28. Scagliotti G, Brodowicz T, Shepherd FA, et al: Treatment-by-histology interaction analyses in three phase III trials show superiority of pemetrexed in nonsquamous non-small cell lung cancer. J Thorac Oncol 6:64-70, 2011
•
jop.ascopubs.org
e253
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
5. Cuffe S, Booth CM, Peng Y, et al: Adjuvant chemotherapy for non-small-cell lung cancer in the elderly: A population-based study in Ontario, Canada. J Clin Oncol 30:1813-1821, 2012
15. Siegel EM, Jacobsen PB, Lee JH, et al: Florida Initiative for Quality Cancer Care: Improvements on colorectal cancer quality care indicators over a 3-year interval. J Am Coll Surg 218:16-25, e1-e4, 2014
Tanvetyanon et al
Appendix Table A1. Multivariable Analysis Adjusting for Histology, Insurance Status, and Practice Site P 2006 Adherence (%)
2009 Adherence (%)
Staging information documented within 1 month of first office visit
94.6
97.4
2.8
.02
For patients who underwent surgical resection of lung cancer, pathology report available in medical record
95.2
99.2
4
.09
For patients who received chemotherapy, flow sheet of chemotherapy and blood counts available
89.2
97.0
7.8
⬍ .001
⬍ .001
For patients who received chemotherapy, signed patient consent for chemotherapy available
69.5
76.3
6.8
.04
.12
Documentation of smoking status assessment within 1 month of first office visit
97.8
99.3
1.5
.07
.06
For patients who underwent surgical resection, surveillance CT scan performed within 6 months of surgery
63.4
77.6
14.2
.06
For patients who underwent surgical resection with negative margins, adjuvant radiation therapy not administered
91.2
97.8
6.6
.08
.04
For patients referred for concurrent chemoradiation, brain staging with CT or MRI performed
57.8
82.8
25.0
⬍ .001
.001
For patients with stage III or IV disease, performance status assessment documented
43.4
51.3
7.9
.03
⬍ .001
Indicator
Change (%)
Univariable
Adjusted
General quality of care .02
NSCLC specific NA*
Abbreviations: CT, computed tomography; MRI, magnetic resonance imaging; NA, not available; NSCLC, non–small-cell lung cancer. * Model does not converge because of sparseness of data.
e254
JOURNAL
OF
ONCOLOGY PRACTICE
•
V O L . 10, I S S U E 4
Copyright © 2014 by American Society of Clinical Oncology
Information downloaded from jop.ascopubs.org and provided by at QUEENS UNIVERSITY on March 7, 2015 from 130.15.241.167 Copyright © 2014 American Society of Clinical Oncology. All rights reserved.
NA*
